KR102717244B1 - Cloud-Multiple Edge Server Collaboration System and Method Based on Service Classification in Intelligent Video Security Environments - Google Patents

Abstract

A cloud-multi-edge server collaboration system and method based on service classification in an intelligent video security environment are disclosed. The system includes one or more terminals; a plurality of edge servers which receive, in an intelligent video security environment, a context of a service generated from the terminals and context information of a computing node through a wired/wireless communication network, and calculate a threshold for service classification, classify the service, transmit the service, and process the service through each module; and a cloud server linked with a reinforcement learning agent which determines an edge server as a collaboration target based on the context of the service received from each edge server and the context information of the computing node, and ensures that the determined collaboration target is serviced.

Description

Cloud-Multiple Edge Server Collaboration System and Method Based on Service Classification in Intelligent Video Security Environments

The present invention relates to a system and method for performing collaboration between cloud-multi-edge servers connected to reinforcement learning agents according to periodically shared services and edge server-related context-based service classification for intelligent image security.

The structure of the collaboration system consists of a cloud connected to a reinforcement learning agent, a number of edge servers, and terminals. The edge server extracts the service received from the connected terminal, the context of the service of the edge server, and the context of the computing node to classify the service. The context of the service includes the size and deadline of the service. The computing node includes a cloud or an edge server. The context of the computing node including the cloud or the edge server includes the computing resources of the computing node (CPU occupancy, RAM occupancy, storage usage), the available bandwidth between the computing nodes, and the number of services stored in the buffer of the computing node. The cloud selects a collaboration target (edge server) based on reinforcement learning using the context of the classified service and the context of the computing nodes. The computing node selected as the collaboration target reduces the service failure rate and shortens the service processing time compared to other computing nodes.

Cloud computing processes transaction data in a client/server manner with a centralized server architecture. Edge computing, on the other hand, is a computing method that contrasts cloud computing in that computing is performed at the edge of a network rather than in a cloud or centralized manner. Edge computing does not process data in a central server where the load of large amounts of data is concentrated, as in cloud computing or a centralized manner, but uses mobile devices and IoT devices themselves or edge servers located physically close to each other, as shown in Fig. 1, to perform data analysis and device operation.

Figure 1 is a conceptual diagram of intelligent edge computing.

In contrast to cloud computing, which centrally processes data, Mobile Edge Computing (MEC) technology collects, analyzes, and processes data individually around mobile devices through distributed edge servers. Recently, with the development of LTE 4G/5G mobile communication technology, drones, and IoT technology, it is developing into an intelligent Edge Computing technology that uses big data analysis using artificial intelligence and machine learning (ML).

For example, CCTV intelligent image processing requires intelligent edge computing technology that uses multiple edge servers distributed by region to process large amounts of image data and to overcome the overload of large amounts of image data stored in the central server.

As a related prior art1, “Distributed processing system between cloud and edge in IoT computing environment” is registered in patent registration number 10-2110592.

Distributed processing system between cloud and edge in IoT computing environment

A distributed processing system between the cloud and the edge in an IoT computing environment, including a plurality of edges that are connected to the object side, collect data, and send control signals to the object side, and a cloud that receives data from each edge, processes it, and transmits the processing results to the edge.

The above cloud includes a model generation unit that generates a model for processing the collected data, a model distribution rule for distributed processing between the cloud and the edge, and a model distribution unit that distributes the model to the edge according to the model distribution rule.

The above model distribution rule is determined based on at least one of the processing performance and memory capacity of each edge, and the type and number of models distributed from the cloud to each edge are different based on the determined distribution rule.

As a related prior art2, “Edge computing-based network monitoring method, device and system” is registered in patent registration number 10-2163280.

Edge computing-based network monitoring method, device and system are provided, wherein a second entity is included in a network including a first entity, a second entity, and a switching device provided between the first entity and the second entity. The second entity includes a transceiver for transmitting and receiving packets with the first entity, and the second entity performs edge computing by having a processor, or is connected to the switching device via an edge computing device having a processor,

The processor is configured to determine whether a security issue exists for a network associated with the first entity and the second entity based on at least some of the information contained in at least one packet received by or transmitted from the second entity.

Determining whether a security issue has occurred based on the above packet information is as follows:

It is determined that a security issue has occurred in the network based on at least one of a determination that the number of connections from a predetermined first source IP to a first destination IP exceeds a predetermined threshold, a determination that a request for a predetermined first URL exceeds a predetermined threshold, a determination that a BPS (Bit Per Second) of a predetermined first server exceeds a predetermined threshold, a determination that a PPS (Packet Per Second) from a predetermined second source IP exceeds a predetermined threshold, a determination that a number of synchronization signal (SYN) packets from a predetermined third source IP exceeds a predetermined threshold, and a determination that a predetermined fourth source IP simultaneously attempts to connect to a number of server IPs exceeding a predetermined threshold.

As a related prior art3, patent registration number 10-2304477 is registered for “Adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security.”

The adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security includes: a terminal equipped with a wired/wireless communication unit; a worker edge server which extracts and formalizes contexts of connected terminals and edge servers in real time and adaptively shares the contexts with a master edge server; and a master edge server which collects and manages contexts shared from the worker edge servers and performs autonomous collaboration between edge servers through terminal connection control.

The above adaptive context sharing is performed by classifying the characteristics of the context and determining the sharing time based on the classified context characteristics, and "adaptive sharing of the context" means that the process in which each Worker edge server transmits the terminal and edge server context it has or extracts to the Master edge server shares the context, and the characteristics of the context are characteristics that indicate whether the context is a core context to be considered when determining an edge server that requires autonomous collaboration and an edge server that will perform autonomous collaboration in autonomous collaboration between Worker edge servers, or a context that plays a supporting role, and the context characteristics are classified according to factors directly involved in autonomous collaboration-type linkage [computing resources (CPU clock, CPU occupancy, RAM occupancy)] and factors indirectly involved [quality (bit rate) of the video transmitted to the edge server, video encoding method, GPS location information of the terminal, storage space size of the edge server, number of connected terminals, IP address of the edge server].

Recently, with the development of network processing technology and smart terminals, the number of simultaneously connected terminals has increased and various types of services have appeared, and intelligent video security technology utilizing cloud-multi-edge servers has been attracting attention. In the case of existing VMS (Video Management System)-based video security technology, since many terminals transmit and process all services through the cloud such as VMS, there was a problem that the service processing efficiency was reduced due to the cloud resource load and delay.

Intelligent video security technology utilizing cloud-edge servers is designed to solve the problem of cloud-based video processing technology by reducing the load on the cloud in edge servers that are geographically close to the terminals, and performing collaboration by processing services that can be processed efficiently in edge servers and processing services that cannot be processed efficiently in the cloud.

In the intelligent video security environment proposed in the present invention, a cloud-multi-edge server collaboration system based on service classification is proposed to minimize service failure in an environment where multiple edge servers and terminals exist, and collaboration target decisions are made based on reinforcement learning to ensure the minimum processing time of the classified services. As a result, it is expected that the service real-timeness and efficiency in intelligent video security services and various cloud-multi-edge server collaboration system environments can be increased.

Patent registration number 10-2110592 (registration date May 7, 2020), "Method and system for distributed processing between cloud and edge in IoT computing environment", Electronics and Telecommunications Research Institute Patent registration number 10-2163280 (registration date September 29, 2020), "Edge computing-based network monitoring method, device and system", Macdata Co., Ltd. Patent registration number 10-2304477 (registration date September 14, 2021), "Adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security", Kwangwoon University Industry-Academic Cooperation Foundation

The purpose of the present invention to solve the above problems is to provide a cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, which performs collaboration between cloud-multi-edge servers connected to reinforcement learning agents according to service classification based on context-based services and edge servers periodically shared for intelligent video security.

Another object of the present invention is to provide a cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment.

In order to achieve the object of the present invention, a cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment includes one or more terminals; a plurality of edge servers that receive, through a wired/wireless communication network, the context of a service generated from the terminals and the context information of a computing node in an intelligent video security environment, and calculate thresholds for service classification, classify services, transmit services, and process services through each module; and a cloud server linked with a reinforcement learning agent that determines an edge server as a collaboration target based on the context of the service received from each edge server and the context information of the computing node, and provides a service to the determined collaboration target.
The above edge server
The edge server includes a threshold calculation module for calculating a threshold for service classification based on the context of the service transmitted from the terminal connected to the edge server (size of the service, deadline of the service), the context of the service stored in the buffer of the edge server, and the computing resources of the edge server; a service classification module for classifying the service received from the terminal into a service that satisfies the deadline and a service that does not satisfy the deadline based on the calculated threshold; a service transmission module for transmitting the context of the service requiring collaboration to the cloud and transmitting the service to a collaboration target (the edge server where collaboration is determined) determined by the reinforcement learning agent linked to the cloud server; and a service processing module for processing services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server.

In order to achieve another object of the present invention, a cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment comprises: (a) a step of receiving, in an intelligent video security environment, a service context generated from one or more terminals and context information of a computing node to a plurality of edge servers through a wired/wireless communication network, and calculating a threshold for service classification, classifying the service, transmitting the service, and processing the service through each module of the plurality of edge servers; and (b) a step of determining an edge server as a collaboration target based on the service context received from each edge server and the context information of the computing node by a reinforcement learning agent connected through a cloud server linked with the plurality of edge servers, so that the determined collaboration target is provided with a service.
The above edge server
The edge server includes a threshold calculation module for calculating a threshold for service classification based on the context of a service transmitted from a terminal connected to the edge server (size of the service, deadline of the service), the context of a service stored in a buffer of the edge server, and computing resources of the edge server; a service classification module for classifying a service received from a terminal into a service that satisfies a deadline and a service that does not satisfy a deadline based on the calculated threshold; a service transmission module for transmitting the context of a service requiring collaboration to the cloud server and transmitting the service to a collaboration target (the edge server where collaboration is determined) determined by a reinforcement learning agent linked to the cloud server; and a service processing module for processing services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server.

The cloud-multi-edge server collaboration system and method based on service classification in an intelligent video security environment of the present invention can reduce the failure rate of services through the cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, by allowing an edge server to classify services based on a threshold value, and by selecting an edge server to collaborate with based on the classified services, to minimize the processing time of the services, thereby minimizing the failure rate and processing time of the services. By minimizing the failure rate and processing time of the services between the dpt server and the terminal to collaborate with, the real-time nature of the intelligent video security service can be increased, and the efficiency of the service can be improved.

Intelligent video security technology utilizing cloud-edge servers solves the problem of cloud-based video processing technology by reducing the load on the cloud on edge servers that are geographically close to the terminal, and processes services that can be processed efficiently on edge servers, and processes services that cannot be processed efficiently on the cloud linked to reinforcement learning agents that determine collaboration, thereby performing collaboration.

The cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment proposed in the present invention is expected to increase the service real-timeness and efficiency in intelligent video security services and various cloud-multi-edge server collaboration system environments by allowing edge servers to classify services to minimize service failure and determine collaboration targets based on reinforcement learning agents to perform terminal connection to ensure the minimum processing time of the classified services in an autonomous collaborative edge computing environment where clouds and multiple edge servers and terminals connected with reinforcement learning agents exist.

This system can be used for intelligent video security services based on autonomous collaboration between cloud and multiple edge servers, autonomous collaboration platforms between cloud and edge servers for smart cities, collaboration platforms between cloud and edge servers that ensure real-time service availability, and command and control systems between automated environments and control centers.

Figure 1 is a conceptual diagram of intelligent edge computing.
FIG. 2 is a configuration diagram showing a system cluster structure and a cloud-multi-edge server autonomous collaboration environment based on service classification in an intelligent video security environment of the present invention.
FIG. 3 shows a framework of a service classification-based cloud-multi-edge server autonomous collaboration system in an intelligent video security environment according to the present invention.
Figure 4 is a graph of the threshold value change of the sigmoid function, calculating the threshold value on the edge server.
Figure 5 is an example diagram of task transmission according to service classification.
Figure 6 shows the learning structure of a reinforcement learning agent linked to the cloud.

Hereinafter, the configuration and operation of a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

The present invention is not limited to the disclosed embodiments, and can be implemented in various different forms by those of ordinary skill in the art. In the description of the present invention, if it is judged that a detailed description of a related known technology or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, when the attached drawing numbers indicate the same configuration, the same drawing numbers are given in different drawings.

This study is not limited to specific embodiments, but should be understood to include all transformations, equivalents, or substitutes included in the spirit and technical scope of the present invention.

FIG. 2 is a configuration diagram showing a system cluster structure and a cloud-multi-edge server autonomous collaboration environment based on service classification in an intelligent video security environment of the present invention.

FIG. 3 shows a framework of a service classification-based cloud-multi-edge server autonomous collaboration system in an intelligent video security environment according to the present invention.

In an intelligent video security environment (smartphone video, CCTV video, car black box video, drone camera video), an edge server collaboration system based on computing resource prediction is composed of a cloud server equipped with/connected with reinforcement learning agents, multiple edge servers, and multiple terminals. For intelligent video security, terminals, edge servers, and contexts are periodically shared, and autonomous collaboration between edge servers is performed based on computing resource prediction according to terminal connection control using the context of the shared service and the context of the computing node.

By classifying the context by elements involved in autonomous collaborative linkage, and determining the connection control between the edge server and the terminal that will perform autonomous collaboration with less computing resource usage than the threshold in a way that minimizes the distribution of computing resources of the entire edge server based on the classified context characteristics to achieve load balancing, the edge server and terminal for which collaboration has been decided by the reinforcement learning agent connected to the cloud server are connected via a wired or wireless communication network. The load on the computing resources (CPU occupancy, RAM occupancy, storage usage) of each edge server can be minimized, and load balancing can be provided to perform autonomous collaboration, thereby increasing the efficiency of intelligent video security services through real-time video analysis.

Computing resources of edge servers include CPU usage, RAM usage, and storage usage. Service context (information about the service) includes the size of the service and the service deadline.

Classification of services means categorizing services into those that satisfy deadlines and those that do not satisfy deadlines.

A reinforcement learning agent connected to a cloud server (100) receives the context of a service requiring collaboration and the context of a computing node from an edge server (200) to the cloud server (100), and the reinforcement learning agent connected to the cloud server (100) that receives the context determines a collaboration target (an edge server where collaboration is determined), and provides a service to the determined collaboration target.

In an intelligent video security environment, an edge server collaboration system based on computing resource prediction is composed of a cloud server (100) linked with a reinforcement learning agent, a plurality of edge servers (200), and terminals (300) equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit).

For example, the terminals (300) use any one of the terminals among a smartphone equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera), a CCTV camera equipped with a camera, a GPS receiver, and a wired or wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a drone equipped with a camera, a GPS receiver, and a wireless communication unit.

The present invention provides a system and method for performing collaboration between cloud and multiple edge servers according to periodically shared services and edge server-related context-based service classification for intelligent image security.

The structure of a system performing collaboration between cloud and multiple edge servers is composed of a cloud server (100) linked with a reinforcement learning agent, a plurality of edge servers (200), and terminals (300). The computing node includes the cloud server (100) or a plurality of edge servers (200). The edge server (200) extracts a service received from one or more connected terminals (300), a service context of the edge server, and a context of the computing node, and classifies the service (classifies it into a service that satisfies a deadline and a service that does not satisfies a deadline). The context of the service includes the size of the service and the deadline (service start time from - service completion time to). The computing node includes the cloud server (100) and a plurality of edge servers (200). The context of the computing node includes computing resources of the computing node (CPU occupancy, RAM occupancy, storage usage), available bandwidth between computing nodes, and the number of services stored in the buffer of the computing node. The cloud server (100) determines/selects a collaboration target (edge server) by using a reinforcement learning agent using the context of the classified service and the context of the computing nodes. The computing node selected as the collaboration target reduces service failure and shortens the service processing time compared to other computing nodes.

A cloud-multi-edge server autonomous collaboration system based on service classification for intelligent image security operates based on a cluster structure in which a cloud server (100) linked with a reinforcement learning agent and a plurality of edge servers (200) are connected.

Figure 2 shows a cloud-multi-edge server autonomous collaboration environment based on a system cluster structure and service classification defined in the present invention.

The terminal (300) is connected to the edge server (200), and the terminal (300) creates a service by the user. The terminal (300) transmits the created task to the edge server (200) because it lacks computing resources to process the service requiring low delay. The service transmitted by the terminal (300) includes the deadline of the service and information on the size of the service. The deadline of the service means the time required from the time the service is created until the processing is completed. If computing nodes such as the cloud server (100) and the edge server (200) do not satisfy the deadline of the service, the user's satisfaction decreases. The edge server (200) that receives the service classifies the service (classifies it into a service that satisfies the deadline and a service that does not satisfy the deadline) based on the computing resources of the edge server, the number of services in the buffer, the information of the buffer, the context of the service of the edge server, and the context of the computing node. The service is classified into a service that satisfies the deadline and a service that does not satisfy the deadline when processed by the edge server (200). Edge server (200) transmits context information of services for which service failure (non-meeting of service deadline) is predicted through service classification to cloud server (100), and a collaboration target (edge server) for service processing is selected by a reinforcement learning agent linked to cloud server (100). Cloud server (100) transmits classified services received from each edge server, service context information of edge server, and computing resource information of cloud server (100) to reinforcement learning agent, thereby selecting edge server (200) or cloud server (100) that has relatively free computing resources and can shorten service processing time. Information on selected collaboration target is transmitted to edge server (200) that received task and decided collaboration, and the edge server (200) transmits the service to collaboration target.

FIG. 3 shows a framework of a service classification-based cloud-multi-edge server autonomous collaboration system in an intelligent video security environment according to the present invention.

In the intelligent video security environment of the present invention, a cloud-multi-edge server collaboration system based on service classification is provided.

One or more terminals (300) in an intelligent video security environment;

A plurality of edge servers (200) that receive the context of the service generated from the terminal (300) and the context information of the computing node through a wired/wireless communication network, calculate thresholds for service classification, classify services, transmit services, and process services through each module; and

A cloud server (100) linked with a reinforcement learning agent that is linked with a plurality of edge servers and determines an edge server as a collaboration target based on the context of the service received from each edge server and the context information of the computing node, and provides the service to the determined collaboration target.
The above edge server (200)
The present invention comprises: a threshold calculation module of an edge server for calculating a threshold for service classification based on a context of a service transmitted from a terminal (300) connected to the edge server (200) (size of the service, deadline of the service), a context of a service stored in a buffer of the edge server, and computing resources of the edge server; a service classification module for classifying a service received from the terminal (300) into a service that satisfies a deadline and a service that does not satisfy a deadline based on the calculated threshold; a service transmission module for transmitting a context of a service requiring collaboration to the cloud server (100) and transmitting the service to a collaboration target (edge server where collaboration is determined) determined by the reinforcement learning agent linked to the cloud server (100); and a service processing module for processing services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server (100).

The edge server (200) includes a threshold calculation module, a service classification module, a service transmission module, and a service processing module.

In addition, the cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment of the present invention comprises: (a) a step of receiving, in an intelligent video security environment, a service context generated from one or more terminals (300) and context information of a computing node to a plurality of edge servers (200) through a wired/wireless communication network, and calculating a threshold for service classification, classifying the service, transmitting the service, and processing the service through each module of the plurality of edge servers (200); and (b) a step of determining an edge server as a collaboration target based on the service context received from each edge server and the context information of the computing node by a reinforcement learning agent connected through a cloud server (100) linked with the plurality of edge servers (200), so that the determined collaboration target is provided with a service.
The above edge server (200)
The present invention comprises: a threshold calculation module of an edge server for calculating a threshold for service classification based on the context of a service transmitted from a terminal (300) connected to the edge server (200) (size of the service, deadline of the service), the context of a service stored in a buffer of the edge server, and computing resources of the edge server; a service classification module for classifying a service received from the terminal (300) into a service that satisfies a deadline and a service that does not satisfy a deadline based on the calculated threshold; a service transmission module for transmitting the context of a service requiring collaboration to the cloud server (100) and transmitting the service to a collaboration target (edge server where collaboration is determined) determined by a reinforcement learning agent linked to the cloud server (100); and a service processing module for processing services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server (100).

A reinforcement learning agent linked to a cloud server (100) receives the context of a service requiring collaboration and the context of a computing node from an edge server (200) to the cloud server (100), and upon receiving this, the reinforcement learning agent linked to the cloud server (100) determines a collaboration target (the edge server where collaboration is determined) and provides a service to the determined collaboration target.

In an intelligent video security environment, an edge server collaboration system based on computing resource prediction is composed of a cloud server (100) linked with a reinforcement learning agent, a plurality of edge servers (200), and terminals (300) equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit).

For example, in an intelligent video security environment, the terminals (300) use any one of the terminals among a smartphone equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera), a CCTV camera equipped with a camera, a GPS receiver, and a wired or wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a drone equipped with a camera, a GPS receiver, and a wireless communication unit.

The above edge server (200) receives the context of a service generated from a terminal (300) and the context information of a computing node, and performs threshold calculation for service classification, service classification, service transmission, and service processing through each module of the edge server.

The threshold calculation module of the edge server (200) calculates a threshold for service classification based on the context of the service transmitted from the terminal (300) connected to the edge server (200) (size of the service, deadline of the service), the context of the service stored in the buffer of the edge server, and the computing resources of the edge server (200). The context of the service includes the size of the service and the deadline. The computing node includes a cloud server (100) or an edge server (200). The context of the computing node includes the computing resources of the computing node (CPU occupancy, RAM occupancy, storage usage), available bandwidth between computing nodes, and the number of services stored in the buffer of the computing node.

The service classification module classifies services received from the terminal (300) based on the computed threshold value based on the computing resources of the edge server (200), the number of services in the buffer, the information of the buffer, the context of the services in the edge server, and the context of the computing node (classifies services into services that satisfy the deadline and services that do not satisfy the deadline).

The service transmission module transmits the context of a service requiring collaboration to a cloud server (100), and transmits the service to a collaboration target (the edge server where collaboration is determined) determined by a reinforcement learning agent linked to the cloud server (100) that receives the context.

The service processing module processes services according to the decisions of the service classification module or reinforcement learning agent linked to the cloud server (100).

The cloud server (100) includes a context receiving module, a context transmitting module, and a service processing module.

The context receiving module receives the context of a service transmitted from an edge server (200), edge server computing resources, information on the buffer of the edge server (the number of services stored in the buffer, memory information of the buffer), and information on network bandwidth between computing nodes (cloud servers, edge servers). The context transfer module transfers information (the context of the service and the context of the computing node) received from the context receiving module to a reinforcement learning agent through the context transfer module.

The service processing module of the cloud server (100) processes services by transmitting services to the cloud server (100) and edge server (200) according to the collaboration target by a reinforcement learning agent among the services requiring collaboration.

The reinforcement learning agent determines a collaboration target (edge server) to process a service based on the context of the service received through the context transfer module and the context of the computing node according to the decision of the reinforcement learning agent. The information transmitted from the cloud server (100) to the reinforcement learning agent is used as an element of the 'state' of the reinforcement learning agent to shorten the service time and determine (act) a collaboration target to the edge server (200) that can reduce service failure. Services are transmitted to the cloud server (100) or the edge server (200) according to the collaboration target determined by the reinforcement learning agent, and the corresponding reward (service failure and service processing time) is measured. The reinforcement learning agent learns an optimal collaboration decision policy by using the 'next state' according to the 'state', 'action', and 'reward', and determines a collaboration target.

Figure 4 shows a graph of threshold changes of a sigmoid function according to threshold calculation for service classification, calculating a threshold value on an edge server.

는 번째 엣지 서버의 임계값,

는 번째 엣지 서버의 가용 컴퓨팅 리소스, 그리고 sigmoid 함수의 인자로 쓰이는

는 식을 통해 계산된다. The autonomous collaboration system between cloud and multi-edge servers based on service classification for intelligent image security performs service classification by setting a threshold based on the sigmoid function for efficient service processing. The threshold is Calculate through the formula.

Is The threshold of the th edge server,

Is The available computing resources of the th edge server, and the arguments of the sigmoid function.

Is It is calculated through the formula.

는 엣지 서버가 처리 중인 서비스의 수, j는 서비스의 index, Lj는 j번째 서비스의 크기,

는 번째 서비스의 deadline, k는 j번째 서비스가 시작될 때까지 버퍼에 저장된 서비스의 index,

는 엣지 서버가 처리 중인 작업의 남은 작업 크기, 그리고 N은 버퍼에 저장된 서비스의 집합을 나타낸다. Here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the jth service, k is the index of the service stored in the buffer until the jth service starts,

is the remaining work size of the work being processed by the edge server, and N represents the set of services stored in the buffer.

The difference between the processing time of tasks stored in the buffer of the edge server (200) and the deadline of the service is calculated. When the edge server (200) is processing a task, computing resources for tasks stored in the buffer are allocated to the service only when the task being processed is completed. Therefore, the threshold is calculated by considering the remaining processing time of the task being processed by the edge server (200). When a large number of tasks are transmitted to the edge server for a certain period of time and the load of the edge server increases and cannot satisfy the requirements of the service, the threshold approaches 0 and the computing node where the service will be processed is determined/selected by the reinforcement learning agent as another new collaboration target (edge server). In the opposite case, services whose threshold values are similar to the computing resource capacity of the edge server are processed by the edge server to which the terminal is connected.

Figure 5 shows an example of task transmission according to service classification on an edge server.

The above edge server (200) calculates a threshold based on the characteristics of the service, the number of services waiting in the buffer of the local edge server, and the computing resource base of the local edge server.

If the required resources of the services stored in the buffer are compared with the above threshold value and the required resources of the service are smaller than the threshold value, the service deadline can be met by processing at the local edge server, and the service is classified as a service that does not require collaboration. The service is stored in the buffer of the local edge server, and the edge server sequentially processes the services stored in the buffer.

If the service's required resources are greater than the above threshold, it is classified as a service requiring collaboration. The service does not remain in the local edge server buffer, but rather, the collaboration target to process the service is determined/selected based on the decision of the reinforcement learning agent located in the cloud (100).

Figure 6 shows the structure in which a reinforcement learning agent linked to the cloud performs learning.

는 서비스 실패 여부, 그리고

는 서비스 완료 시간을 나타낸다. '다음 상태'는 행동 이후에 나타나는 엣지 서버 및 클라우드 서버의 컴퓨팅 리소스, 엣지 서버의 버퍼에 저장된 작업, 서비스의 크기 및 deadline, 그리고 컴퓨팅 노드간 네트워크 대역폭이다. A reinforcement learning agent needs 'state', 'action', 'reward', and 'next state' for learning. 'State' is defined as edge server and cloud computing resources obtained from the autonomous collaboration environment between cloud and multiple edge servers, the number of services and tasks stored in the buffer of the edge server, the size of the service and the deadline of the service (service start time from - service completion time to), and the network bandwidth between computing nodes. 'Action' is defined as a computing node that is the target of collaboration. The computing node includes a cloud or multiple edge servers connected to the reinforcement learning agent. 'Reward' is defined as whether the task fails according to the 'action' and the processing time of the task. The reward is It is calculated as follows. r is the reward (calculated based on service failure and service processing time, which indicates how well the service collaboration was done), α is the weight of service failure, β is the weight of service processing time,

whether the service has failed, and

represents the service completion time. The 'next state' is the computing resources of the edge server and cloud server that appear after the action, the tasks stored in the buffer of the edge server, the size and deadline of the service, and the network bandwidth between the computing nodes.

The training of reinforcement learning agents is performed using the Actor-Critic algorithm. Actor-Critic consists of an actor network and a critic network.

In the autonomous collaboration edge computing environment between cloud and multiple edge servers, the actor network determines the importance (learning weight) of the 'state' information. The learning weight increases when the importance is higher than the reference value, and decreases when the importance is lower than the reference value. According to the learning weight, the actor network shows the collaboration probability for each computing node (cloud-multiple edge servers). The number of collaboration probabilities for each computing node shown is equal to the sum of the number of edge servers and the number of cloud servers. The computing node with the highest probability is sequentially selected among the various probabilities of each edge server, and the collaboration probability for each computing node changes as the learning weight increases and decreases. The collaboration probability of a computing node can process a service within the deadline, and increases as the service is processed quickly.

Critic network evaluates the value of 'state' (computing resource information of computing nodes of cloud and edge servers, network bandwidth between computing nodes, number of services stored in buffers of computing nodes). The value of 'state' is measured based on computing node resources (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of network between computing nodes, and number of services stored in buffers of computing nodes. In addition, the value of 'state' is higher when computing nodes can process many services within deadline and the processing time of services is shorter.

The actor network learns a collaborative decision policy to maximize the values evaluated by the critic network [resources of the compute node (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of the network between compute nodes, number of services stored in the buffer of the compute node], and the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the 'next state' and learns in the direction of reducing the difference between the two values. As a result, the actor network adjusts the learning weights to maximize the reward according to the state-action (indicating how well the collaboration was done - whether the service failed or not and the service processing time), and the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the next state and adjusts the learning weights in the direction of minimizing the difference between the two values.

The cloud-multi-edge server collaboration system and method based on service classification in an intelligent video security environment of the present invention classifies services based on a threshold value in order to lower the failure rate of services through a cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, and selects a collaboration target server through reinforcement learning to minimize the processing time of the service based on the classified services, thereby minimizing the failure rate and processing time of the service. By minimizing the failure rate and processing time of the service, the real-time nature of the intelligent video security service can be increased and the service efficiency can be improved.

Intelligent video security technology utilizing cloud-edge servers solves the problem of cloud-based video processing technology by reducing the load on the cloud in edge servers that are geographically close to the terminal, and processes services that can be processed efficiently in edge servers, and processes services that cannot be processed efficiently in cloud servers linked to reinforcement learning agents that determine collaboration, thereby performing collaboration.

The cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment proposed in the present invention is expected to increase the service real-timeness and efficiency in intelligent video security services and various cloud-multi-edge server collaboration system environments by allowing edge servers to classify services to minimize service failure and determine collaboration targets based on reinforcement learning agents to perform terminal connection to ensure the minimum processing time of the classified services in an autonomous collaborative edge computing environment where cloud servers connected to reinforcement learning agents and multiple edge servers and terminals exist.

This system can be used for intelligent video security services based on autonomous collaboration between cloud and multiple edge servers, autonomous collaboration platforms between cloud and edge servers for smart cities, collaboration platforms between cloud and edge servers that ensure real-time service availability, and command and control systems between automated environments and control centers.

Embodiments according to the present invention may be implemented in the form of program commands that can be executed through various computer means and may be recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, and data structures, either singly or in combination. The computer-readable recording medium may include a storage, a magnetic media such as a hard disk, a floppy disk, and a magnetic tape, an optical media such as a CD-ROM, a DVD, a magneto-optical media such as a floptical disk, and a hardware device configured to store and execute a program command in a storage medium such as a ROM, a RAM, a flash memory, a storage, etc. Examples of the program command may include those generated by a compiler, and high-level language codes that can be executed by a computer using an interpreter as well as machine language codes. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention can be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) in a form readable by using computer software.

Although the present invention has been described with reference to specific embodiments, the present invention is not limited to the same configuration and operation as the specific embodiments in order to illustrate the technical idea as described above, and may be implemented by various modifications without departing from the technical idea and scope of the present invention, and the scope of the present invention should be determined by the claims set forth below.

100: Cloud Server
200: Edge Server
300: Terminals (smartphones, CCTV cameras, drones, etc.)

Claims (17)

One or more terminals;
In an intelligent video security environment, the context of the service generated from the terminal and the context information of the computing node are received through a wired/wireless communication network, and a plurality of edge servers calculate thresholds for service classification, classify services, transmit services, and process services through each module; and
It includes a cloud server linked with a reinforcement learning agent that determines an edge server to be a collaboration target based on the context of the service received from each edge server and the context information of the computing node, and provides the service to the determined collaboration target.
The above edge server
A threshold calculation module of an edge server which calculates a threshold for service classification based on the context of a service transmitted from a terminal connected to the edge server (size of the service, deadline of the service), the context of a service stored in a buffer of the edge server, and the computing resources of the edge server; a service classification module which classifies a service received from a terminal into a service that satisfies a deadline and a service that does not satisfy a deadline based on the calculated threshold; a service transmission module which transmits the context of a service requiring collaboration to the cloud server and transmits the service to a collaboration target (the edge server where collaboration is determined) determined by the reinforcement learning agent linked to the cloud server; and a service processing module which processes services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server. A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment. In the first paragraph,
The above terminals use any one of the terminals among a smartphone equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera) in an intelligent video security environment, a CCTV camera equipped with a camera, a GPS receiver, and a wired or wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a drone equipped with a camera, a GPS receiver, and a wireless communication unit.
The service transmitted by the above terminal includes service deadline and service size information, and the service deadline means the time required from the time the service is created until the processing is completed. A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment. delete In the first paragraph,
A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, wherein the context of the service includes the size of the service and the service deadline, the computing node includes a cloud server or an edge server, and the context of the computing node includes computing resources of the computing node, available bandwidth between computing nodes, and the number of services stored in the buffer of the computing node, and the computing resources include CPU occupancy, RAM occupancy, and storage usage. In the first paragraph,
The above cloud server
A context receiving module that receives the context of a service transmitted from the edge server, edge server computing resource information, information on the number of services stored in the buffer of the edge server, and information on bandwidth between computing nodes (cloud server, edge server);
A context transmission module that transmits information (the context of the service and the context of the computing node) received from the context reception module to the reinforcement learning agent;
A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, including a service processing module of a cloud server that processes services among the services requiring the above collaboration, the collaboration target of which has been determined to be a cloud by the reinforcement learning agent. In paragraph 5,
The above reinforcement learning agent determines an edge server as a collaboration target based on the context of the service received through the context transfer module and the context of the computing node, and the information transmitted from the cloud server to the reinforcement learning agent is used as an element of the 'state' of the reinforcement learning agent to shorten the service time and determine (act) an edge server as a collaboration target that can reduce service failure, and transmits services to the cloud server or edge server determined according to the collaboration target determined by the reinforcement learning agent and measures the corresponding reward (service failure and service processing time), and the reinforcement learning agent learns a collaboration decision policy using the 'next state' according to the 'state', 'action', and 'reward' to determine a collaboration target. A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment. In the first paragraph,
In a service classification-based autonomous collaboration system between cloud and multi-edge servers for intelligent video security, a threshold is set based on the sigmoid function in the edge server to classify services for efficient service processing, and the threshold is Calculate through the formula,

Is The threshold of the th edge server,

Is The available computing resources of the th edge server, and the arguments of the sigmoid function.

Is It is calculated through the formula,
Here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the jth service, k is the index of the service stored in the buffer until the jth service starts,

is the remaining work size of the work being processed by the edge server, and N represents the set of services stored in the buffer.
The difference between the processing time of tasks stored in the buffer of the edge server and the service deadline is calculated, and if the edge server is processing a task, the tasks stored in the buffer are allocated to the service only when the processing task is completed, so the threshold is calculated by considering the remaining processing time of the tasks being processed by the edge server, and if a large number of tasks are transmitted to the edge server during a certain period of time and the load of the edge server increases and the service requirements cannot be satisfied, the threshold approaches 0 and the computing node where the service is to be processed is determined/selected as a new collaboration target by the reinforcement learning agent, and in the opposite case, services whose threshold values are similar to the computing resource capacity of the edge server are processed on the edge server to which the terminal is connected. A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment. In the first paragraph,
The above edge server calculates the threshold based on the characteristics of the service, the number of services waiting in the buffer of the local edge server, and the computing resources of the local edge server.
If the required resources of the services stored in the buffer are compared with the above threshold value and the required resources of the service are smaller than the threshold value, the service deadline can be met by processing on the local edge server, and the service is classified as a service that does not require collaboration. The service is stored in the buffer of the local edge server, and the edge server sequentially processes the services stored in the buffer.
A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, in which a service is classified as requiring collaboration when the service's required resources are greater than the above threshold value, and the service is not left in the buffer of the local edge server, but a collaboration target to process the service is determined/selected based on the decision of the reinforcement learning agent located in the cloud server. In the first paragraph,
The above reinforcement learning agent requires 'state', 'action', 'reward', and 'next state' for learning.
The above 'state' is defined as the computing resources of edge servers and cloud servers obtained from the autonomous collaboration environment between cloud and multi-edge servers, the number of services and tasks stored in the buffer of the edge server, the size of the service and the deadline of the service, and the network bandwidth between computing nodes.
The above 'action' is defined as a computing node that is the target of collaboration, and the computing node includes a cloud server or multiple edge servers connected to the reinforcement learning agent.
The above 'reward' is defined as the failure of the task according to the 'action' and the processing time of the task, and the above reward is It is calculated as follows,
Here, r is the reward (calculated based on service failure and service processing time, which indicates how well the service collaboration was done), α is the weight of service failure, β is the weight of service processing time,

whether the service has failed, and

indicates the service completion time,
'Next state' is a cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment, including computing resources of edge servers and cloud servers that appear after the action, tasks stored in the buffer of the edge server, the size and deadline of the service, and the network bandwidth between computing nodes. In Article 9,
The learning of the above reinforcement learning agent is performed using the Actor-Critic algorithm, and the Actor-Critic consists of an actor network and a critic network.
In an autonomous collaborative edge computing environment between cloud and multiple edge servers, the actor network determines the importance (learning weight) of 'state' information, and the weight of the learning weight increases as the importance is higher than the reference value, and the weight decreases as the importance is lower than the reference value, and according to the learning weight, the actor network shows the collaboration probability for each computing node (cloud-multiple edge servers), and the number of collaboration probabilities of each computing node shown is equal to the sum of the number of edge servers and the number of cloud servers, and the computing node with the highest probability is sequentially selected among the various probabilities of each edge server as the collaboration target, and the collaboration probability for each computing node changes as the learning weight increases and decreases, and the collaboration probability of the computing node can process the service within the deadline and increases as the service is processed quickly.
The above Critic network evaluates the value of 'state' (computing resource information of computing nodes of cloud servers and edge servers, network bandwidth between computing nodes, number of services stored in buffers of computing nodes), and the value of 'state' is measured based on computing node resources (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of network between computing nodes, number of services stored in buffers of computing nodes, and also, the value of 'state' is higher when computing nodes can process many services within deadline and the processing time of services is shorter.
The above actor network learns a collaborative decision policy so that the values evaluated by the critic network [resources of the computing node (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of the network between computing nodes, number of services stored in the buffer of the computing node] are maximized, and the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the 'next state' and learns in the direction of reducing the difference between the two values, and as a result, the actor network adjusts the learning weights so that the reward according to the state-action (indicating how well the collaboration was done - whether the service failed or not and the service processing time) is maximized, and the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the next state and adjusts the learning weights in the direction of minimizing the difference between the two values. A cloud-multi-edge server collaboration system based on service classification in an intelligent video security environment. (a) In an intelligent video security environment, a step of receiving context information of a service generated from one or more terminals and context information of a computing node through a wired/wireless communication network to a plurality of edge servers, calculating a threshold for service classification, classifying the service, transmitting the service, and processing the service through each module of the plurality of edge servers; and
(b) a step of determining an edge server as a collaboration target based on the context of the service received from each edge server and the context information of the computing node by a reinforcement learning agent connected through a cloud server linked to the plurality of edge servers, and providing the service to the determined collaboration target;
The above edge server
A method for collaboration between cloud-multi-edge servers based on service classification in an intelligent video security environment, comprising: a threshold calculation module of an edge server for calculating a threshold for service classification based on the context of a service transmitted from a terminal connected to the edge server (size of the service, deadline of the service), the context of a service stored in a buffer of the edge server, and computing resources of the edge server; a service classification module for classifying a service received from a terminal into a service that satisfies a deadline and a service that does not satisfy a deadline based on the calculated threshold; a service transmission module for transmitting the context of a service requiring collaboration to the cloud server and transmitting the service to a collaboration target (the edge server for which collaboration has been determined) determined by the reinforcement learning agent linked to the cloud server; and a service processing module for processing services according to a collaboration decision of the service classification module or the reinforcement learning agent linked to the cloud server. In Article 11,
The above terminals use any one of the following terminals: a smartphone equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera); a CCTV camera equipped with a camera, a GPS receiver, and a wired or wireless communication unit; a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit; and a drone equipped with a camera, a GPS receiver, and a wireless communication unit.
A service transmitted by the above terminal includes service deadline and service size information, and the service deadline means the time required from the time the service is created until processing is completed. A cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment. delete In Article 11,
A cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment, wherein the context of the service includes the size of the service and the service deadline, the computing node includes a cloud server or an edge server, and the context of the computing node includes computing resources of the computing node, available bandwidth between computing nodes, and the number of services stored in the buffer of the computing node, and the computing resources include CPU occupancy, RAM occupancy, and storage usage. In Article 11,
The above cloud server
A context receiving module that receives the context of a service transmitted from the edge server, edge server computing resource information, information on the number of services stored in the buffer of the edge server, and information on bandwidth between computing nodes (cloud server, edge server);
A context transmission module that transmits information (the context of the service and the context of the computing node) received from the context reception module to the reinforcement learning agent;
A cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment including a service processing module of a cloud server that processes services among the services requiring the above collaboration, the collaboration target of which has been determined to be a cloud by the reinforcement learning agent. In Article 15,
The above reinforcement learning agent determines an edge server as a collaboration target based on the context of the service received through the context transfer module and the context of the computing node, and the information transmitted from the cloud server to the reinforcement learning agent is used as an element of the 'state' of the reinforcement learning agent to shorten the service time and determine (act) an edge server as a collaboration target that can reduce service failure, and transmits services to the cloud server or edge server determined according to the collaboration target determined by the reinforcement learning agent and measures the corresponding reward (service failure and service processing time), and the reinforcement learning agent learns a collaboration decision policy using the 'next state' according to the 'state', 'action', and 'reward' to determine a collaboration target. A cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment. In Article 11,
In a service classification-based autonomous collaboration system between cloud and multi-edge servers for intelligent video security, a threshold is set based on the sigmoid function in the edge server to classify services for efficient service processing, and the threshold is Calculate through the formula,

Is The threshold of the th edge server,

Is The available computing resources of the th edge server, and the arguments of the sigmoid function.

Is It is calculated through the formula,
Here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the jth service, k is the index of the service stored in the buffer until the jth service starts,

is the remaining work size of the work being processed by the edge server, and N represents the set of services stored in the buffer.
The difference between the processing time of tasks stored in the buffer of the edge server and the service deadline is calculated, and if the edge server is processing a task, the tasks stored in the buffer are allocated to the service only when the processing task is completed, so the threshold is calculated by considering the remaining processing time of the tasks being processed by the edge server, and if a large number of tasks are transmitted to the edge server during a certain period of time and the load of the edge server increases and the service requirements cannot be satisfied, the threshold approaches 0 and the computing node where the service is to be processed is determined/selected as a new collaboration target by the reinforcement learning agent, and in the opposite case, services whose threshold values are similar to the computing resource capacity of the edge server are processed on the edge server to which the terminal is connected. A cloud-multi-edge server collaboration method based on service classification in an intelligent video security environment.