KR20210041337A - Control system and control device for patrol and moving out, and operation method thereof - Google Patents

Abstract

The present invention relates to the operation of a drone. A patrol dispatch control system comprises: multiple drones; a patrol dispatch control device performing patrol dispatch control of the multiple drones; and a remote operator device receiving and monitoring images acquired by the multiple drones from the patrol dispatch control device. The patrol dispatch control device may control to dispatch at least one drone among the multiple drones to a designated area as a primary drone, select at least one drone among the remaining drones other than the primary drone as a secondary drone when the occurrence of an abnormal condition is detected in the designated area, and dispatch the secondary drone to the abnormal state occurrence area based on location information on the abnormal state occurrence area obtained from the primary drone.

Description

Patrol dispatch control system, control device, and operation method thereof {Control system and control device for patrol and moving out, and operation method thereof}

The present invention relates to a patrol dispatch operation, and more particularly, a patrol dispatch control system and control that enable more efficient and appropriate patrol operation of an unmanned aerial vehicle performing communication based on a mobile communication network (LTE/5G). It relates to an apparatus and a method of operating the same.

In the case of a conventional patrol dispatch system, a patrol officer patrols a designated area for a specified time using a car or on foot. Accordingly, in the related art, the patrol function was influenced by maintenance of patrol ability or concentration of the patrol officer.

Meanwhile, in recent years, drones are being used to dispatch patrols. However, the current patrol dispatch function using drones has various problems. For example, one of the problems with existing drones is that the drone's patrol time is short. Existing drones have a short flight time of around 20 to 30 minutes due to battery limitations. In order to overcome this problem, various attempts have been made to apply hydrogen fuel, solar cells, and the like to batteries, but they are still insufficient to be used universally.

Since the operation time of the drone is short, the following problems may occur when applied to patrol dispatch work. First, the mobilization drone will return from a situation where the battery is below the standard value, but it is difficult to properly transmit information on the area where the mobilization drone has detected an abnormality to the next drone, and an abnormal condition occurs verbally or over the phone between drone coordinators. The information about it is bound to be delivered. At this time, since the subjective position of each drone coordinator is considered for the information on the occurrence of an abnormal condition, which is transmitted verbally or over the phone, there is a problem that it is difficult to accurately transmit the information, and the abnormal condition occurrence information is transmitted to the next drone. There is a problem that it is difficult to maintain the real-time quality of information delivery because it is delivered to the coordinator.

In addition, since existing drones perform 1:1 adjustment using GCS (Ground Control Station, ground control station) within the range of sight of the drone operator based on the RF communication channel, and monitor the images acquired by the drone in real time, There is a problem that it is difficult to control and operate multiple drones simultaneously. In addition, since the conventional drone operation in patrol mobilization cannot utilize intelligent image analysis (AI) for anomaly detection, and the drone operator must continuously perform monitoring, it is inevitable to rely on the concentration of the drone coordinator or subjective judgment. There was a problem in which it was difficult to determine the occurrence of an exact abnormal state.

In order to solve the above-described problem, the present invention provides a patrol dispatch control system, a control device, and a method of operation thereof that automatically provide patrol dispatch using a drone and a drone control system using a mobile communication network, and intelligent image analysis (AI). It is in the offering.

However, the present invention is not limited to the above object, and other objects not mentioned will be clearly understood from the following description.

The patrol dispatch control system of the present invention for achieving the above-described object includes a plurality of drones capable of obtaining images for a designated area, a patrol dispatch control device that performs patrol dispatch control of the plurality of drones, and the patrol dispatch. A remote operator device receiving and monitoring the images acquired by the plurality of drones from a control device, wherein the patrol dispatch control device dispatches at least one drone of the plurality of drones to the designated area as a primary drone, When the occurrence of an abnormal condition is detected in the designated area, at least one of the drones other than the primary drone is selected as a secondary drone, and based on the location information of the abnormal condition occurrence area obtained from the primary drone As a result, it is possible to control to dispatch the second drone to an area where the abnormal condition occurs.

The patrol dispatch control apparatus of the present invention includes a communication interface forming a communication channel with at least one drone among a plurality of drones, a processor functionally connected to the communication interface, and the processor includes at least one drone among the plurality of drones. When an abnormal condition is detected in the designated area based on the image analysis acquired from the primary drone by selecting it as the primary drone and controlling to dispatch to the designated area, at least one of the remaining drones except the primary drone A drone of is selected as a secondary drone, and the second drone can be controlled to be dispatched to an area where the abnormal condition has occurred based on the location information of the abnormal condition occurring area acquired from the primary drone.

Here, the processor may adjust the number of the secondary drones according to the type of occurrence of the abnormal state.

Alternatively, the processor may adjust the number of the secondary drones according to the size of the abnormal state.

Alternatively, the processor may transmit an alarm message according to the occurrence of the abnormal condition to the remote operator device, and determine the number of the secondary drones according to the response of the remote operator device.

On the other hand, the processor may control the on-site dispatch request of the dispatch device based on the image analysis obtained from the secondary drone.

In particular, the processor may control to transmit the abnormal state occurrence location information and the image acquired from the secondary drone to the dispatch device.

In addition, the processor is a flight to reduce the distance between the secondary drone and the location where the abnormal state occurs so that the resolution of the image related to the abnormal state of the secondary drone is higher than the resolution of the image related to the abnormal state of the primary drone. Control can be performed.

In addition, the processor calculates a time for the secondary drone to arrive at the abnormal state occurrence region before the primary drone returns from the abnormal state occurrence region, and based on the calculated time, the second drone You can decide when to go.

In the patrol dispatch control method of the present invention, the processor of the patrol dispatch control device selects at least one drone among a plurality of drones as the primary drone and dispatches them to a designated area, based on image analysis obtained from the primary drone. Detecting the occurrence of an abnormal condition in the designated area, when the occurrence of the abnormal condition is detected, selecting at least one of the drones other than the primary drone as a secondary drone, obtained from the primary drone It may include the step of dispatching the second drone to the area where the abnormal condition has occurred based on the location information of the abnormal condition occurrence area.

Here, the step of dispatching the secondary drone includes calculating a time for the secondary drone to arrive at the abnormal state occurrence area before the primary drone starts to return, and the second based on the calculated time. It may include the step of dispatching the car drone.

According to the present invention, the present invention supports automatic detection of anomalies through intelligent image analysis even if the drone control operator is not monitoring.

In addition, the present invention supports the patrolling of the following drones to the anomaly detection location of the following drones in real time by appropriately transmitting the location at which the preceding drone has detected an anomaly to the following drones through drone control.

In addition, the present invention enables the following drones to be operated as swarm drones, so that various missions such as monitoring for an abnormal state occurrence area, speaker broadcasting, and emergency equipment dropping can be efficiently distributed and supported.

In addition, the present invention can minimize damage to major facilities, such as early monitoring and response, through a trailing drone for a response to an abnormal state occurrence.

In addition, various effects other than the above-described effects may be directly or implicitly disclosed in the detailed description according to an embodiment of the present invention to be described later.

1 is a view showing an example of a patrol dispatch operation system according to an embodiment of the present invention.
2 is a view showing an example of a signal flow related to the operation of the patrol dispatch operation system according to an embodiment of the present invention.
3 is a view showing an example of a patrol dispatch control device according to an embodiment of the present invention.
4A is a diagram illustrating an example of a remote operator device according to an embodiment of the present invention.
4B is a diagram illustrating an example of a screen interface of a remote operator device according to an embodiment of the present invention.
5 is a diagram showing an example of a method of operating a patrol dispatch control device according to an embodiment of the present invention.
6 is a view showing another example of a method of operating a patrol dispatch control device according to an embodiment of the present invention.

In order to clarify the features and advantages of the problem solving means of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention shown in the accompanying drawings.

However, in the following description and the accompanying drawings, detailed descriptions of known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that the same components are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the following description and drawings should not be interpreted as being limited to their usual or dictionary meanings, and the inventor may appropriately define the concept of terms for describing his or her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all the technical ideas of the present invention. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers such as first and second are used to describe various elements, and are used only for the purpose of distinguishing one element from other elements, and to limit the elements. Not used. For example, without departing from the scope of the present invention, the second element may be referred to as the first element, and similarly, the first element may be referred to as the second element.

In addition, terms used in the present specification are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, terms such as "comprises" or "have" described herein are intended to designate the existence of features, numbers, steps, actions, components, parts, or a combination thereof described in the specification. It is to be understood that the above other features or the possibility of the presence or addition of numbers, steps, actions, components, parts, or combinations thereof are not preliminarily excluded.

In addition, terms such as "unit", "group", and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software. In addition, "a or an", "one", "the" and similar related words are different from the present specification in the context of describing the present invention (especially in the context of the following claims). Unless indicated or clearly contradicted by context, it may be used in the sense of including both the singular and the plural.

In addition to the above terms, specific terms used in the following description are provided to aid understanding of the present invention, and the use of these specific terms may be changed in other forms without departing from the technical spirit of the present invention.

In addition, embodiments within the scope of the present invention include computer-readable media having or transferring computer-executable instructions or data structures stored in the computer-readable media. Such computer-readable media may be any available media accessible by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions, or data structures. It may include, but is not limited to, physical storage media such as any other media that can be used to store or deliver certain program code means of the computer and can be accessed by a general purpose or special purpose computer system. .

1 is a view showing an example of a patrol dispatch operation system according to an embodiment of the present invention.

Referring to FIG. 1, a patrol dispatch operation system 10 according to an embodiment includes a plurality of drones 100 and 500, a base station 150, a patrol dispatch control device 200, a remote operator device 300, and a dispatch Device 400 may be included.

The plurality of drones 100 and 500 may include an LTE/5G modem, a drone camera, and a flight driving module. The plurality of drones 100 and 500 may be divided into a primary drone 100 and a secondary drone 500 according to an order of dispatch. The plurality of drones 100 and 500 may form a communication channel with the patrol dispatch control device 200 through the base station 150 and receive patrol information from the patrol dispatch control device 200. The patrol information may include at least one of locations of an area to be patrolled by the plurality of drones 100 and 500, a patrol order of location area locations, patrol time of local locations, patrol altitude of local locations, and patrol speed. I can. The patrol information may be adjusted according to the performance of the plurality of drones 100 and 500 (eg, flight time, flight speed, flight altitude, etc.).

The plurality of drones 100 and 500 may all be composed of the same drones. Alternatively, the plurality of drones 100 and 500 may have a relatively lower performance than at least some of the drones that patrol in the first order, or have a relatively lower performance than at least some of the drones that patrol in the first order, depending on the policy. At least some components of the drone may have a higher performance than at least some components of the second drone. For example, a primary drone has a relatively high performance flight drive module so that it can patrol for a longer time than a secondary drone, but may include a light camera with low performance. Patrol information applied to the plurality of drones 100 and 500 may vary according to the order of dispatch. For example, the patrol altitude and patrol speed of the primary drone may be set higher than the patrol altitude and speed of the secondary drone. Alternatively, when the available flight time of the primary drone is longer than the available flight time of the secondary drone, the patrol time of the primary drone may be set longer than the patrol time of the secondary drone. The plurality of drones 100 and 500 may perform a patrol according to the patrol information, and transmit a patrol image acquired during the patrol to the patrol dispatch control device 200 through the base station 150.

The base station 150 may support an LTE/5G communication method. The base station 150 may support formation of a communication channel between at least some of the plurality of drones 100 and 500 and the patrol dispatch control device 200.

The patrol dispatch control device 200 may control at least one drone through the base station 150. The patrol dispatch control device 200 may generate at least one patrol information and provide the patrol information to at least one of the plurality of drones 100 and 500. The patrol dispatch control device 200 may control at least one of the plurality of drones 100 and 500 to patrol dispatch as a primary drone. The patrol dispatch control device 200 may receive an image from the primary drone and perform dispatch control of the secondary drone based on the received image analysis. The patrol dispatch control device 200 may control the occurrence of an abnormal condition based on the image collected through the secondary drone, for example, to control an on-site dispatch. In addition, the patrol dispatch control device 200 may control the third drone dispatch including items related to field support. The patrol dispatch control device 200 may include a flight control server 210, an image analysis server 230, and an image transmission server 220.

The flight control server 210 may interlock with at least one of the plurality of drones 100 and 500 to give each drone a mission including a flight area, a flight point, and a camera operation. have. In addition, the flight control server 210 may monitor the flight status of the drone. In this regard, the plurality of drones 100 and 500 may transmit their location information and attitude information to the flight control server 210 in real time.

The image transmission server 220 may collect an image from a drone that has been dispatched among a plurality of drones 100 and 500, and may process the storage and delivery of the collected image. For example, the image transmission server 220 may receive an FHD/4K-class image from a drone patrolling and transmit the received image to the remote operator device 300 in real time. As another example, the image transmission server 220 receives a relatively low resolution image from the primary drone, stores it, and transmits it to the remote operator device 300, and receives a relatively high resolution image from the secondary drone. It may also be stored and transmitted to the remote operator device 300. The image provided by the primary drone may include an image captured at a relatively high altitude compared to a secondary drone, or an image captured at a state in which the speed of the primary drone is relatively higher than the speed of the secondary drone.

The image analysis server 230 may receive an image of a drone in real time, detect a person, a vehicle, a fire, etc., and transmit the result to the flight control server 210. For example, the image analysis server 230 may analyze whether there is a person or vehicle in a designated area or whether a fire has occurred through filtering, and output a corresponding result. In this operation, the image analysis server 230 may operate a filter corresponding to a person or vehicle object, a filter corresponding to a pattern of smoke or fire detected when a fire occurs, and perform deep learning related thereto. The image analysis server 230 may provide analysis results of people, vehicles, and fires to the remote operator device 300.

The remote operator device 300 may include an operator device that controls drone control in an integrated control room. The remote operator device 300 forms a communication channel with the patrol dispatch control device 200, and includes information on patrol control of the patrol dispatch control device 200, information on a patrol result, information on occurrence of an abnormal condition, and abnormality. It is possible to remotely receive processing information about the occurrence of a state. The remote operator device 300 may output the received information according to a specified method and, when receiving a specific input from the operator, may transmit a control message corresponding to the corresponding input to the patrol dispatch control device 200.

The dispatch device 400 may include a device capable of moving field personnel to a site where an abnormal condition has occurred or a site where direct confirmation is required. For example, the dispatch device 400 may establish a communication channel with a boarding means capable of boarding at least one field agent, a moving means capable of moving a boarding field agent, and the patrol dispatch control device 200. It may include a patrol vehicle device and the like including a communication means. The dispatch device 400 receives and outputs information capable of monitoring the arrangement and flight of a plurality of drones 100 and 500 from the patrol dispatch control device 200 to support field personnel to check the information. I can.

2 is a view showing an example of a signal flow related to the operation of the patrol dispatch operation system according to an embodiment of the present invention.

1 and 2, in step 101, the patrol dispatch control device 200 may control the first drone dispatch. In this operation, the flight control server 210 of the patrol dispatch control device 200 may assign a designated mission to the primary drone. The mission may include an area in which the primary drone will be dispatched, a patrol route within the area, a patrol altitude, a patrol speed, and the like.

In step 102, when the first drone that has been dispatched to patrol arrives at an area assigned as a mission, it activates the camera and can shoot an image at a specified altitude and at a specified speed. The primary drone may transmit the captured image to the patrol dispatch control device 200. In this operation, the primary drone may provide drone operation information to the patrol dispatch control device 200 together.

In step 103, the patrol dispatch control device 200 may provide the operation information of the primary drone to the remote operator device 300. The remote operator can monitor the primary drone operation information and check whether the primary drone performs patrol activities according to the assigned mission.

Meanwhile, in step 104, the image transmission server 220 of the patrol dispatch control device 200 may transmit an image analysis request for an image received from the primary drone to the image analysis server 230. In step 105, the image analysis server 230 may access the image transmission server 220 according to the request of the image transmission server 220 and perform analysis on the image requested by the image transmission server 220 for analysis. In step 106, the image analysis server 230 may transmit the image analysis result to the flight control server 210. The image analysis result may include time, object, location coordinates, object type, thumbnail information, and the like.

In step 107, the flight control server 210 may transmit an alarm message for detecting an abnormal object to the remote operator device 300. In this operation, the alarm message transmitted to the remote operator device 300 may include object display information, time information, latitude/longitude information, and the like. When an abnormal condition occurs, in step 108, the remote operator device 300 may request the patrol dispatch control device 200 to dispatch a second drone automatically or according to an operator input.

In step 109, the flight control server 210 of the patrol dispatch control device 200 may control the second drone dispatch. In this operation, the flight control server 210 may provide at least some of the alarm messages received from the primary drone to the mission information of the secondary drone. For example, the flight control server 210 may provide latitude/longitude information and object display information in which an abnormal condition occurs among the alarm messages to the secondary drone. Regarding the timing of the second drone dispatch, the flight control server 210 calculates a time for the second drone to arrive at the abnormal state occurrence area before the first drone starts to return, and based on the calculated time. It is possible to control the dispatch of the second drone. Here, the flight control server 210 may consider the remaining flight time of the primary drone. The remaining flight time may be calculated based on the remaining battery capacity of the primary drone, or may be calculated by subtracting the flight time to the present from the maximum flight time set for the primary drone.

For example, if the remaining flight time of the primary drone is 20 minutes and the return time is 5 minutes, the primary drone can stay in an abnormal condition for 15 minutes. The flight control server 210 may control the secondary drone to arrive before the primary drone leaves a corresponding abnormal state occurrence area in consideration of the movement speed of the secondary drone.

For example, if the secondary drone has the same speed as the primary drone and it takes 5 minutes to move to an abnormal condition, the flight control server 210 determines the time when the remaining flight time of the primary drone remains 20 minutes. As a standard, you can control the dispatch of the second drone after 10 minutes. The abnormal state occurrence area can be set based on the location of the primary drone. Through the above-described control, the flight control server 210 may secure an image of the abnormal state occurrence region by arriving the second drone at the abnormal state occurrence region before the primary drone returns from the abnormal state occurrence region.

The secondary drone may move to an abnormal state occurrence area based on the latitude/longitude information. In this operation, the flight control server 210 may dispatch a plurality of drones as secondary drones. For example, the flight control server 210 may dispatch a certain number of secondary drones according to the request of the remote operator device 300. Alternatively, the flight control server 210 may adjust the number of drones according to the type of abnormal state. For example, when an intrusion occurs in a certain area, the flight control server 210 dispatches two basic drones (one drone photographs the intrusion path, and the other drone tracks the intrusion personnel), Additional drones can be dispatched in response to the number of invading personnel. When a fire occurs, the flight control server 210 is basically 5 (4 drones shoot the fire scene from 4 defenses, and one drone shoots the fire scene at a specified altitude based on the center of the fire area) Dispatch drones, but additional drones can be dispatched depending on the size of the fire. When a vehicle accident occurs, the flight control server 210 dispatches three basic drones, but may further dispatch additional drones according to the size of the vehicle accident. When a plurality of drones are dispatched as secondary drones, the flight control server 210 may automatically set the flight path so that the flight paths do not overlap so that the drones do not collide around the abnormal state occurrence point.

In step 110, the secondary drone performs camera/gimbal control and flying aircraft control according to the control of the flight control server 210, and captures a more precise image at a relatively low altitude compared to the primary drone, and captures The recorded image may be provided to the patrol dispatch control device 200. For example, according to the control of the flight control server 210, the secondary drone detects objects related to abnormal conditions such as people, vehicles, and fires, and more accurately through zooming in/out and horizontal movement of the camera through a camera/gimbal. Object tracking and detection can be performed and, if necessary, the flying aircraft can be moved to a specific location. After the second drone image acquired in step 110 is analyzed by the image analysis server 230 of the patrol dispatch control device 200, the analysis result is transmitted to the remote operator device 300, or a separate image analysis process Without performing, it can be transmitted to the remote operator device 300.

In step 111, the remote operator device 300 may output a corresponding image to the display for monitoring the secondary drone image. In addition, the remote operator device 300 may output information on the secondary drones on the display so as to monitor flight status information on the secondary drones.

In step 112, when the user of the remote operator device 300 needs to be dispatched to the site, the user of the remote operator device 300 may request the dispatching device 400 to be dispatched to the site. Accordingly, the dispatch device 400 may perform field dispatch. In this process, the remote operator device 300 may provide a first drone image, a second drone image, an image analysis result, and location information to the dispatch device 400. The dispatch device 400 may support abnormal status check by sharing the second drone image in real time while moving based on the first drone image and the image analysis result and location information.

3 is a view showing an example of a patrol dispatch control device according to an embodiment of the present invention.

Referring to FIG. 3, the patrol dispatch control apparatus 200 according to an embodiment may include a first communication interface 201, a first processor 260, and a first memory 240.

The first communication interface 201 may support a communication function of the patrol dispatch control device 200. The first communication interface 201 may form a communication channel with at least one of the plurality of drones 100 and 500. In this regard, the first communication interface 201 may support an LTE/5G communication method. The first communication interface 201 may form a communication channel with the primary drone in response to the control of the first processor 260 and transfer mission information to the primary drone. The first communication interface 201 may receive an image transmitted by the primary drone and transmit it to the first processor 260. The first communication interface 201 may form a communication channel with the secondary drone in response to the control of the first processor 260 and provide at least some information of an alarm message to the secondary drone. The first communication interface 201 may establish a communication channel with the remote operator device 300. The first communication interface 201 may provide an image analysis result of the first drone image to the remote operator device 300 and receive a request for a second drone dispatch from the remote operator device 300. The first communication interface 201 may transmit the image acquired by the secondary drone to the remote operator device 300 in response to the control of the first processor 260. The first communication interface 201 may establish a communication channel with the dispatch device 400. The first communication interface 201 may transmit at least some of the first drone image, the image analysis result, the abnormal state occurrence location information, and the second drone image to the dispatch device 400 in response to the control of the first processor 260. .

The first processor 260 may transmit and process signals related to the operation of the patrol dispatch control device 200. In this regard, the first processor 260 may include a flight control unit 261, an image collection unit 263, an image analysis unit 265, and a field dispatch support unit 267. At least one of the flight control unit 261, the image collection unit 263, the image analysis unit 265, and the field dispatch support unit 267 is implemented as at least one hardware processor or configured as a software module, and then loaded into a memory. And can be accessed by the first processor 260. When the patrol dispatch control device 200 is composed of a plurality of servers, the flight control unit 261 corresponds to the flight control server 210 described above, and the image collection unit 263 is the image transmission server 220 ). The image analysis unit 265 may correspond to the image analysis server 230.

The flight control unit 261 may perform flight control of a plurality of drones 100 and 500. For example, the flight controller 261 may select at least one of the plurality of drones 100 and 500 as the primary drone and control the primary patrol dispatch of the selected drone. In addition, when an abnormal condition occurs, the flight control unit 261 may select at least one of the plurality of drones 100 and 500 as a secondary drone and dispatch the selected drone to an abnormal state occurrence area. The flight controller 261 may perform camera control of the plurality of drones 100 and 500 to control acquisition of an image for a designated patrol area and a detailed image for an abnormal state occurrence area.

When the image collection unit 263 receives an image from the primary drone, the image collection unit 263 may store the image in the first memory 240. The image collection unit 263 may request the image analysis unit 265 for image analysis on the primary drone image stored in the first memory 240. The image collection unit 263 may control to transmit the received secondary drone image to the remote operator device 300 when an image is received from the secondary drone according to the occurrence of an abnormal condition.

The image analysis unit 265 may perform image analysis on the primary drone image stored in the first memory 240 at the request of the image collection unit 263. When the image analysis unit 265 detects a pattern (eg, an intruder occurrence pattern, vehicle intrusion or theft occurrence pattern, fire occurrence pattern, etc.) in the image as a result of the image analysis, it may determine that an abnormal state has occurred. The image analysis unit 265 may provide a result of occurrence of an abnormal condition to the image collection unit 263.

The on-site dispatch support unit 267 may support the on-site dispatch of the dispatch device 400 according to the occurrence of an abnormal condition. For example, after it is determined that an abnormal condition has occurred in the first drone image, the on-site dispatch support unit 267 determines the location of the abnormal condition occurrence area in the dispatch device 400 when the abnormal condition has occurred in the second drone image. You can provide information. In addition, the field dispatch support unit 267 may share the second drone image with the dispatch device 400 in real time. Alternatively, when an abnormal condition occurs in the first drone image, the on-site dispatch support unit 267 provides the first drone image related to the occurrence of the abnormal condition to the remote operator device 300, and the dispatch device 400 from the remote operator device 300 ) When a dispatch request is received, the dispatch device 400 may request dispatch to an abnormal state occurrence area while sharing information on the location of the corresponding region. Alternatively, when the on-site dispatch support unit 267 receives a request for on-site dispatch from the remote operator device 300 that has checked the secondary drone image, the dispatch device 400 may process information related to the dispatch. The sharing of information with the dispatch device 400 of the on-site dispatch support unit 267 is determined as a case where the result of the analysis of the first drone image is determined to be an abnormal condition according to the policy of the patrol dispatch control device 200, or the second drone image It may include a case where the analysis result is determined as the occurrence of an abnormal state. Alternatively, the on-site dispatch support unit 267 may request the on-site dispatch of the dispatch device 400 to a designated location at the request of the remote operator device 300, regardless of the determination of occurrence of an abnormal condition.

The first memory 240 may store and manage data related to operation of the patrol dispatch control device 200. For example, the first memory 240 may store an image analysis algorithm 241, a flight control algorithm 243, and abnormal state information 245. The image analysis algorithm 241 may include an algorithm for determining occurrence of an abnormal state through image analysis of a primary drone image and a secondary drone image. For example, the image analysis algorithm 241 may include an algorithm that checks whether a specified pattern (eg, a person intrusion pattern, a vehicle intrusion or vehicle theft pattern, a fire occurrence pattern, etc.) is detected in the drone image acquired by the drone. have. The image analysis algorithm 241 may include a first threshold value (or a reference value) for discriminating the occurrence of an abnormal state in the primary drone image and a second threshold value for determining the occurrence of an abnormal state in the secondary drone image. The flight control algorithm 243 may include a first flight control routine related to the patrol movement of the primary drone, and a second flight control routine related to an abnormal state occurrence area of the secondary drone. The first flight control routine may include a routine set to fly through a designated speed, direction, and route while maintaining a designated altitude in a designated area. The second flight control routine moves to the abnormal state occurrence area, maintains the altitude according to the type of the abnormal state, and maintains a close distance that allows precise photographing of the abnormal state, and a routine set to photograph an abnormal state occurrence object, and a plurality of It may include a flight route deduplication routine to suppress collisions between drones.

4A is a diagram illustrating an example of a remote operator device according to an embodiment of the present invention.

Referring to FIG. 4A, a remote operator device 300 according to an embodiment includes a second communication interface 301, a second processor 360, a second memory 340, an input/output unit 350, and a display 370. It may include.

The second communication interface 301 may establish a remote communication channel with the patrol dispatch control device 200. The second communication interface 301 may receive at least one image from the patrol dispatch control device 200 and output it to the display 370 under the control of the second processor 360. The second communication interface 301 may transmit a message requesting a second drone dispatch to the patrol dispatch control device 200 according to the operation of the input/output unit 350.

The second memory 340 may store and manage data related to operation of the remote operator device 300. For example, the second memory 340 may store at least a part of an image (eg, a primary drone image) received from the patrol dispatch control device 200. Alternatively, the second memory 340 may store an image for a predetermined time based on a time point according to occurrence of an abnormal condition. The second memory 340 may store a record of a second call request. In addition, the second memory 340 may store at least a part of the secondary drone image.

The input/output unit 350 may generate an input signal and output a signal of the remote operator device 300. In this regard, the input/output unit 350 includes, for example, an input unit including at least one of at least one of a physical key, a keypad, a touch screen, and a microphone for a voice command, a speaker capable of outputting a designated signal, a lamp, and a vibration. It may include an output unit including one. The input/output unit 350 may output an audio signal guiding it when it is determined that an abnormal state has occurred in the primary drone image. Accordingly, the user of the remote operator device 300 can recognize the occurrence of an abnormal condition through the audio signal while performing other security related tasks. In addition, the input/output unit 350 may generate a user input related to a second drone dispatch request and a user input requesting the dispatch device 400 to move to the site according to user control.

The display 370 may output at least one screen related to operation of the remote operator device 300. For example, the display 370 may output at least a portion of a primary drone image and at least a portion of a secondary drone image. In addition, the display 370 may output an image analysis result of the primary drone image. The display 370 may output a screen related to a request to dispatch a secondary drone. The display 370 may output a screen related to the dispatch device 400 on-site dispatch.

The second processor 360 may process and transmit signals related to operation control of the remote operator device 300. In this regard, the second processor 360 may include an image processing unit 361. The image processing unit 361 may control to output the image received from the patrol dispatch control device 200 to the display 370 in a specified size. Also, when a pattern according to occurrence of an abnormal condition is detected in the primary drone image, the image processing unit 361 may control to output a message indicating occurrence of the abnormal condition or a display effect to the display 370. The image processing unit 361 outputs a screen interface related to the second drone dispatch, and when a second drone dispatch is requested according to a user input, generates a message requesting the second drone dispatch, and then generates a message requesting the dispatch of the second drone. It can be provided to (200). When the image processing unit 361 receives the image related to the abnormal state occurrence area from the secondary drone, it outputs it to the display 370, but after dividing the screen according to the number of drones, the images of the plurality of secondary drones are displayed. It can be controlled to output. The image processing unit 361 may share at least a portion of the secondary drone image to the dispatch device 400 according to user control of the remote operator device 300 or according to a predefined policy.

4B is a diagram illustrating an example of a screen interface of a remote operator device according to an embodiment of the present invention.

Referring to FIG. 4B, the display 370 of the remote operator device 300 outputs a map of the area patrolled by the primary drone and a screen including flight-related information of the primary drone to the first area 371, and , The image provided by the primary drone may be output to the second area 372. When the video analysis server 230 (or the video analysis unit 265) of the patrol dispatch control device 200 detects an abnormal object related to the occurrence of an abnormal condition, the display 370 of the remote operator device 300 will trigger an event ( Button) is activated. When the event (abnormal) dispatch button of the primary drone is selected, the remote operator device 300 may create a message requesting the dispatch of the second drone and provide it to the patrol dispatch control device 200. In this case, the display 370 of the remote operator device 300 may output a second drone dispatch request message to the third area 373. The first area 371, the second area 372, and the third area 373 are displayed as different screens on one display 370, or after splitting one screen, the divided screen (e.g.: 3 screens), or may be respectively output on a plurality of displays. The second drone dispatch request may be automatically selected according to the occurrence of an abnormal condition.

5 is a diagram showing an example of a method of operating a patrol dispatch control device according to an embodiment of the present invention.

Referring to FIG. 5, the first processor 260 of the patrol dispatch control device 200 may perform a first drone operation in step 501. In this step, the first processor 260 may select a drone to be operated as a primary drone among the plurality of drones 100 and 500, and may provide mission information to the drone through a communication channel. The mission information may include information on an area to be patrolled by the primary drone, patrol speed, altitude, and patrol route.

The first processor 260 may collect and analyze a primary image in step 503. In this regard, the primary drone dispatched to patrol may transmit an image to the patrol dispatch control device 200 that has established a communication channel through the base station 150.

The first processor 260 may check whether an abnormal state has occurred in step 505. During the analysis process, the first processor 260 may store and manage the previous image of the designated area and compare it with the currently acquired current image of the designated area. When the comparison result is greater than or equal to a specified value, the first processor 260 may determine that an abnormal state has occurred. Alternatively, the first processor 260 may check whether a specified pattern (eg, a specified object) corresponding to an abnormal state is included in the corresponding image. If there is no occurrence of the abnormal state specified in this step, it is possible to branch to step 501 and perform the following operation again. In the process of checking whether an abnormal condition has occurred, the first processor 260 may check whether the image analysis result corresponds to a predetermined specific pattern. The predefined specific patterns may include patterns corresponding to various human resources or natural disasters, such as intrusion detection patterns, fire occurrence patterns, structure collapse patterns, and flood occurrence patterns. In this regard, the patrol dispatch control device 200 may pre-store data corresponding to the various specific patterns and perform a matching operation with the currently acquired image analysis result. The first processor 260 may determine the probability of occurrence of an abnormal state as greater than or equal to the specified value when the similarity rate with the specified pattern as a result of matching is equal to or greater than a specified value.

In step 505, when the probability of occurrence of an abnormal condition is greater than or equal to a specified value, or if an abnormal condition occurs, the first processor 260 may perform a secondary drone operation in step 507. In this regard, the first processor 260 may select at least one of the plurality of drones other than the drone used as the primary drone as the secondary drone. During the drone selection process, the first processor 260 may adjust the number of secondary drones according to the type of abnormal state. Alternatively, the first processor 260 may adjust the number of secondary drones according to the size of the abnormal state. For example, the first processor 260 may select one secondary drone when a person intrudes into a designated area, and may add the number of secondary drones according to the number of intruders. In addition, the second processor 360 may select four drones as secondary drones when a fire occurs in a designated area, and add or reduce drones according to the size of the fire.

In addition, the first processor 260 may calculate the dispatch time of the second drone in consideration of the return time of the primary drone, and dispatch the second drone according to the calculation result. For example, the first drone must start returning after 10 minutes from the current point in consideration of the remaining battery capacity or the pre-defined total flight time, and if the time required to the area where the secondary drone has an abnormal condition is 5 minutes, the 1 The processor 260 may dispatch the second drone 5 minutes after the current point in time. Alternatively, the first processor 260 may determine the dispatch time of the secondary drone based on the remaining flight time of the primary drone. For example, if the remaining flight time of the primary drone is 15 minutes, and the return time is 3 minutes, the primary drone can stay in the abnormal condition for 12 minutes longer. In addition, when the time required to the abnormal state occurrence area of the secondary drone is 5 minutes, the first processor 260 may control to launch the secondary drone 7 minutes after the current point in time.

On the other hand, the first processor 260 selects at least one secondary drone at the time when the occurrence of an abnormal condition is determined, regardless of the remaining flight time of the primary drone, and transfers the selected secondary drone to the corresponding abnormal condition occurrence area. You can also control it to be dispatched. In this process, the first processor 260 may control regional division so that the primary drone and the secondary drone do not collide. In addition, when the return time of the primary drone arrives, the first processor 260 repositions the secondary drones so that images of the abnormal state occurrence area can be secured with the secondary drones other than the primary drone. Calculation, and the position of the secondary drones can be adjusted.

In step 509, the first processor 260 may collect and analyze a secondary image. In this regard, the secondary drone may form a communication channel with the patrol dispatch control device 200 through the base station 150, and provide the acquired image to the patrol dispatch control device 200 through the base station 150. . The image acquired by the secondary drone may include an image having a higher resolution of the abnormal state object than that of the primary drone.

In step 511, the first processor 260 may check whether the abnormal state is equal to or greater than a specified reference value. When the abnormal state is less than the reference value, the first processor 260 may branch to step 501 and perform the following operation again. Alternatively, the first processor 260 may branch to before step 507 and perform the following operation again. In step 511, when the abnormal state is equal to or greater than the specified reference value, the first processor 260 may process field dispatch support in step 513. For example, when the object corresponding to the abnormal state is determined to be an intrusive person, the first processor 260 may determine the abnormal state to be equal to or higher than a specified reference value, and request the dispatch device 400 to be dispatched to the site. In connection with the request to be dispatched to the site, the first processor 260 may provide information on the dispatched location and a second drone image to the dispatching device 400.

Thereafter, the first processor 260 may branch to step 501 according to a predefined policy and perform the following operation again, or may perform data processing or signal processing for processing a function defined in the policy.

6 is a view showing another example of a method of operating a patrol dispatch control device according to an embodiment of the present invention.

Referring to FIG. 6, the first processor 260 of the patrol dispatch control apparatus 200 may perform a first drone operation in step 601 and perform a first image collection and analysis in step 603. In addition, the first processor 260 may check whether an abnormal state has occurred in step 605. Steps 601, 603, and 605 may be steps substantially the same as or similar to steps 501, 503, and 505 described in FIG. I will do it.

In operation 605, when the probability of occurrence of an abnormal condition is greater than or equal to a specified value, the first processor 260 may output an alarm in operation 607. Regarding the possibility of occurrence of an abnormal condition, the previous image for the corresponding area is stored and managed, the currently acquired image and the previous image are compared, and if the difference value is greater than or equal to the specified value, the first processor 260 may cause the abnormal condition to occur. It can be determined that there is.

In the process of outputting the alarm, the first processor 260 may transmit a message corresponding to the alarm to the remote operator device 300 designated in advance. Alternatively, the first processor 260 may output the alarm through an output device that is functionally connected. In this regard, the first processor 260 may include an output device (for example, a display capable of outputting at least one of a text and an image corresponding to an alarm, an audio device capable of outputting an audio signal corresponding to an alarm, a lamp, A device including at least one of a siren and a vibration module) may be further included.

The first processor 260 may check whether an input corresponding to the second drone operation request is received in step 609. In this process, the first processor 260 may check whether a message corresponding to the second drone operation request is received from the remote operator device 300. In this regard, the first processor 260 transmits a message requesting the operation of the second drone to the remote operator device 300 by including in the alarm message, or after transmitting the alarm message, a message related to the request for the operation of the second drone May be transmitted to the remote operator device 300. The first processor 260 may determine that if a response message is not received within a specified time after transmission of a message related to a second drone operation request, the second drone operation is not required. Alternatively, the first processor 260 may automatically determine that a second drone operation is required when there is no reception of a designated response message according to a user setting or a predefined policy. As another example, the first processor 260 may output a message requesting a second drone operation request through an output device functionally connected to the patrol dispatch control device 200 and directly receive a user input. In this regard, the first processor 260 may further include at least one of a touch screen capable of receiving a user input, a voice command input means (eg, a microphone), and a physical key related to a user input.

When the second drone operation request is not received, the first processor 260 may branch to step 601 and perform the following operation again. When a second drone operation request is received, the first processor 260 may control the second drone operation in step 611. Regarding the second drone operation, the first processor 260 receives the number of drones to be dispatched secondly from the remote operator device 300, and controls the drones corresponding to the number to be dispatched to the second drones. have. Alternatively, the first processor 260 may control to dispatch a designated number of drones to the secondary drones according to the type of the abnormal state or the size of the abnormal state.

The first processor 260 may collect, analyze, and share a secondary image in step 613. In this regard, the secondary drone may acquire an image of an abnormal state occurrence area transmitted from the primary drone, and transmit the acquired image to the patrol dispatch control device 200 as a secondary image. The first processor 260 may analyze the collected secondary image and transmit it to the remote operator device 300. Alternatively, the first processor 260 analyzes the secondary image to provide more detailed information about the abnormal state (e.g., the number and behavior patterns of intrusion personnel, intrusion routes and exit routes, the size of the fire, and the type of fire). , The size of the collapse of the structure, the location of the collapse of the structure, etc.) can be analyzed.

In step 615, the first processor 260 may check whether the abnormal state is equal to or greater than a specified reference value. The reference value is when a specific pattern corresponding to an abnormal state has a similarity rate with a real human resource or a natural disaster occurrence pattern greater than or equal to a specific ratio (e.g., a ratio relatively higher than the ratio used for detecting a specific pattern in step 605). It can contain the corresponding value. Alternatively, the reference value may include a value corresponding to a case of receiving a user input that determines that an abnormal condition has occurred from the remote operator device 300.

When an abnormal condition occurs, the first processor 260 may process the sage dispatch support in step 617. The field dispatch support may be a step substantially the same as or similar to step 513 described in FIG. 5.

As described above, the specification includes details of a number of specific implementations, but these should not be understood as limiting to any invention or scope of claimable, rather, which may be specific to a particular embodiment of a particular invention It should be understood as a description of features.

Further, although the operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order shown, or that all illustrated operations must be performed in order to obtain a desired result. In certain cases, multitasking and parallel processing can be advantageous. Further, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the program components and systems described are generally integrated together into a single software product or packaged in multiple software products. It should be understood that you can.

The present description presents the best mode of the present invention, and provides examples to illustrate the present invention and to enable those skilled in the art to make and use the present invention. The written specification is not intended to limit the present invention to the specific terms presented. Accordingly, although the present invention has been described in detail with reference to the above-described examples, those of ordinary skill in the art can make modifications, changes, and modifications to these examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be determined by the described embodiments, but should be determined by the claims.

According to the present invention, the present invention provides at least one filter capable of excluding objects not related to intrusion detection, thereby providing an alarm according to more accurate intrusion detection, thereby minimizing false alarms, and detecting false intrusions. We support to minimize support for dispatch according to the situation.

10: Patrol dispatch control system
100, 500: multiple drones
150: base station
200: patrol dispatch control device
201: first communication interface
210: flight control server
220: video transmission server
230: video analysis server
240: first memory
300: remote operator device
301: second communication interface
340: second memory
350: input/output unit
360: second processor
370: display
400: dispatch device

Claims (11)

A plurality of drones acquiring an image through a camera;
A patrol dispatch control device that performs patrol dispatch control of the plurality of drones;
Including; a remote operator device that receives and monitors the images acquired by the plurality of drones from the patrol dispatch control device,
The patrol dispatch control device
At least one of the plurality of drones is dispatched to a designated area as a primary drone,
When an abnormal condition occurs in the designated area, at least one of the remaining drones other than the primary drone is selected as a secondary drone, and based on the location information of the abnormal condition occurrence area obtained from the primary drone Patrol dispatch control system, characterized in that for controlling to dispatch the second drone to the area where the abnormal condition has occurred. A communication interface for forming a communication channel with at least one of the plurality of drones;
Includes; a processor functionally connected to the communication interface,
The processor is
At least one of the plurality of drones is selected as the primary drone and controlled to patrol a designated area,
When an abnormal condition occurs in the designated area based on the image analysis acquired from the primary drone, at least one of the remaining drones excluding the primary drone is selected as the secondary drone,
The patrol dispatch control device, characterized in that for controlling to dispatch the second drone to the area where the abnormal condition has occurred based on the location information on the area where the abnormal condition has been acquired from the primary drone. The method of claim 2,
The processor is
Patrol dispatch control device, characterized in that adjusting the number of the secondary drones according to the type of occurrence of the abnormal state. The method of claim 2,
The processor is
Patrol dispatch control device, characterized in that adjusting the number of the secondary drones according to the size of the abnormal state. The method of claim 2,
The processor is
And transmitting an alarm message according to the occurrence of the abnormal condition to a remote operator device, and determining the number of the secondary drones according to a response of the remote operator device. The method of claim 2,
The processor is
Patrol dispatch control device, characterized in that for controlling the on-site dispatch request of the dispatch device based on the image analysis obtained from the secondary drone. The method of claim 6,
The processor is
Patrol dispatch control device, characterized in that for controlling to transmit the abnormal state occurrence location information and the image obtained from the secondary drone to the dispatch device. The method of claim 2,
The processor is
The flight control is performed to reduce the distance between the secondary drone and the location of the abnormal state so that the resolution of the image related to the abnormal state of the secondary drone is higher than the resolution of the image related to the abnormal state of the primary drone. Patrol dispatch control device made by the The method of claim 2,
The processor is
Before the primary drone returns from the abnormal state occurrence region, a time for the second drone to arrive at the abnormal state occurrence region is calculated, and the dispatch time of the second drone is determined based on the calculated time. Patrol dispatch control device, characterized in that. A step of selecting at least one of the plurality of drones as a primary drone by the processor of the patrol dispatch control device to patrol a designated area;
Detecting occurrence of an abnormal condition in the designated area based on the image analysis obtained from the primary drone;
When the occurrence of the abnormal state is detected, selecting at least one of the drones other than the primary drone as a secondary drone;
And dispatching the second drone to the area where the abnormal condition has occurred based on the location information on the area where the abnormal condition has occurred acquired from the first drone. The method of claim 10,
The step of dispatching the second drone
Calculating a time for the secondary drone to arrive at the abnormal state occurrence area before the primary drone starts to return;
Patrol dispatch control method comprising; dispatching the second drone based on the calculated time.

Images