KR102050821B1 - Method of searching fire image based on imaging area of the ptz camera - Google Patents

Abstract

A fire detection method using a PTZ camera is disclosed. The present invention includes a training step of deep learning training the imaging unit and the image analysis device in advance; A primary detection step of dividing the image photographed by the photographing unit into NxN, and determining whether a specific region is fired among the images divided into trained CNNs based on the divided images; Shooting the second detection step to determine whether the fire by the CNN trained on the basis of the fire occurrence area for the area detected in the first detection step by the image analysis device and the fire occurrence area identified in the second detection step Cooperation between the image capture device and the image analysis device including an intelligent PTZ camera, including a precise control step of analyzing the image of the enlarged fire area by pan, tilt, and zoom control, and a fire determination step of judging the fire. It provides a fire detection method using a PTZ camera that can improve the fire detection accuracy even using a single PTZ camera.

Description

Fire detection method using a PTZ camera {METHOD OF SEARCHING FIRE IMAGE BASED ON IMAGING AREA OF THE PTZ CAMERA}

The present invention relates to a fire detection method, and more particularly to a fire detection method using a PTZ camera that can detect a fire occurrence area based on the shooting area of the PTZ camera.

In general, in the field such as petrochemical, gas, refinery related factories, many dangerous substances are treated at high temperature and high pressure in various processes, and there is always a risk of fire due to explosion.

In addition, the factories that have a risk of explosion as described above have a great impact on the surrounding factories or residential areas that are caused by the explosion of the site, and the casualties and property damage caused by the explosion can be enormous.

At present, the system to prevent the disaster caused by the explosion in the industrial site is simply to check the visual image through the installation of CCTV, and also to detect the heat through various sensors. There is no facility to monitor the area, which raises the problem of always being exposed to fire.

On the other hand, since a PTZ camera can capture an image while moving various photographing regions, the image captured by the PTZ camera includes images of various photographing regions. Therefore, in order to view an image of a specific region, it is necessary to search for an image of a specific region among images captured by a PTZ camera.

Therefore, various attempts to prevent fires using PTZ cameras are underway.

Therefore, an object of the present invention for solving this problem is a training step of deep learning training in advance the imaging unit and the image analysis device; A primary detection step of dividing the image photographed by the photographing unit into NxN, and determining whether a specific region is fired among the images divided into trained CNNs based on the divided images; Shooting the second detection step to determine whether the fire by the CNN trained on the basis of the fire occurrence area for the area detected in the first detection step by the image analysis device and the fire occurrence area identified in the second detection step Cooperation between the image capture device and the image analysis device including an intelligent PTZ camera, including a precise control step of analyzing the image of the enlarged fire area by pan, tilt, and zoom control, and a fire determination step of judging the fire. It provides a fire detection method using a PTZ camera that can improve the fire detection accuracy even using a single PTZ camera.

In order to achieve the above object, the present invention includes a training step of deep learning training the imaging unit and the image analysis device in advance; A primary detection step of dividing the image photographed by the photographing unit into NxN, and determining whether a specific region is fired among the images divided into trained CNNs based on the divided images; Shooting the second detection step to determine whether the fire by the CNN trained on the basis of the fire occurrence area for the area detected in the first detection step by the image analysis device and the fire occurrence area identified in the second detection step A pan, tilt, zoom control unit includes a precise control step of analyzing an enlarged image of the fire area and a fire determination step of determining whether the fire, the second detection step (S300), the photographing unit 100 In order to separate the flame or smoke from the background, the image analysis device 300 to determine whether the fire by obtaining the characteristics of the temporal change through different color channels matched at different times to separate the flame or smoke from the background; In order to separate the flame or smoke from the background, the image analysis device 300 uses a CNN that inputs SRGB (Sequential RGB) as a detection algorithm. The image of the SRGB is made by combining the (t) red channel of the frame, the green channel of the frame t-1, and the blue channel of the t-2. In the precision adjustment step (S400), pan, tilt, and zoom control of the photographing unit 100 analyzes an enlarged image of a fire occurrence area, but includes a first detection step S200 and a second detection step ( In step S300, if the probability of the fire detected in the second detection step S300 is greater than or equal to a threshold value, the pan and tilt of the PTZ camera of the photographing unit 100 are adjusted to the center point of the boundary (BoundBox) of the corresponding area. The image is captured after zooming in in proportion to the size of the BoundBox, and the photographing unit 100 performs the first detection step S200 again based on the enlarged image, and then captures the captured image. ), And the image analysis device 300 detects the received image in the second detection step ( S300) provides a fire detection method using a PTZ camera that is re-executed by deep learning.

In addition, the photographing unit may include one PTZ camera.

Further, the captured image is analyzed by subdividing the image into NxN areas, but N may be divided into 6 to split the screen horizontally and vertically.

delete

delete

delete

delete

delete

According to the fire detection method using the PTZ camera of the present invention described above, even through the use of a single PTZ camera through the collaboration between the imaging unit and the image analysis device including the intelligent PTZ camera to increase the fire detection accuracy and one PTZ camera Only the images from the analysis do not have to have expensive equipment, there is an effect that can minimize the load on the image analysis device.

1 is a schematic diagram of a fire detection system utilizing a PTZ camera according to an embodiment of the present invention.
2 is a flowchart illustrating a fire detection method using a PTZ camera according to the present invention.
3 and 4 are a plurality of pictures for explaining the training step according to the present invention.
5 to 7 are a plurality of photographs for explaining the photographing unit according to the present invention.
8 is a plurality of pictures for explaining the image analysis device according to the present invention.
9 is a plurality of photographs for explaining the precise adjustment step according to the present invention.

In order to fully understand the present invention, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Embodiment of the present invention may be modified in various forms, the scope of the invention should not be construed as limited to the embodiments described in detail below. This embodiment is provided to more completely explain the present invention to those skilled in the art. Therefore, the shape of the elements in the drawings and the like may be exaggerated to emphasize a more clear description. It should be noted that the same members in each drawing are sometimes shown with the same reference numerals. In addition, detailed descriptions of well-known functions and configurations that are determined to unnecessarily obscure the subject matter of the present invention are omitted.

In particular, the present invention is to develop a system for detecting fire with high accuracy and generating an alarm. Since only one pan-tilt-zoom (PTZ) security camera can be used to detect a fire that occurs in 360-degree directions of the camera's installation location, the fire is compared to the fire detection system, which required the installation of multiple cameras. It is characterized by dramatically reducing the installation cost of the camera for prevention.

In this case, the PTZ camera is a camera capable of controlling rotation, pan, tilt, and zoom. In this case, rotation, pan, tilt, and zoom are one method of camera operation, pan is a control method to move the camera left and right, tilt is a control method to move the camera up and down, and zoom changes the focal length of the lens to be photographed. The control method to adjust the size. For example, the pan and tilt operations may be performed by controlling a support for supporting the camera. For example, the camera may be fixed to the support and the support may be rotated left and right to perform a pan operation, and the support may be rotated up and down to perform a tilt operation. The zooming operation may be performed by controlling the focal length of the lens of the camera. Since such a control method is a well-known technique, the description thereof will be omitted.

1 is a schematic diagram of a fire detection system utilizing a PTZ camera according to an embodiment of the present invention.

That is, as shown in Figure 1, the fire detection system using a PTZ camera according to the present invention is the photographing unit 100, the network 200, the image analysis device 300, the image storage server 400 and the user input device 500 ) May be included.

In this case, the photographing unit 100 according to the present invention includes one camera capable of capturing an area to be monitored for fire. The camera of the photographing unit 100 may include the above-described PTZ camera, so that the 360 degree of the photographing place by the photographing unit 100 is a surveillance area.

The image analyzing apparatus 300 may obtain image information from the photographing unit 100 wirelessly through the network 200. For example, the image analyzing apparatus 300 obtains an image representing the entire area to be monitored from the photographing unit 100, and analyzes and determines whether a fire is in a specific region based on the image. In addition, the image analysis device 300 stores the fire analysis information in the image storage server 400.

The image retrieval apparatus 400 is an apparatus capable of storing images captured by the photographing unit 100 and analysis information of the images from the image analyzing apparatus 300 and searching for an image of a region desired by a user among the stored images. . In addition, the user may access the image storage server 400 through the user input device 500 to select a search area, which is an area to search, and input information. For example, the user may look at the enlarged image of the search area by using the user input device 400, and may also search the image storage server 400 in which the information analyzed by the image analysis device 300 is stored. have.

The user input device 500 may obtain data from the user and transmit the data to the image search apparatus 200. The user input device 400 may include a keyboard or a touch panel coupled to the display unit 300. The user input device 400 may receive information about the search area from the user and transmit the information about the search area to the image search device 200.

The user input device 500 and the image storage server 400 may be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

On the other hand, since a PTZ camera can capture an image while moving and rotating various capturing regions, the image captured by the PTZ camera includes images of various capturing regions. Therefore, in order to view an image of a specific fire occurrence area, it is necessary to search and analyze an image of a fire occurrence area among images captured by a PTZ camera.

2 is a flowchart illustrating a fire detection method using a PTZ camera according to the present invention.

Therefore, as shown in FIG. 2, the fire detection method using the PTZ camera according to the present invention trains the photographing unit 100 and the image analyzing apparatus 300 in advance by deep learning and periodically learns the captured image periodically (S100). ), The primary detection step (S200) for detecting the fire occurrence area primarily by dividing a plurality of images taken by the photographing unit 100 in the primary detection step (S200) by the image analysis device 300 A second detection step (S300) of determining whether a fire is performed by a trained CNN based on a fire occurrence area for the detected area (S300), and a precise adjustment step of precisely adjusting and analyzing the photographing unit 100 to photograph the fire occurrence area ( S400 and a fire determination step (S500) of determining whether or not a fire.

3 and 4 are a plurality of pictures for explaining the training step (S100) according to the present invention.

First, the training step (S100) is a step of training the image analysis device 300 by using a deep learning model.

In this case, the deep learning model uses neural networks and objects for image feature extraction through multi-layer CNN using more than 1 million image learning DBs such as public images of public images of ImageNet, MSCOCO, PASCAL VOC and self-collected fire images as shown in FIG. 3. The image analysis apparatus 300 is trained together with regression learning of a neural network representing rectangular information surrounding the data.

At this time, as shown in Figure 4, deep learning training such as data augmentation, batch normalization, multi-crop adaptation, fast connection to solve the under-fitting and over-fitting problems of machine learning and to classify and recognize various objects simultaneously Technology can be applied.

In addition, the present invention utilizes the trained image analysis apparatus 300 and simultaneously detects a fire occurrence area by dividing it into the following two methods.

That is, in the first detection step (S200), the PTZ camera of the photographing unit 100 is used to generate the motion captured by the photographing unit 100, and the primary in the current preset view during auto preset monitoring through the color histogram. The detection position is transmitted to the image analysis device 300 through the network 200.

In this case, as shown in FIG. 5, the photographing unit 100 divides the photographed image into NxN areas and analyzes them in detail. In this case, as shown in FIG. 5, the split value N is preferably divided into six horizontally and vertically with 6 as the default value.

In addition, as shown in FIG. 6, the HSI color histogram is extracted from each of the subdivided regions, and the fire region having a threshold probability or higher is detected through SVM linear classification. That is, the PTZ camera of the photographing unit 100 selects a screen area above the threshold probability among the NxN split screen inverses through the color histogram according to the motion occurrence.

Subsequently, as shown in FIG. 7, the photographing unit 100 obtains feature points using a Harris corner detector in a detailed area suspected of a fire area having a threshold probability or higher, and detects the motion of the feature point in time. In this case, the flame shows up, down, left, and right movements, and smoke moves the feature points in a certain direction of the upper left, upper right, and upper sides. The first fire detection information and the captured image are transmitted to the image analyzing apparatus 300, and the second analyzing step S300 is performed by the image analyzing apparatus 300 based on the image.

In the second detection step S300, when the image analyzing apparatus 300 receives the primary detection information and the image from the photographing unit 100, the image analysis apparatus 300 attempts the second precision detection using the deep learning model.

In this case, the image analysis apparatus 300 uses a CNN as an input of SRGB (Sequential RGB) as a detection algorithm for separating the white smoke and the background which are fluctuated in the fire. In this case, as an aspect of the present invention, the SRGB image obtains a feature for temporal change through different color channels matching the image received from the photographing unit 100 at different times. For example, the image of the SRGB may be a red channel of (t) frame, a green channel of (t-1) frame, and a blue channel of (t-2). Is made by combining. That is, the above-mentioned SRGB image is used to obtain a characteristic about the temporal change of the fluctuating fire smoke.

In detail, as shown in FIG. 8, since the background has little movement in the image received from the photographing unit 100, changes of RGB (Red, Green, Blue) components in t, t-1, and t-2 frames There are few, but in the case of fluctuating flames or smoke, the RGB component changes from frame to frame. In this case, the dimension of the SRGB image should be at least 608x608x3 or more to ensure accurate detection.

The output of each layer of the CNN that takes an SRGB as an input can be expressed as in Equation (1) below. That is the node of the layer l {1, 2, ..., j } as the layer l -1 of the node when the called {1, 2, ..., i }, weight W b and the bias for each node The output O of node has the form of Equation (1) as below.

를 계산하고 최적의 parameter를 찾기 위해 back-propagation을 수행하여 parameter를 update한다. 마찬가지로 Box Regression을 위해 영상에서 검출하려는 객체의 폭 OW _k 와 높이 OH _b 를 참값과 비교하는 항을 loss function에 추가하여 아래와 같은 식 (2)를 도출한다.In addition, when the node of layer l +1 is {1, 2, ..., k }, the loss function is defined as follows by comparing the estimation value O _k with the actual label t _k . using the loss function

The parameter is updated by performing back-propagation to find the optimal parameter. Likewise, the equation (2) is derived by adding the term comparing the width OW _k and the height OH _b with the true value to the loss function for Box Regression.

Looking at the structure of the CNN that takes the SRGB as an input, it first generates the SRGB, and then has 30 convolution layer feature extraction structures and a final stage detection structure. For example, kernel uses a kernel of convolution 3x3 size and max-pooling uses a kernel of 2x2 size. And the structure of each CNN is applied Batch Normalization to optimize the training process. It also uses RELU (rectified linear unit) as the layer's activation function.

In this case, it is preferable to adjust the reproducibility of about 84% at 0.01 FPR through SRGB CNN and BoundBox Regression to automatically detect the fire occurrence area.

In the fine adjustment step (S400), as shown in FIG. 9, the PTZ camera is moved to the fire suspect area by pan, tilt, and zoom control of the fire occurrence area detected in the first detection step S200 and the second detection step S300. This is the step of precisely adjusting the image so that it can be zoomed in intensively.

That is, in the fine adjustment step (S400), if the probability of the fire detected in the second detection step (S300) is greater than or equal to the threshold, the pan and tilt of the PTZ camera of the photographing unit 100 are adjusted to the center point of the boundary (BoundBox) of the corresponding area. The image is captured after zooming in in proportion to the size of the BoundBox, and the photographing unit 100 performs the first detection step S200 again based on the enlarged image, and then captures the captured image. Will be sent). In addition, the image analysis apparatus 300 re-performs the received image by deep learning in the second detection step (S300). That is, since the first detection step S200 and the second detection step S300 are performed with the enlarged image of the PTZ camera, the detection accuracy may be further increased.

In the step of determining whether or not the fire (S500) in the image analysis device 300 to control the pan, tilt, zoom for the PTZ camera of the photographing unit 100 to receive a magnified image and analyze it to determine whether the fire alarm or not to be.

In this case, the second analysis step S300 of the enlarged area is performed by the image analysis apparatus 300 to generate an alarm when the deep learning result is also a fire threshold or more. That is, if the possibility of a fire is greater than or equal to a threshold, the image analysis device 300 wirelessly transmits an alarm to the user input device 500 through the network 200, and an alarm signal (for example, near the photographing unit 100). , Patterned light emission or alarm sound). In addition, the image analysis device 300 is to store the image of the location, time and the high probability of fire in the image storage server 400. The neural network of the training step S100 used initially is periodically based on the captured image and event information analyzed in the second detection step S300 and the fine adjustment step S400, and the fire information confirmed by the user input device 500. The automatic detail learning to increase the fire detection accuracy, reduce the false detection and intelligently manage the number of times the photographing unit 100 enters the fine adjustment step (S400).

Through the collaboration between the photographing unit 100 and the image analysis device 300 including such an intelligent PTZ camera, even if one PTZ camera is used, the accuracy of fire detection is improved and only the images from one PTZ camera are analyzed. It is not necessary to have the equipment, there is an effect that can minimize the load of the image analysis device 300.

The embodiment of the fire detection method using the PTZ camera of the present invention described above is merely exemplary, and those skilled in the art to which the present invention pertains various modifications and other equivalent embodiments therefrom. You can see that. Therefore, it will be understood that the present invention is not limited to the forms mentioned in the above detailed description. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims. It is also to be understood that the present invention includes all modifications, equivalents, and substitutes within the spirit and scope of the invention as defined by the appended claims.

100: shooting unit
200: network
300: image analysis device
400: video storage server
500: user input device
S100: training stage
S200: first detection step
S300: second detection step
S400: Precision Adjustment Step
S500: fire determination stage

Claims (8)

A training step of deep learning training the photographing unit 100 and the image analyzing apparatus 300 in advance (S100);
A primary detection step (S200) of dividing the image photographed by the photographing unit (100) into N × N, and determining whether a specific region is fired from the image divided into the trained CNN based on the divided image;
A second detection step (S300) of determining whether a fire is detected by a CNN trained on the basis of the fire occurrence area of the area detected by the image analysis device (300) in the first detection step (S200);
A fine adjustment step (S400) of analyzing the image of the enlarged fire occurrence area by pan, tilt and zoom control of the photographing unit 100 to capture the fire occurrence area determined in the second detection step (S300);
Including a fire determination step (S500) to determine whether the fire,
The secondary detection step (S300),
Image analysis device for receiving the information captured by the photographing unit 100 to determine whether the fire by obtaining the characteristics of the temporal change through different color channels matched at different times to separate the flame or smoke from the background ( 300),
The image analysis device 300,
To separate the flame or smoke from the background, we use a CNN with SRGB (Sequential RGB) as the detection algorithm.
The image of the SRGB,
(t) is made by combining the red channel of the frame (red channel), the green channel of the (t-1) frame and the blue channel of the (t-2),
The fine adjustment step (S400),
Pan, tilt, and zoom control of the photographing unit 100 to analyze the image of the enlarged fire area, the first detection step (S200) and the second detection step (S300),
If the probability of the fire detected in the second detection step S300 is greater than or equal to a threshold value, the pan and tilt of the PTZ camera of the photographing unit 100 are adjusted to the center point of the boundary (BoundBox) of the corresponding area and is proportional to the size of the boundary (BoundBox). After the zoom in (zoom in) to shoot, the photographing unit 100 performs the first detection step (S200) again based on the enlarged image and transmits the captured image to the image analysis device 300, the image analysis Apparatus 300 fire detection method using a PTZ camera to re-receive the transmitted image to the deep learning of the second detection step (S300). The method according to claim 1,
The photographing unit 100 is a fire detection method using a PTZ camera including a single PTZ camera. delete delete delete The method according to claim 1,
The photographing unit 100,
Analyzing the captured image by dividing the image into the NxN area in detail, but N is divided to 6 to split the screen horizontally and vertically. delete delete

Images