KR102507540B1 - Integrated fire monitoring system based on artificial intelligence - Google Patents

Abstract

This technology relates to an artificial intelligence-based integrated fire monitoring system. The artificial intelligence-based integrated fire monitoring system of this technology includes a server; and a visible light image sensor collecting image data of the monitoring area, a thermal image sensor collecting temperature data of the monitoring area, and a light receiving sensor collecting infrared or ultraviolet data of the monitoring area. a plurality of fire detection devices; wherein each of the fire detection devices includes a first determination unit determining that a first event occurs when a predetermined condition is satisfied from the image data; a second determination unit determining that a second event occurs when a predetermined condition is satisfied from the temperature data; a third determination unit determining that a third event occurs when a predetermined condition is satisfied from the infrared or ultraviolet data; An overlapping unit for overlapping the visible light image (first image) and a thermal image (second image) generated from temperature data that satisfies a predetermined condition among the temperature data when at least one of the first to third events occurs. ; and a communication unit transmitting the overlapped image to the server. The artificial intelligence-based integrated fire monitoring system of this technology can provide a device that can increase fire detection efficiency by more accurately determining a non-fire situation in an edge-type detection system.

Description

Integrated fire monitoring system based on artificial intelligence}

The present invention relates to an artificial intelligence-based integrated fire monitoring system, and more particularly, to an artificial intelligence-based integrated fire monitoring system to which UV sensing technology is applied.

CCTV (Closed Circuit Television) is a system that transmits an image to a specific receiver in a specific building or facility using a wired or special wireless transmission line.

CCTV systems include a centralized monitoring system and an edge monitoring system. The centralized type simply collects images from cameras at a central center and analyzes them in batches. In the edge type, a video analysis function is configured in each camera itself, and surveillance video is transmitted only when a specific event occurs. The latter is one of the methods that have been widely used and studied recently because it can distribute the load of the central center and greatly reduce the total amount of traffic.

In addition, in order to increase the accuracy of video surveillance, research on CCTV, which is applied with a thermal imaging camera as well as a visible light camera, is also active.

In addition to this, research to increase the accuracy of video surveillance by utilizing UV sensors is also active. Korean Registration Publication No. 10-2088198 shows a technology for real-time detection of a fire by integrating a collection of visible light images and thermal images and a function of detecting ultraviolet rays into one monitoring device.

The more data collected, the higher the accuracy of video surveillance. However, there is a trade-off. Not only the cost of the equipment itself increases, but also the amount of data to be processed greatly increases, which increases the cost of building and maintaining the system, and reveals the limit of miniaturization.

This is why a system that can be built at low cost while using various types of information while reducing the total amount of traffic is required. Efforts are needed to design systems more efficiently under limited resources.

Meanwhile, the above-described edge-type detection system transmits the event to the server only when an event occurs at the edge in order to reduce server load. This has the advantage of reducing the load of the server and reducing the workload of the manager (or the worker in the control room). However, it is difficult to enjoy the above advantages when a continuous flame detection event occurs, such as in a welding operation. In the case of welding work, since it lasts for a long time, even with an edge-type monitoring system, a load is applied to the server continuously during that time, and a problem arises in that a manager must also continuously concentrate on the screen. This in turn exposes vulnerabilities in fire detection.

Accordingly, in a situation where there is a high possibility of false fire detection, the operation of the fire detection device may be temporarily blocked. Although fire detection is more actively required in case of high risk of fire, such as welding work, in reality, the problems occurring in the existing centralized monitoring still exist in the edge type as it is.

The inventor of the present invention has completed the present invention after long research and trial and error in order to solve these problems.

An embodiment of the present invention provides an apparatus for further increasing fire detection efficiency in an edge type detection system.

Meanwhile, other unspecified objects of the present invention will be additionally considered within the scope that can be easily inferred from the following detailed description and effects thereof.

An artificial intelligence-based integrated fire monitoring system according to an embodiment of the present invention includes a server; and a visible light image sensor collecting image data of the monitoring area, a thermal image sensor collecting temperature data of the monitoring area, and a light receiving sensor collecting infrared or ultraviolet data of the monitoring area. a plurality of fire detection devices; wherein each of the fire detection devices includes a first determination unit determining that a first event occurs when a predetermined condition is satisfied from the image data; a second determination unit determining that a second event occurs when a predetermined condition is satisfied from the temperature data; a third determination unit determining that a third event occurs when a predetermined condition is satisfied from the infrared or ultraviolet data; An overlapping unit for overlapping the visible light image (first image) and a thermal image (second image) generated from temperature data that satisfies a predetermined condition among the temperature data when at least one of the first to third events occurs. ; and a communication unit transmitting the overlapped image to the server.

Each of the fire detection devices determines whether a non-fire event occurs from the overlapping image, and if it is determined to be a non-fire event, the AI blocks transmission of the overlapping image to the server and transmits a non-fire message to the server. It may further include a processing unit;

The AI processing unit may determine whether or not a non-fire event occurs in the overlapped image in which it is determined that all of the first to third events have occurred.

The AI processing unit may determine whether a non-fire event occurs when transmission of the overlapped image continues for a predetermined time or more.

The AI processing unit recognizes whether a worker exists from the overlapped image, and based on the pre-learned distance between the worker and the flame, when the distance between the operator's region recognized in the overlapped image and the region of interest is within a preset value It can be judged as a non-fire event.

The region of interest may be a high-temperature region defined by the thermal image.

The region of interest may be a flame region defined by a flame pattern generated from image data that satisfies a predetermined condition among the image data.

When two or more worker areas or two or more areas of interest exist in the overlapping image, a non-fire event may be determined based on a distance between the farthest operator area and the ROI.

When receiving the non-fire message from any one of the fire detection devices, the server may broadcast an event video transmission blocking message to other fire detection devices.

The present technology can provide a device capable of increasing fire detection efficiency by more accurately determining a non-fire situation in an edge-type detection system.

1 is a diagram showing the overall configuration of an artificial intelligence-based integrated fire monitoring system according to an embodiment of the present invention.
2 is a diagram showing the configuration of a fire detection device according to an embodiment of the present invention in more detail.
3A to 3D are diagrams schematically illustrating a process of generating an overlapping image by a fire detection device according to an embodiment of the present invention.
Figure 4a is a diagram showing a schematic situation viewed from above a monitoring area in which a fire detection system according to an embodiment of the present invention is installed.
FIG. 4B is a diagram illustrating an example in which event images (ie, overlapping images) received from each fire detection device in FIG. 4A are displayed through a display device connected to a server.
FIG. 5 is a diagram showing an image analysis process for limiting such transmission by an AI processing unit according to an embodiment of the present invention when each fire detection device transmits an event image to a server in FIG. 4 .
6 is a diagram illustrating an image analysis process of an AI processing unit according to an embodiment of the present invention in more detail.
It is revealed that the accompanying drawings are illustrated as references for understanding the technical idea of the present invention, and thereby the scope of the present invention is not limited thereto.

In the following, the most preferred embodiment of the present invention is described. In the drawings, the thickness and interval are expressed for convenience of explanation, and may be exaggerated compared to the actual physical thickness. In describing the present invention, well-known configurations irrelevant to the gist of the present invention may be omitted. In adding reference numerals to the components of each drawing, it should be noted that the same components have the same numbers as much as possible, even if they are displayed on different drawings.

1 is a diagram showing the overall configuration of an artificial intelligence-based integrated fire monitoring system according to an embodiment of the present invention.

As shown in FIG. 1, the AI-based integrated fire monitoring system 1 (hereinafter simply referred to as a 'fire detection system') includes a server 100 and a plurality of fire detection devices 200A to 200D. do.

The server 100 is connected to a plurality of fire detection devices 200A to 200D through a wired/wireless communication network.

The server 100 may receive and store various data transmitted from the plurality of fire detection devices 200A to 200D, and process the data according to a user's request.

The server may display monitoring results collected from the plurality of fire detection devices 200A to 200D through the plurality of display devices DS. The display device may display colors corresponding to a first event based on visible light data, a second event based on thermal image data, and a third event based on infrared/ultraviolet data. Patterns may be displayed instead of or together with colors, and signs such as lights, sounds, and vibrations may be displayed together.

The server includes a storage (not shown) therein. The storage may store visible light data, thermal image data, and infrared/ultraviolet data for a certain period of time. Conditions such as the type of information to be stored and the storage time may be changed in consideration of the user's purpose of use or the storage capacity of the storage.

The storage may be a Network Video Recorder (NVR).

Interface devices such as a keyboard, speaker, mouse, screen, and display device may be connected to the server. It can receive the user's intention so that the monitoring result can be changed or adjusted in response to the user's intention. Alternatively, an emergency light, a warning light, an alarm, etc. determined through a user's intention or an automated program may be performed.

In the present invention, it may be referred to as one fire detection device 200 representing the plurality of fire detection devices 200A to 200D.

Referring to part A of FIG. 1 (a front view of the fire detection device), the fire detection device 200 includes a visible light camera (VC) for obtaining a visible light image of a predetermined area in front, and a heat sensor for the predetermined area. It includes a thermal imaging camera (TC) that acquires a visual image, and a light receiving sensor (IV) that collects infrared/ultraviolet data for the predetermined area.

Image data and infrared/ultraviolet data generated by the fire detection device may be generated in a form that can be stored in a storage of a server.

The visible light camera VC includes a visible light image sensor.

The visible light image sensor includes a form in which minute pixels composed of a light conversion semiconductor and a coupling device are finely integrated, and each pixel can accumulate electric charges generated by light energy through a lens and transmit them. In the present invention, for convenience of description, it will be described that a device for obtaining a visible light image is a visible light camera, but the present invention is not limited thereto, and a video camera including a visible light image sensor, a dome camera, a low light camera, a zoom camera, It may be a variety of devices such as a web camera.

The visible light image sensor collects image data for a surveillance area that is preset or determined by a user's request.

The thermal imaging camera (TC) includes a thermal image sensor.

A thermal image sensor is a device that converts infrared rays emitted from a heat-generating object into electrical signals and displays them in colors according to their size. In the present invention, for convenience of explanation, it will be described that a device for obtaining a thermal image is a thermal imaging camera, but is not limited thereto, and may be various devices such as an infrared camera including a thermal image sensor.

The thermal image sensor collects temperature data for a monitoring area that is preset or determined by a user's request.

The light receiving sensor} (IV) includes an infrared sensor or an ultraviolet sensor. The light receiving sensor may be referred to as an infrared/ultraviolet ray sensor in the present invention.

An infrared sensor operates when the change in infrared radiation emitted from a flame exceeds a certain amount, and is a device using the principle of resonance radiation in the infrared wavelength range (eg, 4.35 um) of carbon dioxide gas molecules during combustion. It is not limited thereto, and may be various devices that detect flames by receiving infrared rays.

An ultraviolet sensor is a device that responds to ultraviolet rays of a specific wavelength range (for example, 185 nm to 260 nm) among ultraviolet rays. When a certain range of wavelengths enters the UV sensor, current is passed through photoelectron generation, and the relative intensity of the detected wavelength is measured according to the amount of current, and the flame is sensed using this relative intensity. It is not limited thereto, and may be various devices for detecting flames by receiving ultraviolet rays.

The light-receiving sensor collects infrared or ultraviolet data for a monitoring area that is previously set or determined by a user's request.

Meanwhile, in the fire detection device shown in part A of FIG. 1 , the distance between the visible light camera and the thermal imaging camera may be used to generate an overlapping image to be described later. The focal length of the visible light camera and the thermal imaging camera may be used in a mapping process in which a visible light image and a thermal image obtained from each are overlapped.

As such, the fire detection device increases the accuracy of fire detection by utilizing all of a visible light image, a thermal image, and infrared/ultraviolet rays for fire detection. For example, for some situations in which the false positive rate is high when using only the visible light image, the false positive rate for expected false positive situations (such as red gloves) is reduced when the thermal image related to temperature and infrared/ultraviolet are used together. can be significantly lowered.

To this end, the fire detection device according to an embodiment of the present invention generates a visible light image and a thermal image, transmits them to the server, and transmits infrared/ultraviolet detection results to the server. At this time, the fire detection device, as an edge-type monitoring system, determines the occurrence of an event by itself and transmits only the event image to the server.

Hereinafter, a process of transmitting an event image of the fire detection device will be described in detail.

2 is a diagram showing the configuration of a fire detection device according to an embodiment of the present invention in more detail.

As shown in FIG. 2, the fire detection device 200 includes a visible light image sensor 210, a first determination unit 220, a thermal image sensor 230, a second determination unit 240, an infrared/ultraviolet ray sensor ( 250), a third determination unit 260, an overlapping unit 270, an AI processing unit 280, and a communication unit 290.

Operations of the visible light image sensor 210, the thermal image sensor 230, and the infrared/ultraviolet ray sensor 250 are the same as those described above.

The first determination unit 220 analyzes the image data collected by the visible light image sensor 210 and determines that a first event occurs when a predetermined condition is satisfied.

For example, a first event may be generated when a flame pattern is included by analyzing a visible light image acquired by the visible light image sensor.

At this time, one or more of several techniques for recognizing a flame pattern from a visible light image may be applied. For example, an algorithm using a brightness threshold value for determining whether a flame level value of a color of the flame exceeds a threshold value may be applied. As another example, an algorithm using spatial domain analysis for analyzing the texture of the flame candidate region or the frequency component of the contour may be applied. As another example, a time domain frequency analysis algorithm for analyzing a frequency of a specific level value of a flame candidate region that changes over time may be applied. In any case, it is common in that the first event is generated based on whether or not a predetermined condition (ie, setting a threshold value and exceeding the threshold value) is satisfied to determine occurrence of an event.

Meanwhile, the first event may be generated when a smoke pattern is included instead of or together with the flame. Similarly, one or more techniques for recognizing smoke patterns from visible light images may be applied. For example, an adaptive Gaussian mixture model, a temporary accumulation technique of a temporal derivative image, and the like may be applied.

The second determination unit 240 analyzes the temperature data collected by the thermal image sensor 230 and determines that a second event occurs when a predetermined condition is satisfied.

For example, when the temperature data is analyzed and a temperature value exceeding a preset temperature is included, a second event may be generated.

For example, when the threshold value is set to 50°C, and a value exceeding 50°C exists in temperature data collected by the thermal image sensor, a second event is generated.

In addition, an event may be generated when there is even one value exceeding the threshold in the temperature data, but an event may be generated only when two or more neighboring values exist. It can be changed according to the purpose of use or use environment of the fire detection device.

On the other hand, the thermal image sensor collects temperature data and converts it into color data (i.e., thermal image), and uses the color data after conversion (i.e., determines whether color values that exceed the threshold value exist in the thermal image). It is also possible to determine the occurrence of the second event), but in the present invention, an embodiment in which temperature data before conversion is utilized will be mainly described.

The third determination unit 260 analyzes the infrared or ultraviolet data collected by the infrared/ultraviolet sensor 250 and determines that a third event occurs when a predetermined condition is satisfied.

For example, a third event may be generated when infrared data in a predetermined wavelength range is greater than or equal to a predetermined amount by analyzing infrared data. For example, if the amount of light received in the 4.35 um band is greater than or equal to a threshold value, a third event may be generated.

Alternatively, for example, a third event may be generated when ultraviolet light in a predetermined wavelength range is greater than or equal to a predetermined amount by analyzing ultraviolet light data. For example, when the amount of light received in the 185 nm to 260 nm band is greater than or equal to a threshold value, a third event may be generated.

Meanwhile, the infrared/ultraviolet sensor may use both infrared data and ultraviolet data, but in the present invention, for convenience of description, an embodiment using only ultraviolet data will be mainly described. In addition, in the present invention, for convenience of explanation, it is expressed as collecting or storing infrared data or ultraviolet data, but the light receiving sensor collects these data and converts them into current values, and what is actually collected or stored may be current values. .

The overlapping unit 270 overlaps the visible light image obtained by the visible light camera VC and the thermal image obtained by the thermal imaging camera TC when at least one of the first to third events occurs.

Transmission of the superimposed image to the server 100 may be performed by the communication unit 290 . The fire detection device further includes a communication unit for communicating with the server.

An overlapping process of the overlapping unit according to an embodiment of the present invention may include a mapping process of overlapping the first layer of the visible light image and the second layer of the thermal image.

At this time, the first layer of the visible light image (hereinafter simply referred to as 'first layer') is a visible light image for a predetermined area in the front, while the second layer of the thermal image (hereinafter simply referred to as 'second layer') ) is a thermal image from which only some temperature values (ie, temperature values exceeding a threshold value) for a predetermined area in front are extracted.

Specifically, the second layer is overlapped with the first layer after an extraction process is performed only on a portion exceeding a threshold before overlapping with the first layer.

According to an embodiment of the present invention, the threshold value for the second layer and the above-described threshold value for determining occurrence of the second event may be set differently. This is because the process of determining the occurrence of the second event and the process of creating the second layer correspond to separate processes. This is because event occurrence determination is related to the transmission start point of the monitoring image, and generation of the second layer is related to the monitoring image to be actually provided to the manager.

For example, the threshold for the second layer may be lower than the threshold for determining the occurrence of the second event. For example, the former may be 30°C and the latter may be 50°C. By setting the second layer threshold to be relatively low, actual monitoring by the manager (which is performed while the manager visually checks through the display device) is performed more accurately at a lower temperature.

According to an embodiment of the present invention, the extraction process includes a process of converting only temperature data exceeding a threshold value into color data. That is, after only the part where the second event has occurred is extracted, it becomes the second layer. If there is no part where the second event has occurred, since there is nothing to be extracted, zero data can be the second layer. Alternatively, the zero data may include data representing a transparent image.

Figures 3a to 3d schematically show this process.

FIG. 3A relates to a situation in which all events occur, and FIGS. 3B to 3D relate to a situation in which only one of the first to third events occurs sequentially.

As shown in FIG. 3A, the first to third events occur due to the welding operation in a predetermined area in front (for example, the heat recovery boiler installation area inside the thermal power plant), and the first to third events of the visible light image for this occur. The layer L1 and the second layer L2 of the thermal image may be overlapped and transmitted to the server. The server manager can quickly determine whether or not a fire has occurred through a thermal image as well as a visible light image by looking at the overlapped image OL through the display device.

In the drawings, the flame region is referred to as WF, and the high temperature region is referred to as T. In the overlapping image of FIG. 3A , the flame area WF and the high temperature area T generally coincide.

As shown in FIG. 3B, a first event occurs due to a welding operation in a predetermined area in front, and the first layer L1 of the visible light image and the second layer L2 of the thermal image are overlapped with respect to the welding operation. can be sent to the server. Similarly, the server manager can quickly determine whether or not a fire has occurred through a thermal image as well as a visible light image by looking at the overlapped image OL through the display device.

At this time, note that the second layer L2 of the thermal image is zero data. This is because the second event did not occur. When detecting a fire, it may be assumed that the second event does not occur due to various circumstances. For example, since a thermal imaging camera has a predetermined decomposition distance for each device, if the distance is too long, there may be no temperature data that satisfies a predetermined condition. As such, even though only the first event occurs, an overlapping image is generated and transmitted to the server so that the server manager can determine whether a fire has occurred.

In the drawings, the flame region is referred to by the reference WF. In the drawing, since there is no high-temperature region that satisfies the predetermined condition, there is no high-temperature region in the overlapping image OL.

Next, as shown in FIG. 3C, a second event occurred due to a welding operation in a predetermined area in front, and a first layer L1 of a visible light image and a second layer L2 of a thermal image for this event occurred. may be overlapped and transmitted to the server. Note that, as in the case of FIG. 3B described above, even though only one of the three events occurs, the overlapping image OL is generated and transmitted to the server. The server manager can quickly determine the occurrence of a fire through the display device through heat detection that is not revealed from the visible light image.

In FIG. 3C , the flame region is not separately displayed in the visible light image, and the high temperature region exists in the thermal image, and only the high temperature region exists without the flame region in the overlapped image OL. This is because the first event did not occur. When detecting a fire, it may be assumed that the first event does not occur due to various circumstances. For example, in the case of a visible light camera, since each device has a predetermined decomposition distance, if the distance is too long, there may be no image data that satisfies the predetermined condition. As such, even though only the second event occurs, an overlapping image is generated and transmitted to the server so that the server manager can determine whether a fire has occurred.

And, as shown in FIG. 3D, a third event occurred due to the welding operation in a predetermined area in front, and the first layer L1 of the visible light image and the second layer L2 of the thermal image for this event occurred. It can be nested and sent to the server. Note that, as in the case of FIGS. 3B and 3C described above, even though only one of the three events occurs, the overlapping image OL is generated and transmitted to the server. The server administrator can quickly determine the occurrence of a fire through the display device through infrared/ultraviolet detection that is not revealed from visible light images or thermal images.

In FIG. 3D , since the flame region is not separately displayed in the visible light image and the high temperature region is not present in the thermal image, neither the flame region nor the high temperature region exists in the overlapped image OL. This is because the first and second events did not occur. When detecting a fire, it may be assumed that the first event and the second event do not occur due to various circumstances. As described above, this is because there is a respective resolution distance. As such, even though only the third event occurs, an overlapping image is generated and transmitted to the server so that the server administrator can visually check whether or not a fire has occurred.

As the event occurs as described above, the image is transmitted from the fire detection device end to the server end. Hereinafter, a process of limiting transmission of an image will be described.

Since the edge-type monitoring system is intended to distribute the load of the central center and reduce the total amount of traffic, it is necessary to set a processing procedure for a situation with a very high false positive probability, such as a welding operation.

Figure 4a is a diagram showing a schematic situation viewed from above a monitoring area in which a fire detection system according to an embodiment of the present invention is installed.

FIG. 4B is a diagram illustrating an example in which event images (ie, overlapping images) received from each fire detection device in FIG. 4A are displayed through a display device connected to a server.

And, FIG. 5 is a diagram showing an image analysis process for limiting such transmission by an AI processing unit according to an embodiment of the present invention when each fire detection device transmits an event image to a server in FIG. 4 .

First, referring to FIG. 4A , four fire detection devices 200A to 200D are installed in a predetermined monitoring area. The predetermined monitoring area may be a heat recovery boiler installation area inside a thermal power plant. For convenience of description, it is assumed that there are four fire detection devices, but the present invention is not limited to the number.

Each fire detection device transmits an overlapping image to the server 100 when at least one of the first to third events occurs while monitoring a front area (surveillable area indicated by a dotted line). As a result, the surveillance image as shown in FIG. 5 is displayed through the display device (Monitor).

The first fire detection device 200A, the second fire detection device 200B, the third fire detection device 200C, and the fourth fire detection device 200D are sequentially referred to from the fire detection device at the upper left in the drawing. Referring to the first heat recovery boiler (B1), the second heat recovery boiler (B2), the third heat recovery boiler (B3), and the fourth heat recovery boiler (B4) in order from the heat recovery boiler at the upper left in the drawing.

When an event occurs, the event image transmitted by the first fire detection device 200A is displayed as channel 1 (CH1) on the display device. In order, the event image transmitted by the second fire detection device (200B) is channel 2 (CH2), the event image transmitted by the third fire detection device (200C) is channel 3 (CH3), and the 24th fire The event image transmitted by the sensing device 200D is referred to as channel 4 (CH4).

As shown in FIG. 4 , as the welding operation proceeds in the third heat recovery boiler B3, the first fire detection device 200A detecting the first to third events determines the flame area WF and the high temperature The event image including the region (T) is sent through channel 1 (CH1). Relatively, the second fire detection device, the third fire detection device, and the fourth fire detection device did not detect the first event, but detected the second event, thereby transmitting the event image including the high temperature region T to each channel. sent through the fields (CH2 to CH4).

Although the position or size of flames caused by welding can appear in a variety of ways, in the present invention, to assume a situation in which flames due to welding are generated on one side of the third heat recovery boiler (B3) on the drawing and at a midpoint in terms of height do it with

On the other hand, as at least one of the first to third events is detected, each channel transmits an event image, but from the server manager's point of view, who has to continuously look at the display device (Monitor) for a long time, it is easy to miss the determination of whether a fire has occurred. this happens

That is, according to the characteristics of the edge-type surveillance system, only the video where the event occurred is sent instead of all the video, but the burden of having to judge the situation by viewing all the video is put on. In particular, in the case of the welding work as shown, fatigue of the server manager is accumulated as it is continuously transmitted as an event image accompanied by flames.

In order to increase the accuracy of fire detection, when any one of the first to third events occurs, it is determined that an event has occurred, but supplementarily, it is necessary to process it as an image rather than an event. According to an embodiment of the present invention, the AI processing unit may perform this. will be described later

Referring to FIG. 5 , the AI processing unit 280 determines whether there is a non-fire event from the overlapped image OL. The AI processing unit 280 analyzes the overlapping image and blocks transmission of the overlapping image to the server when it is determined to be a non-fire event and transmits a non-fire message to the server.

The AI processing unit 280 analyzes the overlapped image OL, and sets the overlapped image as the target for which it is determined that all of the first to third events have occurred. This is because the process by the AI processing unit is a process for actively processing the event image as not to be transmitted, and thus it is necessary to make a more conservative judgment.

The AI processing unit 280 operates at a predetermined point in time.

For example, it may operate when transmission of overlapping images continues for a predetermined period of time or longer. The predetermined time may be, for example, 5 minutes, which is a time point at which a person's concentration on one object decreases.

When it starts operating, it has the following non-fire determination process.

A welding operator is recognized in the overlapping image in which all of the first to third events are determined to have occurred (S1).

Various known algorithms for recognizing a welding operator from image data may be applied.

In addition, as a safety protective equipment during welding, welding mask recognition can be performed together to improve the accuracy of worker recognition. A process of comparing the color information of the safety protective equipment with the DB may be performed.

In addition, a step of improving recognition accuracy through artificial intelligence learning for recognizing a welding worker from image data may be performed. Artificial intelligence learning can be performed in a server, and the fire detection device can perform welding operator recognition by transmitting the learning result from the server end to the fire detection device end.

A welding operator recognized from the overlapping image may be referred to as a welding operator area W.

Based on the previously learned distance between the welding operator and the welding flame, it is determined whether the distance between the region of interest and the region of interest recognized in the overlapping image is within a preset value (S2).

The welding operator's area W is a part recognized in the above step S1. In the drawing, the center of the welding operator's area W is referenced by reference numeral W_C. Here, the center may mean the center of a frame (center of a pixel position). Various image processing algorithms for deriving the center of a predetermined area may be applied.

The region of interest is a high temperature region T defined by the thermal image. In the drawing, the center of the high-temperature region T is referred to as reference numeral T_C.

Alternatively, the region of interest is a flame region WF defined by the flame pattern of the visible light image. In the drawings, the center of the flame area WF is referred to as WF_C. The flame pattern may be generated from image data that satisfies a predetermined condition among image data. This is similar to the process of determining the occurrence of the first event described above, but the threshold value for generating the flame area and the threshold value for determining the occurrence of the first event may be set differently. This is because the process of determining the occurrence of the first event and the process of generating the flame area correspond to separate processes. This is because event occurrence determination is related to the start point of transmission of surveillance images, and flame area generation is related to surveillance images to be actually provided to a manager. In FIG. 5, it is assumed that the center of the high-temperature region (T_C) and the center of the flame region (WF_C) are the same in the overlapped image.

The distance between the welding operator's area and the area of interest is referred to in the drawing as reference number d. The distance may be the shortest distance connecting the center of each area.

In a normal welding operation, a process of learning the distance between the welding operator and the welding flame may be performed. Similarly, learning may be performed in a server, and the fire detection device may perform step S2 by transmitting the result of learning from the server to the fire detection device.

The reason learning is necessary is that the distance varies according to the size of the screen (frame) or the ratio of the welding worker (or welding flame) occupying the screen. For example, if the fire detection device is a CCTV, it is necessary to learn the distance between the welding operator and the welding flame that appears to match the size (resolution, etc.) of the image that the CCTV can generate, and also the welding operator occupied in the image ( or the welding flame) (that is, depending on how close the picture was taken), it is necessary to learn the distance between the welding operator and the welding flame. For example, in a 1920x1080 frame provided by HD-level CCTV, when a welding worker occupies a ratio of about 1/50 of the entire frame, the distance corresponding to 100 pixels is the distance between the welding worker and the welding flame in normal welding work. It learns from the distance, and it can be judged whether or not it is satisfied based on this.

Step (S2) is performed based on the previously learned distance between the welding operator and the welding flame.

As a result of the determination in step (S2), if it is within the preset value, it is determined as a non-fire event and transmission to the server is blocked (S3).

That is, although it is determined as an event video, it is determined as a non-fire event so that the event video is not transmitted to the server.

Since it corresponds to the distance between the welding operator and the welding flame in a normal welding operation, it is determined that the event image is a non-fire event.

Conversely, if it exceeds the predetermined value, it corresponds to a situation other than a normal welding operation, so that the transmission of the event image continues.

When the transmission of the event video is blocked as a non-fire event, a non-fire message is transmitted to the server (S4).

Accordingly, the event image is no longer displayed through CH1.

In this case, the server receiving the non-fire message broadcasts a non-fire message (ie, event video transmission blocking message) to other fire detection devices installed in the same monitoring area, so that the remaining channels (CH2 to CH4) are no longer The event image may not be displayed. For broadcasting, the above-described communication channel for receiving the event video may be utilized.

However, this is a measure to limit the transmission of the event video, and if it is determined that a new event has occurred, the restriction measure is immediately lifted and transmission can be resumed. For example, the second fire detection device that stopped transmitting the event video by receiving a non-fire message from the server while sending the event video through CH2 is a first event or a second event different from the second event determined to have occurred previously. 3 If it is determined that an event has occurred, the interruption action can be canceled and transmission can be resumed according to the process described above.

Hereinafter, referring to FIG. 6 , a processing process of the AI processing unit when a plurality of regions of interest exist in an overlapping image will be described.

6 is a diagram illustrating an image analysis process of an AI processing unit according to an embodiment of the present invention in more detail.

Referring to FIG. 6 , a case in which the center of the high-temperature region T_C and the center of the flame region WF_C are different in the overlapped image is illustrated.

The distance between the welding operator's area W and the center of the flame area WF_C is referred to as d1. The distance between the welding operator's region W and the center of the high-temperature region T_C is referred to as d2. It is shown as d1<d2.

In this case, it is preferable to perform the above-described non-fire determination process based on a region of interest farther from the region of the welding operator. That is, based on d2, it is determined whether it is within the pre-learned distance. Since the non-fire determination process is a process of actively processing an event video as a non-fire event, this is to make a judgment as conservative as possible.

Continuing to refer to FIG. 6 , another high-temperature region T' and the center of the high-temperature region T'_C are shown in the overlapping image. It could be welding work in another invisible place, or it could be a fire situation. The distance between the welding operator's region (W) and the center of the high-temperature region (T'_C) is referred to as d3. It is shown as d1<d3.

In this case, similarly, the above-described non-fire determination process may be performed based on a region of interest farther from the region of the welding operator. That is, based on d3, it is determined whether it is within the pre-learned distance.

Although not shown in the drawings, the number of welding worker regions may be two or more, and even in this case, the above-described process may be performed based on the distance between the farthest welding worker region and the region of interest.

Embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. Examples of program instructions such as magneto-optical, ROM, RAM, flash memory, etc. can be executed by a computer using an interpreter, etc. as well as machine code generated by a compiler. contains high-level language code. The hardware device described above may be configured to operate as at least one software module to perform the operation of one embodiment of the present invention, and vice versa.

As described above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions. Therefore, the spirit of the present invention should not be limited to the described embodiments, and it will be said that not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the present invention. .

1: Smart fire detection system
100: server
200, 200A, 200B, 200C, 200D: Fire detection device
210: visible light image sensor
220: first judgment unit
230: thermal image sensor
240: second judgment unit
250: infrared / ultraviolet sensor
260: third judgment unit
270: overlapping part
280: AI processing unit
290: Ministry of Communications

Claims (9)

As an artificial intelligence-based integrated fire monitoring system,
server; and
A visible light image sensor that collects image data for the surveillance area, a thermal image sensor that collects temperature data for the surveillance area, and a light-receiving sensor that collects infrared or ultraviolet data for the surveillance area. A plurality of fire detection devices; including,
Each of the fire detection devices,
a first determination unit determining that a first event has occurred when a predetermined condition is satisfied from the image data;
a second determination unit determining that a second event occurs when a predetermined condition is satisfied from the temperature data;
a third determination unit determining that a third event occurs when a predetermined condition is satisfied from the infrared or ultraviolet data;
When at least one of the first to third events occurs, a visible light image (first image) and a thermal image (second image) generated from temperature data satisfying a predetermined condition among the temperature data are overlapped to form an overlapping image. overlapping parts to create; and
And a communication unit for transmitting the overlapping image to the server,
Each of the fire detection devices,
An AI processing unit that determines whether a non-fire event is determined from the overlapped image, and if it is determined to be a non-fire event, blocks transmission of the overlapped image to the server and transmits a non-fire message to the server;
The AI processing unit,
Determining whether a non-fire event exists for the overlapping image in which it is determined that all of the first to third events have occurred;
The AI processing unit,
When the transmission of the overlapped image continues for more than a predetermined time, it is determined whether or not a non-fire event occurs,
The AI processing unit,
Recognizes whether a worker exists from the overlapping image, and determines a non-fire event when the distance between the operator's area recognized in the overlapping image and the area of interest is within a preset value based on the previously learned distance between the operator and the flame. and -The region of interest includes a high-temperature region defined by the thermal image and a flame region defined by a flame pattern generated from image data that satisfies a predetermined condition among the image data-;
The AI processing unit, when the center of the high-temperature region and the center of the flame region are different from each other in the overlapping image, the longer distance between the distance from the center of the worker region to the center of the high-temperature region and the distance to the center of the flame region. Artificial intelligence-based integrated fire monitoring system, characterized in that for determining whether the non-fire event by determining whether it is within the predetermined value based on. delete delete delete delete delete delete According to claim 1,
If there are two or more operator areas or two or more areas of interest in the overlapped image, a non-fire event is determined based on the distance between the farthest operator area and the area of interest. surveillance system. According to claim 1,
The server,
When the non-fire message is received from any one of the fire detection devices, an artificial intelligence-based integrated fire monitoring system, characterized in that for broadcasting an event video transmission blocking message to other fire detection devices.

Images