KR20190079047A - A supporting system and method that assist partial inspections of suspicious objects in cctv video streams by using multi-level object recognition technology to reduce workload of human-eye based inspectors - Google Patents

Abstract

According to the present invention, a system for supporting partial inspection of a suspicious object in a CCTV image using an image recognition technology for each level to reduce a load of a naked-eye recognition inspector comprises: an external source to provide notified open data in a preset format; a CCTV system device to input image information including an event in real time; and an image recognition support device to input open data of the external source and real-time image data from the CCTV system device to support image recognition. The image recognition support device includes: a data input mechanism; an artificial intelligent mechanism including a neural network artificial intelligence generation unit to generate essence neural network intelligence having a function of accumulating reference information to distinguish elementary image recognition information, intermediate image recognition information, and advanced image recognition information to recognize a target element; and an essence data providing mechanism to use the essence neural network intelligence to automatically recognize an occurrence of an event and a target element associated with the corresponding event in a real-time CCTV image inputted from a real-time data input unit to provide essence data which is at least one among the elementary image recognition information, the intermediate image recognition information, and the advanced image recognition information from recognition results. Therefore, inspection manpower required for an elementary image recognition function and an intermediate image recognition function can be reduced, and manpower can be concentrated on an advanced image recognition function to improve both efficiency and qualitative aspects.

Description

Field of the Invention < RTI ID = 0.0 > [1] < / RTI > A system and method for supporting suspicious object part inspection in a CCTV image using a class- TO REDUCE WORKLOAD OF HUMAN-EYE BASED INSPECTORS}

The present invention relates to a system and a method for supporting suspicious object part inspection in a CCTV image by using a gradation image recognition technology for reducing the load of a visual recognition surveiller using moving object detection technology, and more particularly, By using machine learning technology, it is possible to record the meaningful specific scenes such as the movement of specific objects among all the images to be inspected and to allow the visual inspection person to perform only the partial inspection, thereby reducing the manpower required for the complete inspection The present invention relates to a system and method for supporting a suspicious object part inspection in a CCTV image using a class-based image recognition technology for reducing the load of a visual recognition sensor.

The terms used in the present invention are as follows.

Target element (or target or target image): Information about the result of all the image object activities that are useful for discriminating the image object to be inspected, the change status information such as stop and movement of the image object, and other events and accidents.

Environment element: Information excluding objects except for the target, actions and results of the objects, objects that are stopped or moving around the target, shadows of the surrounding environment due to temporal and physical changes, and the like.

MPEG4 is an international standard for video and audio compression technology. It has a low transmission rate of 64 kbps or less, a medium transmission rate of 4 to 384 kbps, and a high transmission rate of 384 to 4 Mbps. MPEG4 supports both constant bit rate (CBR) and variable bit rate (VBR).

Real-time data: Real-time image data coming into the system from the CCTV screen.

Open Source: Pre-recorded video, moving picture, analysis data necessary for learning of vehicle and pedestrian object recognition system from external sources.

DB archive data: This data contains both open data and real-time data entered into the system and stored in the DB, and corresponds to data required for artificial intelligence learning or CCTV image processing or recognition.

Events: Concepts covering events, events, events, and the resulting material and phenomenal changes.

Cascading Software Organization: It is a structure in which software subsystems that process detection process, detection work including decoding work, convolution work, pooling work, and tracking work in sequential connection process of preprocessing process or essence data providing process, The result is the form that acts as input to the next subsystem. A concept that allows bottlenecks to be avoided by allowing video streams to be processed differently and alternately by alternate control mechanisms.

Casking processing: Tasks performed using a cascading software architecture.

Convolution work: One way of sliding windows to find local patterns. Mainly used in deep running.

Pooling: A process for determining the existence of a pattern.

Feature Map: Indicates similarity with local pattern in image.

Detection: A step in the image processing technique in which a meaningful object appears in the input image and searches for it. It differs from a simple method of detecting the appearance of a specific object by comparing the difference between the front and back screens, and a method of detecting through deep running.

Tracking: When an object is recognized in the image recognition system and then the same object moves on the same screen, it is judged that the recognized object is moving and it can be traced

Intelligent IP Camera: In addition to the usual analog CCTV camera system technology, an intelligent IP camera (part of the configuration included in the CCTV system) equipped with a camera embedded embedded computer converts the input analog image into digital, After compression is performed by the encoding technique, the output is finally output through the IP communication. In particular, IP cameras are features that utilize IP-based communication networks and are powered by Power over Ethernet (PoE). Furthermore, some IP cameras include advanced intelligent CCTV cameras that process only the image recognition such as motion detection, vehicle detection, and pedestrian detection on the built-in computer, and output only the motion information on the screen to the external output. At this level, algorithms and hardware designs are integrated to enable real-time image analysis. According to the present invention, at this level, basic functions of image analysis are mounted on an embedded computer in an IP camera, and the detected information is sent to the server, so that the server can solve You can realize a way to handle the problems you need to do.

CCTV system: The concept that encompasses intelligent IP CCTV, IP CCTV, general CCTV, and CCTV. It is a concept covering all the accessories that are related to CCTV and used or supplemented to CCTV while maintaining or enhancing the functions of CCTV. Is used. Here, intelligent IP CCTV or IP CCTV is used as a concept including an intelligent IP camera, and when it is expressed simply as CCTV or general CCTV, it is used as a concept including a general CCTV camera rather than an intelligent IP camera.

Elementary Image Recognition Level: Visual event modeling and algorithms that reliably detect, track, and classify objects, and in particular image recognition levels such as motion detection for unusual conditions. The beginner level of image recognition level acts as a bridge to develop from "signal processing level" (eg brightness, color, motion change, etc.) to "semantic" level (eg traffic accident detection, terrorism detection, etc.). By analyzing the information with high semantic complexity in the image hierarchically, the image analysis ability becomes bigger and the ambiguity decreases, and the beginner image recognition level corresponds to the lower level.

Intermediate Image Recognition Level: Assuming that only the motion detection at the beginner image recognition level is processed, at the intermediate level, the process of handling traffic accident detection, terror detection, etc. at a level of semantic level for one CCTV camera image at a more complex image recognition level .

Advanced Image Recognition Levels: This advanced level of image recognition is an evolutionary step from analyzing information input from multiple camera networks rather than a single CCTV camera. In other words, it is possible to track escape vehicles by interworking image processing between CCTV cameras installed in various intersections. This is achieved by maximizing information by multiple observers for image analysis data fusion model and one same visual phenomenon, It is to solve the problem connected with the camera network.

Conventionally, it is installed for the purpose of policing, actual condition check by each local self-government. The CCTV is mainly installed on major arterial roads, back roads and alleyways. The size of each CCTV is roughly 1,500 units in each jurisdiction area. In general, the output image is 24 hours per day Video is stored in a video server called VMS (Video Management System). In this way, most of the stored images are dependent on the naked eye of the surveillant at the present time for inspecting meaningful images relating to specific events, accident related images, unknown unknown events, and accidents to the level of the beginner image recognition described in the definition of terms. Therefore, it takes a lot of time and labor to inspect the vast amount of CCTV video at the judgment of the inspector.

As a CCTV image recognition technology that assists the inspecting work of the inspector, there is, for example, Patent Application Publication No. 1997-0014321 (first patent document). In the case of the first patent document, when a camera signal photographed through a plurality of surveillance cameras is recorded on one recording tape, an identification code is generated for each event such as an alarm, a video loss, and an intruder, The identification code for the date and time of the viewpoint is recorded in the vertical retrace time of the camera signal, the identification code recorded at the time of image playback is detected, and the detected case is compared with the event and time zone set by the user, By reproducing the camera signal in which the identification code is recorded, the inspection time is shortened. Particularly, in the case of the first patent document, a technology for finding a meaningful object or target through movement or change of an object or the like by using the difference between frames is proposed.

In the case of the first patent document, when an identification code corresponding to a significant image target is specified in advance, a vast amount of CCTV images are visually inspected by the naked eye, and the visual inspection of the significant target is judged by the inspector himself / herself, So that a great amount of labor and time are inevitably needed. In addition, because CCTV images are accumulated continuously, it is almost impossible to find out all the meaningful data in real time without mistakes by continuously repeating the process of finding such a visual inspection in advance and preparing a corresponding identification code. In addition, since the frame switching method simply reacts to the movement or change of the object, it can be applied to all the environmental changes such as the motion of the object due to the wind, the temporal and spatial variation of the shade, the shaking of the meaningless tree, And the result is that the means of separating the resulting information into meaningful and insignificant information is a classification through human eyes, so that much labor is forced to be added to this part as well.

As another conventional technique, a method of inspecting an incident or an accident corresponding to a time when a CCTV image that has already been photographed is recorded in correspondence with time and is inspected is disclosed, for example, in Japanese Patent Application Laid-Open No. 10-2017-0096838 ≪ / RTI >

In the case of the second patent document, it is not necessary to perform all visual inspection of the identification code in advance in view of recording and storing the CCTV image by time, but since the reference code is simply time only, Although it is useful for examining events corresponding to time, for example, even in a specific time zone recorded in all CCTV images corresponding to 1500 units, the amount thereof is considerably large, and in addition, As a result, there is no countermeasure against the image target. Consequently, in order to inspect the meaningful target from all recorded CCTV images, much labor and time are consumed separately as in the second patent document.

On the other hand, in the case of the patent registration No. 10-1589823 (third patent document), there is provided a display section which can be monitored for a photographed image, an inspector inputs an area of interest at a specific place among the images represented through the display section, A motion of a significant object in the specific surveillance region is displayed as an event occurrence image, and a necessary image can be quickly inspected.

In the case of the third patent document, it is possible to provide a function of inspecting an event occurring in a specific local area or region among a vast area, and it is useful in the security dimension for a specific institution (for example, a bank, etc.) However, in many cases, incidents and accidents occur in a wide range of places and times that can not be predicted at all (for example, if a criminal escapes to an automobile after escaping from an unspecified road, or is now escaping, In addition, it is also necessary to use a large number of manpower to inspect all significant targets in relation to the past, present, and future in a vast area of CCTV images. Therefore, the first and second patents It can be said that they have the same problems in the literature.

As a result, in the case of related arts including the first to third patent documents, a CCTV image is inspected. In particular, a target image is preset in a CCTV image that has already been photographed to facilitate subsequent inspection, A lot of time and workforce are administered to the post-test. In case of the real-time test, there is a risk that a meaningful target element is always missed, and also it is difficult to respond quickly in real time.

In addition, in the prior arts such as the first to third patent documents, factors that delay the inspection include, for example, video decoding in an inspection process, object detection in video, classification in video, tracking Of software subsystems. Many CPUs, memory buffers, and GPU resources are used in the processing of these tasks. Therefore, when dozens of video streams are concurrently performed on the one subsystem at the same time by a heavy software processing process such as decoding, detection, etc., a bottleneck may occur in this part. Therefore, there is a problem in that, in order to process as many streams as possible simultaneously and in parallel on the software on a given hardware device, many bottlenecks that can occur in the data stream processing process must be solved.

Patent Application Publication No. 1997-0014321 (published on March 29, 1997) Patent Application Publication No. 10-2017-0096838 (published on Aug. 25, 2017) Patent Registration No. 10-1589823 (Bulletin issued on June 29, 2016)

The object of the present invention is to solve the problems of the prior art described above, and it is an object of the present invention to significantly reduce labor and time to inspect a target component from a plurality of CCTV images, And to provide a system and method for supporting a suspicious object part inspection in a CCTV image using a class-specific image recognition technology for reducing the load of a visual recognition sensor.

Another object of the present invention is to narrow down a portion of the entire time-of-day image of the photographed CCTV, which is unnecessary to be viewed, in order to minimize the beginner-level image recognition task described in the definition of the visual- Such as scenes in which meaningful objects such as passers-by are moving. Recognize the motion of the objects in the image, but it will be the part of the image that filters out the shaking of the wind by the wind, the paper of the waste paper, the movements of the animal, etc. The image part to be left is the image object recognition And to provide a system and method for supporting a suspicious object part inspection in a CCTV image using a class-specific image recognition technology for reducing the load of a visual recognition examiner.

Another object of the present invention is to provide a method and apparatus for simultaneously decoding a plurality of video streams in a process of video decoding in an inspection process, object detection in video, classification and tracking of video, Visual recognition that can simultaneously process the maximum number of video data streams while avoiding bottlenecks even when heavy software processing is performed simultaneously CCTV detection of suspicious objects in CCTV images using image recognition technology for load reduction And to provide a support system and method.

According to another embodiment of the present invention, when the intelligent IP CCTV camera described in the definition of the term is applied to the present invention, a technique of a basic image recognition level such as the above-described motion detection is performed directly on an IP camera, When the image has been transmitted to the server, the inspector performing the visual inspection can perform a task of complex judgment of a higher-level intermediate-level and higher-level event recognition. The present invention relates to a complex event that can help the inspection personnel to perform such tasks, including face recognition, pedestrian recognition, pedestrian recognition, vehicle recognition, color recognition, and the like And a deep learning object recognition and behavior recognition system and method capable of performing the task of advanced image recognition level described in the definition of the term tracking vehicle escaping via various intersections in a server, It can serve as a system that helps to use it more effectively.

The first aspect of the present invention for achieving the above object is to provide a method and apparatus for recording a meaningful specific scene including at least movement of a specific object among all the images to be inspected by using a machine learning technique, A basic image recognition function for detecting at least a motion in a strange state different from a normal state in order to provide a technique for reducing the manpower required for the inspection; Intermediate image recognition function that generates a semantic level meaning that a specific event is defined in a CCTV camera and advanced image recognition that generates intermediate interlock information from a plurality of CCTV cameras by collecting intermediate image recognition information detected according to an intermediate image recognition function Level image recognition technology to reduce the load of the visual cognition surveyor As a support system for suspicious object part in CCTV images; An external source that provides a pre-arranged open source of known open data that includes at least CCTV image information of events covering events, accidents, events, and predictable or unpredictable events occurring in the jurisdiction of the general administrative district; ; A CCTV system for inputting video information including the event in real time; And an image recognition support device for supporting image recognition by inputting open data of the external source and real-time image data from a CCTV system, wherein the image recognition support device comprises an open data inputting unit for inputting the open data from the external source, A data input mechanism including an input unit and a real-time data input unit for inputting real-time image data from a CCTV system for photographing the event in real time; Extracting and preparing an action or an action related to the event and a corresponding event and a result according to an action or an action using the open data input from the open data input unit of the input mechanism in the open data, And the result of the action or action associated with the object other than the element or the object and the temporal and natural phenomenon changes expected to occur in the road are extracted and prepared as simulation environment elements and the simulation environment element and the simulation target A simulation target learning unit for applying a learning result to a CCTV image including an event among the open data to perform a machine learning for distinguishing a target element of the event from an environmental element, The image recognition information, the intermediate image recognition information, An artificial intelligence device including an artificial intelligence generating unit for generating an essence neural network artificial intelligence having a function of recognizing a target element by itself as a reference information for discriminating a beam; The real-time data input unit receives the real-time CCTV image information using the essence-based neural network artificial intelligence and automatically recognizes the event generation and the target element related to the corresponding event, And an essence data providing mechanism for providing essence data that is at least one of the advanced image recognition information; Respectively.

Preferably, the essence data providing apparatus removes a portion of the real-time CCTV image information that is not directly related to the target element from the real-time CCTV image information before the essence data providing apparatus recognizes the target from the real- Further comprising an image preprocessing mechanism for generating preprocessing information for reducing the data amount of the real-time CCTV image information to be recognized and reduced, wherein the artificial intelligence device comprises a preprocessing element prepara- tion section for preparing a preprocessing element that is an identification element related to the preprocessing Further comprising a simulation pre-processing learning unit that mechanically learns a simulation pre-processing based on the simulation pre-processing element by an image difference or a deep learning method, and the neural network artificial intelligence generating unit generates an automatic learning algorithm based on the simulation preprocessing learning results accumulated in the simulation pre- Which can perform pre-processing Further create artificial intelligence, the image pre-processing mechanism is the essence data using the neural network pre-AI and automatically generates a pre-processing the information, is generated based on the pre-information.

Preferably, the image preprocessing mechanism includes a resolution reducing unit for reducing the resolution of the CCTV image, a color switching unit for simplifying the color type of the CCTV image, and a color conversion unit for excluding the object not related to the target in the CCTV, A color reduction unit, a color conversion unit, a required area setting unit, a frame reduction unit, and a frame reduction unit. The frame size reduction unit reduces the number of overlapping portions of frames per second in the CCTV image. And a preprocessing information generating unit for generating at least a part of the operation result as preprocessing information for image recognition.

Preferably, the elementary image recognition information includes at least a result extracted from the inspection workforce or formed directly from the IP CCTV, and the essence data providing mechanism may include at least one of the intermediate image recognition information and the advanced image recognition information At least one of them is provided as essence data of at least 80% to 99% of maturity corresponding to the exclusion rate of the inspection personnel.

Preferably, the essence data may include location information and target elements of an original image related to generation of a meaningful event and related to the corresponding event in real-time CCTV image information input from the real-time data input unit using the essence neural network artificial intelligence The metadata is stored as metadata, and if necessary, the metadata is configured to be inspectable only for the meaningful scenes, or the data is inspected.

Preferably, the essence data providing apparatus further includes an image difference processing unit for detecting a change in the front and rear CCTV images, generating a difference between the before and after images as image difference information, and notifying the generation of an event or a change in a specific area, The essence data providing mechanism reflects the image difference information to the essence metadata or the essence image or directly generates image difference information based on the preprocessing information from the preprocessing information generating unit of the image processing apparatus.

Preferably, the system further comprises a control mechanism for controlling the system as a whole, and the control mechanism is integrally or directly telecommunicationly connected to the image recognition support device, And a case processing unit for performing a cascading process for separating a decoding operation, a detection operation, a tracking operation, and the like in the CCTV image processing operation of the essence data providing apparatus, and alternately assigning the decoding operation, the detection operation, , The AI is periodically or intermittently updated according to the addition or modification of open data from the external source and real-time data provided from the CCTV system device, and environmental and target elements.

Preferably, the control mechanism includes: an input / output unit that sets an operation condition of the image preprocessing mechanism or the essence data providing mechanism, monitors the operation of the image recognition support apparatus and has a UI function, a simulation preprocessing element, A GUI part that provides a function of manually or automatically generating a part that can be prepared in advance in a shape among environmental elements and simulated target elements, and an essence data providing mechanism that performs the recognition of the CCTV image, Wherein the machine learning includes at least one of Deep Learning and Shallow learning, wherein the machine learning includes at least one of Deep Learning and Shallow learning, And a comprehensive database (DB) for storing data generated during the entire operation of the apparatus.

The second aspect of the present invention for achieving the above object is to provide a method and apparatus for recording a meaningful specific scene including at least movement of a specific object among all the images to be inspected by using a machine learning technique, A basic image recognition function for detecting at least a motion in a strange state different from a normal state to provide a technique for reducing the manpower required for the basic image recognition function, Intermediate image recognition function that generates meaning of semantics that defines a specific event in camera, Advanced image recognition function that generates intermediate interlock information from multiple CCTV cameras by collecting intermediate image recognition information detected by intermediate image recognition function Level image recognition technology to reduce load of visual cognition surveyor As my doubts with CCTV video inspection body parts support methods; The method may include providing previously open publicly known information including at least CCTV image information of events, accidents, events, and events covering predictable or unpredictable events occurring in a general administrative district jurisdiction An external source; A CCTV system for inputting video information including the event in real time; The system is used in a support system for a suspicious object visual recognition part inspection in a CCTV image using a moving object detection technology including an open source of the external source and an image recognition support device for inputting real time image data from a CCTV system device, A data input step in which the image recognition support apparatus inputs the open data from the external source and inputs real-time image data from a CCTV system which photographs the event in real time; A target element preparation step in which the image recognition support apparatus extracts, from the open data, an action or an action related to the event and a corresponding event and a result according to an action or an action thereof as a target element; An environment in which the image recognition support device extracts and prepares the temporal and natural phenomenon changes expected to occur on the road and the results of the actions or actions related to the objects or the objects other than the target element, Element preparation step; A simulated target marking learning step of performing the machine learning in which the image recognition support device applies the simulated environment element and the simulated target element to the CCTV image including the event among the open data to discriminate the target element of the event from the environmental element Wow; Wherein the neural network artificial intelligence generating unit of the image recognition support apparatus accumulates the learning result in the simulated target marking learning unit as reference information for discriminating the elementary image recognition information, the intermediate image recognition information, and the advanced image recognition information, A neural network artificial intelligence generating step of generating an essence neural network artificial intelligence having a function of recognizing a target element; The image recognition support apparatus automatically recognizes the generation of an event and a target element related to the event in real-time CCTV image information input from the real-time data input unit using the essence neural network artificial intelligence, , Essence data providing at least one of the intermediate image recognition information and the advanced image recognition information; Respectively.

Preferably, before the essence data providing step, an image preprocessing step for generating a preprocessing information for reducing the data amount of the real-time CCTV image information to be recognized by removing or reducing a portion not directly related to the target element in the real- Preparing a preprocessing element preprocessing element, which is an identification element related to the preprocessing, before the image preprocessing step; A simulation preprocessing learning step of mechanically learning the simulation preprocessing based on the simulation preprocessing element by a deep learning method; Wherein the neural network artificial intelligence generating unit generates a pre-processing neural network artificial intelligence capable of automatically performing pre-processing based on the simulation pre-processing learning results accumulated in the simulation pre-processing learning unit, The neural network artificial intelligence is used to automatically generate the pre-processing information, and the essence data is generated based on the pre-processing information.

Preferably, the image preprocessing step includes a resolution reducing sub-step of lowering the resolution of the CCTV image, a color conversion unit step of simplifying the color type of the CCTV image, a color conversion unit step of removing the object not related to the target, , A frame reduction sub-step of lowering the number of overlapping portions of frames per second in the CCTV image, and a step of reducing the resolution, a color conversion sub-step, And a preprocessing information generating sub-step of generating at least a part of the operation result of the sub-step of setting the area, the sub-step of reducing the frame, as the preprocessing information for image recognition.

Preferably, in the method, the elementary image recognition information includes at least the result extracted by the inspection personnel or the result directly formed from the IP CCTV, and the essence data providing mechanism may include the intermediate image recognition information or the advanced image At least one of the recognition information is provided as essence data of 80% to 99% of completion degree corresponding to the exclusion rate of the inspection personnel.

Preferably, in the method, the essence data may include at least one of genuine events generated from real-time data input from the real-time data input unit using the essence neural network artificial intelligence, location information of an original image related to the event, Is metadata that can be used to check only the meaningful scenes that are meta-data, if necessary, or semi-processed data that is the finished data or the final inspection is completed through the additional inspection of the inspection personnel.

Preferably, after the pre-processing step, the essence data providing apparatus senses a change in the front and rear CCTV images, generates a difference between the front and rear images as image difference information, and performs image difference processing Wherein the essence data providing mechanism reflects the image difference information with respect to the essence metadata or the essence image or uses the preprocessing information generated from the preprocessing information generating unit of the image processing apparatus as a basis To generate direct image difference information.

Preferably, in the method, the system further comprises a control mechanism for controlling the system as a whole, and the control mechanism is integrally constructed or directly telecommunicated with the image recognition assistant device, and the image preprocessing step In the essence data providing step, in the CCTV image processing operation of the image preprocessing mechanism and the essence data providing mechanism, the caching processing unit of the control mechanism separates the decoding operation of the recognition operation, the detection operation, the tracking operation, A cascading processing step of assigning video streams one by one alternately; The neural network artificial intelligence is periodically or intermittently updated according to the addition or modification of open data from the external source and real-time data provided from the CCTV system device, and environment elements and target elements.

Preferably, in the method, the system further comprises a control mechanism for controlling the system as a whole, and the control mechanism is integrally formed or directly telecommunicated with the image recognition assistant device, and the simulated target element preparation At least one of the steps of preparing the simulated environmental element, preparing the simulated preprocessing element, and preparing the simulated preprocessing element may include the steps of: setting the operating conditions of the image preprocessing step or the essence data providing step, A GUI step of manually or automatically generating a part of the simulated preprocessing element, a simulation environment element, and a simulated target element that can be prepared in advance in a shape, As the recognition of the image progresses and the essence data is provided, the appearance of the target element is performed in real time Wherein the machine learning includes at least one of Deep Learning and Shallow Learning, and wherein the comprehensive database of the image recognition support apparatus includes at least one of an overall And storing the data generated during the operation of the mobile terminal.

According to the above configuration, the present invention provides the following effects.

First of all, one of the key functions required in the market in the field of image recognition technology in the field is the ability to simultaneously process real-time video streams inputted from as many CCTVs as possible with a plurality of CPUs and GPUs mounted on one host server to be. In other words, the value of this system can vary depending on how efficiently the software that handles the functions to be processed in a given hardware device specification processes the video data stream. There may be a software system that simultaneously processes 30 CCTV video streams from a single host server, and a software system that simultaneously handles more than 50 CCTV video streams from the same host server. Of course, when processing output of the same or similar quality, a software system that processes more CCTV video streams is more valuable.

In this regard, there is a need to simplify the ability to identify and formalize environmental and target elements as well as the environmental elements that interfere with the separation of target elements. The present invention addresses this need by providing a technique capable of detecting as much as 90% or more of a specific object in a video stream as much as possible and simultaneously processing as many video data streams as possible in parallel It has a key effect that can be maximized. The specific sub-effects that support these key effects are as follows.

First, in the case of the conventional image recognition system, a discriminator or an inspector manually performed an operation of visually identifying all the images beforehand to find a meaningful target image and assigning codes thereto. In addition, this code assignment work must be performed manually by every CCTV image coming in real time. In case of real-time inspection of real-time CCTV images, since all images are required to be visually inspected by a discriminator or an inspector, not only missed or inaccurate inspection results but also labor and time are consumed.

However, in the system and method for supporting the inspection of suspicious objects in the CCTV image using the moving object sensing technology of the present invention, it is necessary to perform the pre-inspection code marking work as well as the neural network Artificial intelligence can provide an environment that can inspect or inspect meaningful targets in CCTV images by eliminating or minimizing human labor involvement. Thus, it provides advanced capabilities over the prior art in the accuracy of testing, minimization of labor force, and expediting of test results.

Second, in the case of the conventional automatic object recognition and behavior recognition system using the video object recognition, the host server processing the object requires a great amount of capacity. The reason for this is that the input image is HD-level, that is, at least 1,080 x 720 (usually 1,280 x 720) pixels or more, and occupies three bytes of memory per pixel with RGB color values.

However, since the artificial intelligence-based image preprocessing function of the present invention can greatly reduce the data amount of the processing target image, when the same host capacity is used as a reference, the image processing process can be performed several times to several times Stream processing is possible. That is, the artificial intelligence-based image preprocessing function of the present invention includes a resolution reduction method for reducing the resolution of the CCTV image, a color switching function for simplifying the color type of the CCTV image, It is possible to greatly reduce the amount of data of a search target image through a necessary area setting function for setting a relevant part as an image processing target area and a frame reduction function for reducing the number of overlapping parts among frames per second in a CCTV image.

More specifically,

a. The resolution reduction method reduces the amount of data by converting the HD class image into a compressed or reduced image of CIF (352 x 240 in NTSC format) or QCIF (176 x 120) format,

b. The color conversion function converts the color image into a monochrome image to reduce the processing data amount,

c. Since the background of a road such as a roadside in a road image is not a main concern of an image observation through a necessary area setting function, it is possible to set a discrimination area for discriminating presence or absence of a person traveling on a roadway and a person traveling on a roadway The number of bitmaps in the image to be processed through the image is significantly reduced,

d. In the frame reduction function, one image is usually composed of 30 frames per second. In the compression method such as MPEG, it is displayed by the technique of I frame, B frame, P frame, etc. In order to minimize the image processing load It is not a method of processing the entire 30 frames per second, but the processing data amount can be reduced by selecting only one to several frames per second. In addition, it has the function of discriminating the difference between the previous frame and the subsequent frame, and further the inspection capacity can be simplified or reduced.

Third, the recognition rate of most fully automatic object recognition and behavior recognition systems through the currently available image difference, deep learning, or shallow running learning machine is practically about 50% ~ 80%. However, in the case of the present invention, Since the data amount is largely reduced and the target such as a moving vehicle or a person moving by using the neural network artificial intelligence is detected, the accuracy can be further increased, and specifically, accuracy of 90% or more can be secured. Since the notation is marked with meta data or another method and the image corresponding to the marked scene is provided at a later inspection time, the partial inspection is carried out with minimal time and effort Can be completed close to perfection, and the search labor and search time for the entire image can be greatly reduced compared to the prior art.

Fourth, as described above, in the related art, a bottleneck may occur in the process of processing a data stream in order to process as many streams as possible simultaneously in parallel while software CCTV video streams are simultaneously input on a given hardware device. For example, video decoding, object detection in video, video classification, and tracking use a large number of CPUs, memory buffers, and GPU resources for processing, so dozens of video streams can be decoded, And so on. Therefore, in the present invention, decoding, detection, and tracking are performed as separate bottleneck resolution methods to separate and simultaneously perform processing for different video data streams Called cascading or pipelining technique (hereinafter referred to as " cascading process ") is used. That is, in the case of the present invention, when the video stream # 1 performs the decoding module according to the cascading process, the other video stream waits in the queue, and the video # 2 starts decoding again after the video decoding # 1 ends. On the other hand, the video # 1 processes the video detection process after the decoding ends. This way, video detection can eliminate bottlenecks in which all video streams wait for heavy software processes. If the video decoder module and the video detection module are assumed to be present, the decoder module processes the video 1 and then passes the result to the detection module. The decoder module receives the video 2 and decodes the video, And passes it to the detection module. In this case, if the detection module is a heavy processing process, the bottleneck phenomenon can be solved by putting a light processing module in the process before or after the detection module.

According to the above configuration, the present invention provides the following effects.

First of all, one of the key functions required in the market in the field of image recognition technology in the field is the ability to simultaneously process real-time video streams inputted from as many CCTVs as possible with a plurality of CPUs and GPUs mounted on one host server to be. In other words, the value of this system can vary depending on how efficiently the software that handles the functions to be processed in a given hardware device specification processes the video data stream. There may be a software system that simultaneously processes 30 CCTV video streams from a single host server, and a software system that simultaneously handles more than 50 CCTV video streams from the same host server. Of course, when processing output of the same or similar quality, a software system that processes more CCTV video streams is more valuable.

In this regard, there is a need to simplify the ability to identify and formalize environmental and target elements as well as the environmental elements that interfere with the separation of target elements. The present invention addresses this need by providing a technique capable of simultaneously processing as many image data streams as possible while maintaining accuracy of 90% or more of detection of a specific object in a video stream as much as possible It has a key effect that can be maximized. The specific sub-effects that support these key effects are as follows.

First, in the case of the conventional image recognition system, a discriminator or an inspector manually performed an operation of visually identifying all the images beforehand to find a meaningful target image and assigning codes thereto. In addition, this code assignment work must be performed manually by every CCTV image coming in real time. In case of real-time inspection of real-time CCTV images, since all images are required to be visually inspected by a discriminator or an inspector, not only missed or inaccurate inspection results but also labor and time are consumed.

However, in the system and method for supporting the inspection of suspicious objects in the CCTV image using the moving object sensing technology of the present invention, it is necessary to perform the pre-inspection code marking work as well as the neural network Artificial intelligence can provide an environment that can inspect or inspect meaningful targets in CCTV images by eliminating or minimizing human labor involvement. Thus, it provides advanced capabilities over the prior art in the accuracy of testing, minimization of labor force, and expediting of test results.

Second, in the case of the conventional automatic object recognition and behavior recognition system using the video object recognition, the host server processing the object requires a great amount of capacity. The reason for this is that the input image is HD-level, that is, at least 1,080 x 720 (usually 1,280 x 720) pixels or more, and occupies three bytes of memory per pixel with RGB color values.

However, since the artificial intelligence-based image preprocessing function of the present invention can greatly reduce the data amount of the processing target image, when the same host capacity is used as a reference, the image processing process can be performed several times to several times Stream processing is possible. That is, the artificial intelligence-based image preprocessing function of the present invention includes a resolution reduction method for reducing the resolution of the CCTV image, a color switching function for simplifying the color type of the CCTV image, It is possible to greatly reduce the amount of data of a search target image through a necessary area setting function for setting a relevant part as an image processing target area and a frame reduction function for reducing the number of overlapping parts among frames per second in a CCTV image.

More specifically,

a. The resolution reduction method reduces the amount of data by converting the HD class image into a compressed or reduced image of CIF (352 x 240 in NTSC format) or QCIF (176 x 120) format,

b. The color conversion function converts the color image into a monochrome image to reduce the processing data amount,

c. Since the background of a road such as a roadside in a road image is not a main concern of an image observation through a necessary area setting function, it is possible to set a discrimination area for discriminating presence or absence of a person traveling on a roadway and a person traveling on a roadway The number of bitmaps in the image to be processed through the image is significantly reduced,

d. In the frame reduction function, one image is usually composed of 30 frames per second. In the compression method such as MPEG, it is displayed by the technique of I frame, B frame, P frame, etc. In order to minimize the image processing load It is not a method of processing the entire 30 frames per second, but the processing data amount can be reduced by selecting only one to several frames per second. In addition, it has the function of discriminating the difference between the previous frame and the subsequent frame, and further the inspection capacity can be simplified or reduced.

Third, the recognition rate of most fully automatic object recognition and behavior recognition systems through the currently available image difference, deep learning, or shallow running learning machine is practically about 50% ~ 80%. However, in the case of the present invention, Since the data amount is largely reduced and the target such as a moving vehicle or a person moving by using the neural network artificial intelligence is detected, the accuracy can be further increased, and specifically, accuracy of 90% or more can be secured. Since the notation is marked with meta data or another method and the image corresponding to the marked scene is provided at a later inspection time, the partial inspection is carried out with minimal time and effort Can be completed close to perfection, and the search labor and search time for the entire image can be greatly reduced compared to the prior art.

Fourth, as described above, in the related art, a bottleneck may occur in the process of processing a data stream in order to process as many streams as possible simultaneously in parallel while software CCTV video streams are simultaneously input on a given hardware device. For example, video decoding, object detection in video, video classification, and tracking use a large number of CPUs, memory buffers, and GPU resources in the process, so dozens of video streams can be decoded, And so on. Therefore, in the present invention, decoding, detection, and tracking are performed as separate bottleneck resolution methods to separate and simultaneously perform processing for different video data streams Called cascading or pipelining technique (hereinafter referred to as " cascading process ") is used. That is, in the case of the present invention, when the video stream # 1 performs the decoding module according to the cascading process, the other video stream waits in the queue, and the video # 2 starts decoding again after the video decoding # 1 ends. On the other hand, the video # 1 processes the video detection process after the decoding ends. This way, video detection can eliminate bottlenecks in which all video streams wait for heavy software processes. If the video decoder module and the video detection module are assumed to be present, the decoder module processes the video 1 and then passes the result to the detection module. The decoder module receives the video 2 and decodes the video, And passes it to the detection module. In this case, if the detection module is a heavy processing process, the bottleneck phenomenon can be solved by putting a light processing module in the process before or after the detection module.

Fifth, as described above, in the conventional technology, some of the first, second, third, and fourth functions of the recently released IP cameras are directly processed on the embedded computer mounted on the IP camera, There are also some advanced cameras that output only the scene with the output, or perform basic image recognition motion recognition and so forth. In the case of such advanced IP CCTV cameras, when a basic image recognition level such as the above-described motion detection is performed directly on the intelligent IP camera and a meaningful image is transmitted to the server, Which is a deep running object that performs functions of face recognition, pedestrian recognition, pedestrian recognition, vehicle recognition, and color recognition, which can help intermediate and advanced events such as vehicle tracking of a vehicle, A recognition and behavior recognition system and method can be provided.

In other words, when IP CCTV is used, the labor required when detecting meaningful image changes at the level of image recognition is almost unnecessary. Therefore, in the server level, the personnel used at the beginner image level is dedicated to use for event recognition of intermediate or advanced . Further, when the artificial intelligence function according to the present invention performs at least one of intermediate and advanced image recognition, it is unnecessary to perform intermediate image recognition or advanced image recognition operations performed by a search agent, or intermediate image recognition operations Or a large portion of the overall load of the advanced image recognition task can be handled by the artificial intelligence function of the present invention, the intermediate image recognition workload and the advanced image recognition workload of the search agent can be greatly reduced.

FIG. 1 is a schematic diagram showing a system for supporting a suspicious object part inspection in a CCTV image using a class-specific image recognition technology for reducing the load of a visual recognition sensor according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating an operation of a system and method for supporting a suspicious object part inspection in a CCTV image using a class-specific image recognition technology for load reduction of a visual recognition surveillant according to an embodiment of the present invention.
3 is a flowchart of an image preprocessing process according to an embodiment of the present invention.
4 is a diagram illustrating a process of generating artificial intelligence based on image difference or machine learning according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a schematic view of a suspicious object part inspection support system (hereinafter, referred to as a partial inspection support system) in a CCTV image using a class-based image recognition technology for reducing the load of a visual recognition surveillant constructed according to an embodiment of the present invention FIG. Referring to FIG. 1, a partial test support system according to an exemplary embodiment of the present invention finds a target data required to identify meaningful reference data in the basic image data input from the CCTV, (4000) for generating an essence image based on the essence metadata or its essence meta data obtained by minimizing the amount of data necessary for the examination, and a plurality of CCTVs An external source 2000 capable of providing a plurality of existing information resources that can be used as machine learning materials as well as known open data, and an external source 2000 that is included in the image recognition support apparatus 4000 And a control mechanism 3000 that controls the image recognition support apparatus 4000 or the system as a whole . A plurality of CCTV system apparatuses 1000 and 1100 can be classified into a CCTV system apparatus 1000 including a general CCTV camera 1010 and an IP CCTV system apparatus 1100 including an IP CCTV camera 1110, The configuration is the same between the two. Functionally, the difference between the general CCTV system 1000 and the IP CCTV system 1100 in the present invention is that the IP CCTV system 1100 can embed an artificial intelligence capable of generating basic video recognition information on its own, The system 1000 does not have the artificial intelligence, and the basic image recognition information is outputted with the help of the inspection personnel and the artificial intelligence device 4300 described later of the present invention. However, both the IP CCTV system device 1100 and the general CCTV system device 1000 may additionally receive the assistance of the artificial intelligence device 4300 to generate basic image recognition information.

The external source 2000 represented in the present invention is generally a stationary object such as a stationary object such as a street situation, a traffic situation, a road situation, a scenery, a temporal change, a seasonal change, a meteorological change, CCTVs are installed and operated in administrative agencies, research institutes, government offices, and private institutions that can provide images and videos. Using open data provided from external sources (2000) For example, a person who is involved in an event, an accident, or an event (hereinafter referred to as an "event" with a concept covering events, accidents, events and consequently material and phenomenal changes) (Hereinafter referred to as "object") and a moving or changing scene (hereinafter, referred to as "moving object" or "moving object") of the object , "Target" is an object to be inspected and examined in many image data) through machine learning to classify the target image meaningless and meaningful in the event and accident scene by itself It is one of the features of the present invention that it generates neural network artificial intelligence that can be used.

The image recognition support apparatus 4000 includes a data input mechanism 4100 for inputting image information directly from the external source 2000 or the CCTV system 1000 and an external source data input unit In general, it is possible to learn machine learning such as shallow running and deep learning method by using learning materials that are open data or real time data (real time data input from CCTV) A preprocessing neural network capable of acquiring the ability to extract test data useful for inspecting a final target through an artificial intelligence instrument 4300 for generating artificial intelligence and essence neural network artificial intelligence, An image preprocessing unit 4400 for reducing unnecessary data in massive real-time data by using neural network artificial intelligence to reduce the amount of data required for inspection, Generation of essence metadata including an essential target based on essence neural network artificial intelligence using pre-processing information provided from the preprocessing mechanism 4400 or real-time data, or essence data providing essence images based on essence metadata And has a mechanism 4500. In addition, the image recognition support apparatus 4000 includes a comprehensive DB 4200 that stores the results of the flow, generation, and generation of all information generated in the system. Essence data provided by the essence data providing mechanism 4500 represented in the present invention includes both basic image recognition information, intermediate image recognition information, and advanced image recognition information defined in the present invention, or includes at least one of these information .

In addition, the image recognition support apparatus 4000 further includes a control mechanism 3000 that controls the partial inspection support system or the image recognition support apparatus 4000 as a whole. In another embodiment, the control mechanism 3000 may be separately provided separately from the image recognition support apparatus 4000 to externally control the image recognition support apparatus 4000 or the partial inspection support system as a whole.

First, in the image recognition support apparatus 4000, the open data input unit 4110 of the data input mechanism 4100 is an open data input unit 4110 from the external source 2000, for example, Roads, movement of people, and the like, and stores them as open data in the comprehensive DB 4200. On the other hand, the data input mechanism 4100 is a real-time data inputting unit for inputting the activity state of the traffic of the road in real time from the CCTV system 1000 disposed in the jurisdiction of a specific municipality, (4120). The real-time data from the real-time data input unit 4120 may be stored in the comprehensive database 4200, and may be directly input to the image preprocessing unit 4400 and the essence data providing unit 4500 in real time.

The artificial intelligence instrument 4300 basically provides the artificial intelligence artificial intelligence supporting the functions of the pre-processing neural network artificial intelligence supporting function of the image preprocessing mechanism 4400 and the essence data providing mechanism 4500. For this purpose, A simulation preprocessing element preparation unit 4310 for preparing a simulation preprocessing element for preparing the simulation preprocessing element and a simulation preprocessing learning unit 4320 for performing simulation preprocessing learning based on the prepared simulation preprocessing element, A mock environment element preparation unit 4330 that prepares a mock environment element including a moving object around the object, a mock target element preparation unit 4340 that includes mock object state information and movement information of the mock object, A mock target marking learning unit 4350 for searching and marking a mock target for an event such as an event, an event, an event based on the mock target element, And it may include an artificial intelligence neural generator (4360) to create a machine learning, artificial intelligence neural network capabilities in pre-treatment of artificial intelligence and artificial intelligence neural essence by the simulated target marking the learning unit (4350).

The simulated pre-processing element of the simulated preprocessing element preparation unit 4310, the simulated environment element of the simulated environment element preparation unit 4330, and the simulated target elements of the simulated target element preparation unit 4340 are stored in the open database Or real-time data, which will be described in more detail in connection with the following description of the image preprocessing mechanism 4400 or the target data providing mechanism 4600.

The image preprocessing mechanism 4400 corresponds to a function mechanism that focuses on reducing the amount of data to be recognized. For this purpose, for example, the HD resolution of the CCTV video is divided into CIF (352, based on NTSC format) , Or QCIF (176x120) class to reduce the amount of data through conversion of the resolution part, a resolution reduction part 4410 for changing the color color to black and white to reduce the amount of data For example, the background around the road, such as the number of avenues in a specific area, is likely to not correspond to the target image to be inspected. Therefore, And a frame reducing unit 4430 for reducing the number of image frames per second from, for example, 30 frames to one to several frames. Generally, the number of frames per second is about 30, and the number of frames per second can be reduced to between 1% and 99%, except for a portion that overlaps most of them (especially in the case of a still image).

 In order to generate the pre-processing neural network artificial intelligence provided for the preprocessing operation of the image preprocessing mechanism 4400, the simulation preprocessor preparation unit 4310 simulates the resolution, color, number of similar frames per second, necessary area, In particular, in the case of a model shape that can be collected and formatted in advance as a case through the open data collected from the external source data input unit 4110 in advance, the necessary area or the unnecessary area is collected in advance as a pre-processing element. 3000 via the GUI unit 3200. [ The other elements of the preprocessing element preparation unit 4310 may be previously set in the input / output unit 3100 of the control mechanism 3000 through the setting of the user, among various parameters such as the type, time, shade level, Can be selected. In addition, the input / output unit 3100 includes a function of a user interface (UI), performs additional visual inspection of essence metadata or essence image to be described later through a user interface of the input / output unit 3100, The completion data may be generated and output or stored in the database 4200. The simulation pre-processing learning unit 4320 performs machine learning using, for example, a shallow running or a deep learning technique, a myriad of open data images stored in the comprehensive DB 4200 based on the prepared pre-processing elements, Intelligence can be provided as an engine for pre-processing work.

 The essence data providing mechanism 4500 removes the environmental element directly from the real-time input image from the real-time data inputting unit 4120 using the essence-based neural network artificial intelligence generated from the neural network artificial intelligence generating unit 4360, And an essence image generation unit 4520 for generating an essence image corresponding to the essence meta data from the essence meta data generation unit 4510. The essence image generation unit 4520 generates an essence image corresponding to the essence meta data from the essence meta data generation unit 4510, ).

The essence data providing mechanism 4500 further includes an image difference processing unit 4530 for detecting a change in the front and rear CCTV images and generating a difference between the before and after images as image difference information to notify occurrence of an event or change of a situation in a specific area . The essence data providing mechanism 4500 reflects the image difference information on the essence metadata or the essence image or generates the direct image difference information based on the preprocessing information from the preprocessing information generating unit 4450 of the image preprocessing unit 4400 Output section 3100 of the control mechanism 3000 for the inspection of the naked eye examiner, for example. When the image difference processing unit 4530 provides the image difference information to the input / output unit 3100, the time of the alarm unit 3300 or the audible alarm information may be provided together with the change of the image.

The essence meta data or essence image may correspond to a target related image corresponding to a specific event and a certain portion may have a reduction ratio of an entire image excluding an environmental factor, that is, a target related image, for example, 50 to 99% It is possible to reduce the size of the inspection target image. In the case of the reduction inspection target image, for example, only the final target related image may be extracted through the eyes of the people.

In the case of the control mechanism 3000, the control unit 3000 may be responsible for controlling the entire partial inspection support system, controlling only the image recognition support apparatus 4000, (A real-time image, a preprocessed image, an essence metadata, an essence image, etc.) and a target image (a real-time image, a preprocessed image, an essence metadata, an essence image, etc.) An alarm unit 3300 for providing an alarm signal (sound, notification display) to the image recognition support apparatus 4000 and an input / output unit 3100), and an update unit 3500 for updating the neural network artificial intelligence.

In addition, the control mechanism 3000 may further include a cascading processing unit 3400 that performs cascading processing for allocating a decoding operation, a detection operation, a tracking operation, , And the categorization processing unit 3400 includes software subsystems in which a series of sequential processes such as decoating and detection are connected, so that video data is adjusted to be sequentially processed corresponding to these subsystems. As described above, in the related art, a bottleneck may occur in the process of processing a large number of data streams simultaneously in parallel on a given hardware device by simultaneously inputting CCTV video streams. For example, video decoding, object detection in video, video classification, and tracking use a large number of CPUs, memory buffers, and GPU resources for processing, so dozens of video streams can be decoded, This can become a bottleneck when a software subsystem simultaneously processes multiple video data streams, such as software subsystems. Accordingly, in the present invention, the cascading processing unit, which is a cascading software mechanism, uses a so-called cascading technique for separately performing decoding, detection, and tracking on different video data streams in order to solve the bottleneck . That is, in the process of generating the metadata by processing the video stream during the operation of the image preprocessing mechanism 4400 or the operation of the essence data providing mechanism 4500, the cascading processing unit 3400 of the present invention decodes According to the cascading or pipelining method of allocating the task and detection task to different video streams separately, when decoding the video stream 1, the other video stream is queued and the video stream 1 After decoding ends, the decoding of the second video stream is started. On the other hand, the video stream # 1 processes the video detection process after the decoding ends. That is, when one single video stream sequentially decodes, detects, and tracks the parallax, the other video stream proceeds to the detection decoding and tracking after one step. This can eliminate bottlenecks in which all video streams wait during a software process with high processing capacity, such as video detection. More specifically, assuming that there is a video decoder module (one of the sub-cascading software) and a video detection module (one sub-cascading software), the decoder module processes the video 1, The decoder module receives the video 2 and decodes it, and passes the result to the detection module. In this case, if the detection module is a process having a large processing capacity (heavy processing), the bottleneck phenomenon can be solved by placing a light processing module in the process before or after the detection module.

Next, referring to FIG. 2, an operation of a system and method for supporting a suspicious object part inspection in a CCTV image using a class-specific image recognition technology for reducing the load of a visual recognition surveillant according to an embodiment of the present invention will be described.

Step S1: The open data input unit 41110 of the data input storage mechanism 4100 of the image recognition support apparatus 4000 inputs image information and the like taken from the external source 2000 by photographing the alley road, 4200). Meanwhile, the real-time data input unit 4120 inputs real-time CCTV images input in real-time from the CCTV system 1000 installed in the residential area, the street area, and the road in the specific administrative area and inputs them into the comprehensive DB 4200, To the preprocessing mechanism 4400 or the essence data providing mechanism 4500.

Step S2: The simulated preprocessing element preparation unit 4310 of the AI unit 4300 prepares a simulation preprocessing element using the open data stored in the comprehensive DB 4200. The simulated preprocessing element may be, for example, a part of the image not related to the target image, such as the resolution of a specific image, the color of the image, the number of lines of the road, the background around the road, The reason for choosing the simulation preprocessing element is to reduce the amount of image data as a whole by excluding the portion of the image data from the image if the image is not related to the target image or the relevance is relatively low. Accordingly, if the image portion has little relevance to the target image, the simulated preprocessing element preparation unit 4310 can be prepared as a simulated preprocessing element. The tree line, which can be expressed as a shape among the simulation preprocessing elements, may be prepared in all predictable shapes through the GUI unit 3200 of the control mechanism 3000.

The simulated preprocessing learning unit 4320 performs simulated preprocessing learning based on the simulated image and the simulated preprocessing element using the open data, for example, as a learning method of shallow running or deep learning. do. That is, the simulation preprocessing learning unit 4320 recognizes the resolution of the open source image, and when the resolution of the open source image is higher than that of the simulated resolution element or through the input / output unit 3100 of the control mechanism 3000, The resolution of the simulated open data image can be lowered. For example, if the resolution of the simulated open source image is set to HD and the simulated resolution element for compression or reduction is set to CIF (352 × 240) or QCIF (176 × 120) And performs marine reduction learning for converting an image into a CIF or QCIF image. If the simulated image of the open material is color, it can learn the color conversion to convert the color to a lower grade color such as black and white. Further, when the simulated image of the open data is composed of, for example, 30 frames per second, it is possible to perform frame reduction learning in which the simulation image is reduced to one to several frames. In addition, if the simulated image of the open data is a road on which the vehicle is traveling, it is possible to perform necessary area setting learning to be limited to roads, vehicles on the road, people in India, etc., In this way, the simulation pre-processing learning unit 4320 performs learning to reduce the amount of image data in various situations by performing resolution reduction learning, color conversion learning, frame reduction learning, and required area setting learning for a simulation image of a number of open data And accumulate learning results. Learning or deep learning applied to learning is a well-known technique that has already been confirmed in the field of artificial intelligence. With regard to this technique, explanation of applying deep learning to a target element marking learning unit in the future will be briefly described with reference to FIG. Explain.

Step S3: Based on the simulated preprocessing learning results obtained from the simulated preprocessing learning unit 4320, the artificial intelligence generating unit 4360 generates a preprocessing neural network Generate artificial intelligence. The most significant difference between the prior art and the present invention is that the subject performing the preprocessing process is the pre-processing neural network artificial intelligence. According to the present invention, since not only the human being perceives the CCTV image input in real time, but performs the preprocessing in a manner that the neural network artificial intelligence as the function engine of search and processing is already learned, The effect according to the embodiment of the present invention is different from the effect of the prior art.

Step S4: The simulated environment element preparation unit 4330 of the artificial intelligence apparatus 4300 collects the simulated environment elements from the open data stored in the comprehensive DB 4200. In the simulated environmental element, environmental factors may include, for example, a shade change on a day, a seasonal change, a weather change, a change of a roadside tree or a sign according to the wind, and a movement of an animal such as a dog or a cat. If a formal model needs to be set in advance, it may be manually or automatically rendered into a graphic form through the GUI unit 3200 of the control mechanism 3000 to provide environment elements of various models.

On the other hand, the mock target element preparation unit 4340 is configured to prepare a mock target element preparation unit 4340 for the object such as an object to be inspected, for example, a criminal, a criminal vehicle, and the influence of other objects (E.g., movement of a person due to a collision, breakage of another vehicle or object, etc.), or the result of the action may be included in the target element. In the case of the simulated target element, a portion that needs to be stylized as a shape may be prepared as a graphic form through the GUI unit 3200 as in the case of the simulated environment element.

Based on the prepared mock environment element and the mock target element, the mock target marking learning unit 4350 performs a lot of mock learning on various situations of the open data from tens of thousands of times to tens of thousands of times to discriminate between the mock environment elements and the mock target elements Ability artificial intelligence abilities are achieved, for example, using shallow running or deep learning techniques.

Step S5: The neural network artificial intelligence generating unit 4360 generates the essence-based neural network artificial intelligence serving as a target search and processing engine based on the learning data accumulated through the iterative learning of the simulated target marking learning unit 4350, Is as follows.

4, the simulated environment element preparation unit 4330 takes, for example, a dragged vehicle as an object, and transmits a target element including the object and all related operations or situations to the GUI unit 3200 through the GUI unit 3200 It is possible to manually or automatically create a simulation element D100 as a simulation target element or a simulation environment element by cutting it into a small-sized moving image or snapshot image D100a, D100b. It is possible to create an environmental element by using the snapshot image D100c as an environmental element not related to the target element. In the snapshot photographs (D100a, D100b) of the target element, the rectangular box represents an element related to the object or the object, that is, the target element, and what is represented by a circle or an ellipse represents an environmental element unrelated to the target element. In this manner, the target element data D100 is generated to distinguish between the object and the non-object and to learn by changing the position of the polygon displaying the specific object in the target element by shape, color, and space-time. For convenience of explanation, it is assumed that the object or the target element in the embodiment is a black sedan vehicle and includes a specific vehicle number. In the case of the snapshot D100a, the environmental element corresponds to the shade (in the evening) and the surrounding trees, the target element corresponds to the dragon vehicle, and the snapshot D100b corresponds to a specific place (for example, ), A target element containing an object, and a surrounding forest as an environmental element, represented in the red square of the image in the image. Snapshot D100C represents only the environmental element, not the target element, Indicating that the site is a parking area rather than a road.

4 (b), the simulated target marking learning unit 4220 uses the open data D111a, D111b, and D111c (for example, road traffic situation data set) , Performs simulated target marking learning that can distinguish a simulated target element from environmental factors in a rain or snowy weather situation.

The open data includes open data D111a to D111c reflecting a number of road situation images and one or more target elements and environment elements. In addition, one of the open data D111a can distinguish the road area information including the xml file D111b included in the image image by including one with the same name.

The simulated target marking learning unit 4350 uses simulated element learning data D100 and open data D111a to D111c as a result of environmental change using Deep Learning or Shallow Learning techniques, Learning is performed to have the ability to grasp the target state using one or more of the data sets (e.g., the data set in Figure 4 (b)). That is, referring to FIG. 4A, a target marking learning is performed on a road by inputting a simulation element D100 in which a simulation element is displayed in a red square or a green circular (or oval) (D100b) that reflects a specific place and object on an object, or target marking learning information capable of performing target marking learning by inputting a video image in which an object is displayed as a polygon and distinguishing the object from road image data, And stores it as a learned neural network artificial intelligence or as a backup in a comprehensive DB (4200). In this case, the target is one example in which the simulated target marking learning unit 4350 is used as information for discriminating whether or not the input image is a target, and the target element and the environment Element area information may be stored in a separate xml file D111b so that the simulated target marking learning unit 4350 may generate a complex element set.

The simulated target marking learning unit 4350 uses or classifies a supervised learning method such as ANN (Artificial Neural Network), Boosting, or random forest, ICA It is possible to conduct simulation target marking learning using an independent learning method such as Independent Component Analysis (PCA), Principle Component Analysis (PCA), and Singular Value Decomposition (SVD).

The operation of the essence neural network artificial intelligence, which is the target marking learning and search or processing engine of the simulated target marking learning unit 4400 and the neural network artificial intelligence generating unit 4360, will be described in more detail with reference to FIG. 4 (c) .

The simulated target marking learning unit 4400 may classify the filter F100 constituted by K × K pixels from the left upper end to the right end of the video images D100a to D100c and D111a to D111c included in the simulated element D100 or the open data D110 The dot product is obtained from the pixels of the snapshot and open data (D100a to D100c, D111a to D111c) of the simulation element and the filter composed of the K × K pixels, and the feature maps A convolution layer D210a is generated. At this time, in order to distinguish the various image features included in the target image provided by the simulation element D100, the convolution layer D210a applies a plurality of multiple filters F100, rather than one filter F100, .

A quantitative value indicated in a feature map in a ReLU (Rectified Linear Unit) layer is converted into a nonlinear value capable of determining the degree of characteristics of a target element and an environmental element, using the feature map generated through the filter F100 as an input Creates an activation map that applies the ReLu function, which is an activation function.

The activation map is inputted into the MaxPool layer, for example, a Max-Pulling filter F200 composed of 2 pixels is stride 2, and two spaces are sequentially arranged from the upper left to the lower right of the feature map Scanning and extracting the maximum value from 2 pixels to generate the applied sampling map.

2C) of the simulation element D100 and the convolution layers D210a and D210b, the ReLu layer and the MaxPool layers D220a and D220b are repeatedly combined (corresponding to D200 in FIG. 2C) Or environment elements) can be generated.

A softmax function D300c, which is an activation function, is applied to a so-called "pulley connected layer " D300a in which the pixel values of the last outputted sampling map are concatenated to obtain a target element D400a, (D300) that can obtain the probability of the neural network (D300) in the range of 0 to 1.0 and construct a randomly extracted neuron of the neural network by using a so-called "dropout layer (D300b) (D300c) is applied to change the result value between 0 and 1. In this case, it is necessary to change the result value between 0 and 1 by applying Softmax function (D300c) The convolution layer, the ReLu layer, the MaxPool layer, and the Fully connected layer are each provided with a computer resource Since a commonly used matrix operation is used, a GUI unit 3200 (see FIG. 1) including a graphics processing unit (GPU) can be used to improve the performance of such a matrix operation. Further, Repeat times, execution positions and option values of layers can be used according to Deep Learning or Shallow Learning methods such as LeNet, AlexNet, ZFnet, GoogleNet, VGGNet and Res Net. have.

Step S6: Pretreatment Neural Network Artificial Intelligence and Essence Neural Network Artificial intelligence can function as an electronic or semi-automatic preprocessing engine and a search or processing engine, which requires a minimum of manpower once the simulation is complete, When simulation elements need to be added or changed (for example, changes in environmental factors due to abnormal weather trends or changes in simulated environmental elements or simulation pretreatment elements due to technological development), input of new simulation elements Or updating of the machine learning and new neural network artifacts as a result of the change and the application of the new open data may then proceed through the update unit 3500 of the control mechanism 3000. Therefore, when updating is required, the process from step S1 to step S5 can be repeated, and if there is no update, the process proceeds to step S7.

Step S7: It is determined whether the CCTV system 1000 or 1100 is a general CCTV camera 1010 of the general CCTV system 1000 or an IP CCTV camera 1110 of the IP CCTV system 1100. [

Step S8: If the CCTV systems 1000 and 1100 are general CCTV cameras 1010 of the general CCTV system 1000 in step S7, at least a part of the basic video recognition information is acquired through the work of the inspection personnel. If a part of the user acquires the basic image recognition information through the inspection personnel, the rest is acquired through the essence data providing mechanism 4500.

Step S9: When the CCTV systems 1000 and 1100 are the IP CCTV cameras 1110 of the IP CCTV system 1100 in step S7, at least a part of the entry-level image recognition information is transmitted to the IP CCTV camera 1110 . If only a part of the user acquires the basic image recognition information through the IP CCTV camera 1110, the rest is acquired through the essence data providing mechanism 4500.

Step S10: The present invention can proceed in two modes. The first mode includes a pre-processing neural network artificial intelligence for real-time CCTV images and a cascading function by the caching processing unit 3400 of the control mechanism 3000 (assignment of decoding operation, detection operation, and tracking operation for an individual video stream) And generating essence metadata or essence images by using the essence neural network artificial intelligence and cascading processing function based on the preprocessing information after the preprocessing information is generated based on the preprocessing information 2 mode is a process of generating essence metadata and essence image using direct essence neural network artificial intelligence for real time CCTV image. When the first mode is set in step S7, the process goes to step S8. That is, in the first mode and the second mode of operation, it is possible to record specific scenes such as movement of a specific object among the whole images to be inspected by using a machine learning technique, Can be provided. In addition, as CCTV video processing divides decoding work, detection work, and tracking work into different video streams at the same time, it becomes a bottleneck compared to the conventional art in which other video streams are used during detection work, decoding work, To greatly reduce the work delay and the load on the host server or the computer.

Step S11: The image preprocessing mechanism 4400 generates at least one of a resolution reduction process, a color conversion process, a frame reduction process, and a required area setting process using the pre-processing neural network artificial intelligence to generate preprocessing information. In this case, a frame switching process may also be performed.

FIG. 3 is a flowchart illustrating a preprocessing process.

Substep PS1: The image preprocessing mechanism 4400 downloads the pre-processing neural network artificial intelligence from the neural network artificial intelligence generating unit 4360, for example, or simply activates the neural network artificial intelligence generating unit 4360 to activate do.

Substep PS2: The image preprocessing mechanism 4400 receives the real-time CCTV image from the real-time data input unit 4120 of the data input mechanism 4100. [

Sub-step PS3: The resolution reducing unit 4410 lowers the resolution of the real-time CCTV image using the pre-processing neural network artificial intelligence. For example, the image of HD is reduced to a level of CIF or QCIF to generate a resolution reduced image.

Substep PS4: The color switching unit 4420 converts the color image into the monochrome image using the pre-processing neural network artificial intelligence to generate the color conversion image.

Sub-step PS5: The necessary area setting unit 4430 sets the area required for inspection or image processing around the target and the target-related part excluding the background around the road or the non-target area such as a street tree in the image by using the pre-processing neural network artificial intelligence And generates a necessary area setting image.

Sub-step PS6: The frame reduction unit 4440 performs frame reduction using the pre-processing neural network artificial intelligence to generate a frame reduction image.

During the respective steps of the sub-steps PS2 to PS6, the cascading processing by the cascading processing unit 3400 of the control mechanism 3000 may be performed.

PS7: The preprocessing information generating unit 4450 appropriately combines the resolution reduction image, the color conversion image, the required area setting image, and the frame reduction image, which are the result of the preprocessing steps of PS3 to PS6, to generate preprocessing information. The preprocessing information may be a result obtained in the order of the sub-steps, or may be a result of selectively selecting sub-steps of PS3 to PS6, if necessary.

Step S12: After the preprocessing process, the essence data providing mechanism 4520 generates essence metadata and essence image for the preprocessing information using the essence neural network artificial intelligence. The generated essence metadata or essence image may be a final examination that does not require additional inspection or treatment by the human hand, or may complete more than 50% of the entire imaging procedure and additional testing by manpower in the remaining 50% to 1% range It may be a semi-processed or retrieved product that completes the final inspection. In step S12, the control mechanism 3000 may activate the cascading processing section 3400 to execute the cascading processing.

If the second mode is not the first mode in step S10, the flow advances to step S13 to transmit the essence data providing mechanism 4500 from the real-time data input unit 4120 of the data input mechanism 4100 directly to the real- Or input non-real-time data already stored from the comprehensive DB 4200 to generate essence metadata through the essence neural network artificial intelligence or generate essence images corresponding to the corresponding metadata.

Step S14: The essence data providing mechanism 4500 detects a change in the front and rear CCTV images and generates a difference between the before and after images as image difference information, and executes an image difference processing process to notify generation of an event or change in a specific area . The essence data providing mechanism 4500 reflects the image difference information on the essence metadata or the essence image in step S12 or directly on the basis of the preprocessing information from the preprocessing information generating unit 4450 of the image preprocessing mechanism 4400 The image difference information may be generated and provided to the input / output unit 3100 of the control mechanism 3000 for inspection of, for example, the naked eye examiner (step S13). When the image difference processing unit 4530 provides the image difference information to the input / output unit 3100, the time of the alarm unit 3300 or the audible alarm information may be provided together with the change of the image. Essence data including at least one of the essence metadata and the essence image provided by the essence data providing mechanism 4500 in the case of the visual inspection by the inspection staff can be obtained by using the artificial intelligence apparatus 4300 of the present invention, % To 99%, it is possible to minimize the manpower required for the visual inspection accompanied by the workforce of the inspection personnel for the final completion inspection. Especially, when the IP CCTV camera 1110 is used, it is possible to reduce the visual inspection of the basic image recognition information by as much as that. Therefore, after the generation of the intermediate image recognition information and the advanced image recognition information, there is more room for the labor input of the visual inspection, High efficiency can be obtained in terms of efficiency and quality.

If the essence meta data or the essence image is a semi-processed image, additional visual inspection is performed. Otherwise, the process proceeds to step S7 and the process up to step S14 is repeated.

Step S15: When visual inspection is performed in step S12, the control unit 3000 inputs the recognition condition to the input / output unit 3100, and when an alarm is requested, the alarm unit 3300 generates an alarm every time a target appears in the essence image And informs the inspectors. When the operation of the essence metadata generation unit 4510, the essence image generation unit 4520, and the image difference processing unit 4530 is proceeded, the alarm function of the alarm unit 3300 is operated according to the setting condition, It is possible to inform the inspection personnel in real time about the appearance of the target image.

According to the present invention, a test manpower is input to search for a meaningful scene through analysis of the whole CCTV image to be inspected by using a machine learning technology. This method is a method for obtaining the maximum effect with a minimum manpower, Is configured to be able to perform a partial inspection rather than a full inspection. In the beginner image recognition function, it is possible to provide a technique of allowing the inspectors to perform only partial visual inspection by recording meaningful specific scenes including at least movement of a specific object, thereby reducing the manpower required for the complete inspection.

In the intermediate image recognition function, the intelligent CCTV IP camera is equipped with a beginner image recognition function. When significant image scenes such as motion detection and simple object recognition and recognized metadata information are outputted from the camera, It is aimed to be able to carry out tasks of intermediate and advanced image recognition level as required. When the first level image recognition data is input from one or multiple CCTVs, the system collects specific scenes such as more complicated situation recognition and suspect tracking linked to the camera network, and provides the system to the inspection personnel. It is possible to provide a technique for reducing the manpower required for the whole inspection by allowing only partial visual inspection of the advanced part.

As a result, the present invention can enhance the intermediate-level image recognition function and the intermediate-level image recognition function so that the inspection manpower can be largely reduced but the quality of the idle inspection workforce can be concentratedly applied to the high- . Of course, the present invention can also provide an advanced image recognition function, which can ultimately provide an infrastructure technology capable of drastically alleviating inspection personnel.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is possible.

1000: General CCTV equipment 1010: General CCTV camera
1020: VMS 1030: Video DB
1100: IP CCTV system device 1110: IP CCTV camera
1120: VMS 1130: Video DB
2000: external source 3000: control device
3100: input / output unit 3200: GUI unit
3300: alarm part 3400: cascade processing part
3500: Update part 4000: Image recognition support device
4100: data input device 4110: open data input device
4120: real-time data input unit 4200: comprehensive DB
4300: Artificial intelligence instrument 4310: Simulation preprocessing element preparation part
4320: simulation preprocessing learning unit 4330: simulation environment element preparation unit
4340: simulated target element preparing unit 4350: simulated target marking learning unit
4360: Neural network artificial intelligence generating unit 4400: Image preprocessing mechanism
4410: resolution reduction unit 4420: color conversion unit
4430: Required area setting unit 4440: Frame narrowing unit
4450: preprocessing information generating unit 4500: essence data providing mechanism
4510: Essence metadata generation unit 4520: Essence image generation unit
4530: Image difference processing unit

Claims (16)

To provide a technique which can reduce the manpower required for the total inspection by allowing only partial visual inspection by recording meaningful specific scenes including at least the movement of a specific object among the whole images to be inspected by using the machine learning technology A beginner image recognition function that detects at least a motion in a strange state different from a normal state and a beginner image recognition function that is detected according to the beginner image recognition function and a semantic level meaning that a specific event is defined in a single CCTV camera is generated And the intermediate image recognition function according to the intermediate image recognition function and collects the intermediate image recognition information according to the intermediate image recognition function to generate an advanced image recognition function for generating the interlocking information from a plurality of CCTV cameras. Supporting inspection of suspicious objects in CCTV images using recognition technology A system;
An external source that provides a pre-arranged open source of known open data that includes at least CCTV image information of events covering events, accidents, events, and predictable or unpredictable events occurring in the jurisdiction of the general administrative district; ;
A CCTV system for inputting video information including the event in real time;
And an image recognition support device for supporting image recognition by inputting open data of the external source and real-time image data from the CCTV system,
Wherein the image recognition support apparatus comprises:
A data input mechanism including an open data input unit for inputting the open data from the external source and a real time data input unit for inputting real time image data from the CCTV system for photographing the event in real time;
Extracting and preparing an action or an action related to the event and a corresponding event and a result according to an action or an action using the open data input from the open data input unit of the input mechanism in the open data, And the result of the action or action associated with the object other than the element or the object and the temporal and natural phenomenon changes expected to occur in the road are extracted and prepared as simulation environment elements and the simulation environment element and the simulation target A simulation target learning unit for applying a learning result to a CCTV image including an event among the open data to perform a machine learning for distinguishing a target element of the event from an environmental element, The image recognition information, the intermediate image recognition information, An artificial intelligence device including an artificial intelligence generating unit for generating an essence neural network artificial intelligence having a function of recognizing a target element by itself as a reference information for discriminating a beam;
The real-time data input unit receives the real-time CCTV image information using the essence-based neural network artificial intelligence and automatically recognizes the event generation and the target element related to the corresponding event, And an essence data providing mechanism for providing essence data that is at least one of the advanced image recognition information;
And the system for supporting suspicious object part inspection in a CCTV image using a class image recognition technology for load reduction of a visual recognition surveillant. The method according to claim 1,
Wherein the image recognition support apparatus comprises:
Before the essence data providing apparatus recognizes the target from the real-time CCTV image information and provides the essence data, the portion of the real-time CCTV image information that is not directly related to the target element is removed or reduced to obtain the data amount of the real- Further comprising an image preprocessing mechanism for generating preprocessing information for reducing the image quality of the image,
The artificial intelligence device further includes a simulation preprocessing element preparation unit for preparing a preprocessing element which is an identification element related to the preprocessing and a simulation preprocessing learning unit for mechanically learning the simulation preprocessing based on the simulation preprocessing element by an image difference or a deep learning method ,
Wherein the neural network artificial intelligence generating unit further generates a pre-processing neural network artificial intelligence capable of automatically performing a pre-processing based on the simulation pre-processing learning results accumulated in the simulation pre-processing learning unit,
Wherein the image preprocessing mechanism automatically generates the preprocessing information using the preprocessing neural network artificial intelligence and the essence data is generated based on the preprocessing information. Support system for suspicious object part inspection in CCTV image using. 3. The method of claim 2,
The image preprocessing mechanism includes a resolution reduction unit that reduces the resolution of the CCTV image, a color conversion unit that simplifies the color type of the CCTV image, a CCTV object that is not related to the target, A frame reduction unit for reducing the number of overlapping portions among the frames per second in the CCTV image; and at least one of an operation result of the resolution reduction unit, the color switching unit, the required area setting unit, And a pre-processing information generating unit for generating a part of the pre-processing information for image recognition as a preprocessing information for image recognition. 4. The method according to any one of claims 1 to 3,
The basic image recognition information includes at least the result extracted by the inspection personnel or at least the output formed directly from the IP CCTV,
Wherein the essence data providing mechanism provides at least one of the intermediate image recognition information or the advanced image recognition information as essence data at a completion rate of 80% to 99% or more corresponding to the exclusion rate of the inspection personnel. A Support System for Inspection of Suspicious Objects in CCTV Images Using Classified Image Recognition Technology for Mitigation. 5. The method of claim 4,
The essence data automatically displays meaningful events in the real time CCTV image information input from the real time data input unit using the essence neural network artificial intelligence, and displays positional information and target elements of the original image related to the corresponding events, And then, if necessary, the metadata is the data configured to be inspectable only for the meaningful scenes, or the data is the inspection completed. In the CCTV image using the class-based image recognition technology for reducing the load of the visual recognition examiner, Support system. 6. The method of claim 5,
The essence data providing apparatus further includes an image difference processing unit for detecting a change of the front and rear CCTV images and generating a difference between the before and after images as image difference information to notify occurrence of an event or change of a situation in a specific area,
Wherein the essence data providing mechanism reflects the image difference information to the essence metadata or the essence image or generates direct image difference information based on the preprocessing information from the preprocessing information generating unit of the image processing apparatus A System for Supporting Suspect Body Partial Inspection in CCTV Images Using Classified Image Recognition Technology for Reducing Load of Visual Recognition Surveyor. 6. The method of claim 5,
Wherein the system further comprises a control mechanism for controlling the entire system, and the control mechanism is integrally or directly telecommunicated with the image recognition support apparatus,
The control mechanism includes:
A casing for performing a cascading process of separating a decoding operation, a detection operation, a tracking operation and the like in the CCTV image processing operation of the image preprocessing mechanism and the essence data providing mechanism alternately one by one to different recognition target video streams, Processing unit,
Wherein the artificial intelligence mechanism is periodically or intermittently updated according to addition or modification of open data from the external source, real-time data provided from the CCTV system device, and environment elements and target elements. Supporting System for Suspect Body Partial Inspection in CCTV Images Using Level - Aware Image Recognition Technology. 8. The method of claim 7,
The control mechanism includes:
An input / output unit which sets an operation condition of an image preprocessing mechanism or an essence data providing mechanism, monitors the operation of the image recognition support apparatus and has a UI function,
A GUI unit that provides a function of manually or automatically generating a portion of a simulation preprocessing element, a simulation environment element, and a simulated target element that can be prepared in advance in a shape,
An alarm unit for notifying the appearance of a target element in real time when essence data is provided by the essence data providing apparatus,
At least,
Wherein the machine learning includes at least one of Deep Learning and Shallow learning,
Wherein the image recognition support apparatus further comprises a comprehensive database (DB) for storing data generated during the entire operation of the apparatus. Partial inspection support system. To provide a technique which can reduce the manpower required for the total inspection by allowing only partial visual inspection by recording meaningful specific scenes including at least the movement of a specific object among the whole images to be inspected by using the machine learning technology A beginner image recognition function that detects at least a motion in a strange state different from a normal state and a beginner image recognition function that is detected according to the beginner image recognition function and a semantic level meaning that a specific event is defined in a single CCTV camera is generated And the intermediate image recognition function according to the intermediate image recognition function and collects the intermediate image recognition information according to the intermediate image recognition function to generate an advanced image recognition function for generating the interlocking information from a plurality of CCTV cameras. Supporting inspection of suspect objects in CCTV images using recognition technology A;
The method may include providing previously open publicly known information including at least CCTV image information of events, accidents, events, and events covering predictable or unpredictable events occurring in a general administrative district jurisdiction An external source; A CCTV system for inputting video information including the event in real time; The system is used in a support system for a suspicious object visual recognition part inspection in a CCTV image using a moving object detection technology including an open source of the external source and an image recognition support device for inputting real time image data from a CCTV system device,
A data input step in which the image recognition support apparatus inputs the open data from the external source and inputs real-time image data from a CCTV system which photographs the event in real time;
A target element preparation step in which the image recognition support apparatus extracts, from the open data, an action or an action related to the event and a corresponding event and a result according to an action or an action thereof as a target element;
An environment in which the image recognition support device extracts and prepares the temporal and natural phenomenon changes expected to occur on the road and the results of the actions or actions related to the objects or the objects other than the target element, Element preparation step;
A simulated target marking learning step of performing the machine learning in which the image recognition support device applies the simulated environment element and the simulated target element to the CCTV image including the event among the open data to discriminate the target element of the event from the environmental element Wow;
Wherein the neural network artificial intelligence generating unit of the image recognition support apparatus accumulates the learning result in the simulated target marking learning unit as reference information for discriminating the elementary image recognition information, the intermediate image recognition information, and the advanced image recognition information, A neural network artificial intelligence generating step of generating an essence neural network artificial intelligence having a function of recognizing a target element;
The image recognition support apparatus automatically recognizes the generation of an event and a target element related to the event in real-time CCTV image information input from the real-time data input unit using the essence neural network artificial intelligence, , Essence data providing at least one of the intermediate image recognition information and the advanced image recognition information;
A method for supporting a suspicious object part inspection in a CCTV image using a class image recognition technology for reducing the load of a visual recognition surveillant. 10. The method of claim 9,
Processing step of generating pre-processing information for reducing the amount of real-time CCTV image information to be recognized by removing or reducing a part not directly related to the target element from the real-time CCTV image information before the essence data providing step,
Further included,
Before the image preprocessing step,
A preprocessing element preparation step of preparing a preprocessing element which is an identification element related to the preprocessing;
A simulation preprocessing learning step of mechanically learning the simulation preprocessing based on the simulation preprocessing element by a deep learning method;
Wherein the neural network artificial intelligence generator generates a pre-processing neural network artificial intelligence capable of automatically performing pre-processing based on the simulation pre-processing learning results accumulated in the simulation pre-processing learning unit;
Further comprising:
Wherein the pre-processing information is automatically generated using the pre-processing neural network artificial intelligence in the image preprocessing step, and the essence data is generated based on the preprocessing information. A Method for Supporting Suspect Object Partial Inspection in CCTV Images Using. 11. The method of claim 10,
Wherein the image preprocessing step comprises:
A lower resolution sub-step for lowering the resolution of the CCTV image,
A color conversion unit step for simplifying a color type of a CCTV image,
A necessary region setting sub-step of excluding an object not related to a target in the CCTV and setting a portion, which is relevant to the target or the target,
A frame reduction sub-step of lowering the number of redundant portions of frames per second in the CCTV image,
And a preprocessing information generating sub-step of generating at least a part of the operation result of the resolution reducing sub-step, the color switching sub-step, the required area setting sub-step, and the frame reducing sub-step as preprocessing information for image recognition, A Supporting Method of Suspect Body Partial Inspection in CCTV Images Using Class - level Image Recognition Technology for Load Reduction of Surveyor. 12. The method according to any one of claims 9 to 11,
The basic image recognition information includes at least the result extracted by the inspection personnel or at least the output formed directly from the IP CCTV,
Wherein the essence data providing mechanism provides at least one of the intermediate image recognition information or the advanced image recognition information as essence data at a completion rate of 80% to 99% or more corresponding to the exclusion rate of the inspection personnel. A Method for Supporting Suspicious Object Detection in CCTV Images Using Classified Image Recognition Technology for Mitigation. 13. The method of claim 12,
The essence data automatically displays meaningful events in the real time CCTV image information input from the real time data input unit using the essence neural network artificial intelligence, and displays positional information and target elements of the original image related to the corresponding events, And if it is necessary, it is the data configured to be able to inspect only the significant scenes that are meta data if necessary, or the semi-processed data in which the inspection is completed or the final inspection is completed through the additional inspection of the inspection personnel. A Method for Supporting Suspicious Object Detection in CCTV Images Using Classified Image Recognition Technology for Mitigation. 14. The method of claim 13,
After the pre-processing step, the essence data providing apparatus further includes an image difference processing step of detecting a change in the front and rear CCTV images and generating a difference between the before and after images as image difference information to notify occurrence of an event or change in a specific area In addition,
In the image difference processing step, the essence data providing mechanism may reflect the image difference information to the essence metadata or the essence image, or may directly reflect the image difference information based on the preprocessing information from the preprocessing information generating unit of the image processing apparatus A method for supporting a suspicious object part inspection in a CCTV image using a class image recognition technology for load reduction of a visual recognition surveillant. 14. The method of claim 13,
Wherein the system further comprises a control mechanism for controlling the entire system, and the control mechanism is integrally or directly telecommunicated with the image recognition support apparatus, and during the image preprocessing step or the essence data providing step,
In the CCTV image processing operation of the image preprocessing mechanism and the essence data providing mechanism, the caching processing unit of the control mechanism separates the decoding operation of the recognition operation, the detection operation, the tracking operation, etc. and assigns them alternately to the different recognition target video streams A cascading process step of:
Further comprising the step of periodically or intermittently updating the neural network artificial intelligence according to addition or change of open data from the external source and real-time data provided from the CCTV system device, and environment elements and target elements, A Supporting Method for Suspect Body Partial Inspection in CCTV Images Using Classified Image Recognition Technology for Reducing the Load of Awareness Surveyor. 14. The method of claim 13,
Wherein the system further comprises a control mechanism for controlling the entire system, and the control mechanism is integrally or directly telecommunicated with the image recognition support apparatus,
At least one of the steps of preparing the simulated target element, preparing the simulated environment element, and preparing the simulated preprocessing element,
An input / output step of setting the operating condition of the image preprocessing step or the essence data providing step and monitoring the operation of the image recognition support apparatus,
A GUI step for providing a function of manually or automatically generating a part which can be preliminarily prepared in a shape among a simulation preprocessing element, a simulation environment element, and a simulated target element,
An alarm step for notifying the appearance of the target element in real time when the essence data providing apparatus is provided with the recognition work of the CCTV image by the essence data providing apparatus,
At least,
Wherein the machine learning includes at least one of Deep Learning and Shallow learning,
The method as claimed in claim 1, further comprising the step of storing data generated by an overall database (DB) of the image recognition support apparatus in the entire operation of the apparatus. How to support the inspection of my suspicious objects.

Images