KR101092472B1 - Video indexing system using surveillance camera and the method thereof - Google Patents

Abstract

PURPOSE: A video indexing system using a monitoring camera is provided to improve searching performance by indexing index information in a people image and an object image and storing index information at a database. CONSTITUTION: An image storage module(21) stores a photographed image. An image collection module(22) collects the photographed image and the event image. An object extracting module(23) extracts the people image and the object image from the collected image. An object index module(24) inserts index information into the detected people image and the object image. A database(25) records index data. An image searching module(26) extracts the index data. A face recognition module(27) recognizes a face.

Description

Video indexing system using surveillance camera and the method

The present invention relates to a video indexing system using a surveillance camera and an indexing method thereof. The present invention relates to a video indexing system and an indexing method. The present invention relates to a video indexing system using a surveillance camera that can easily perform indexed vehicle or face searching and face recognition in a short time by recording and managing the same, and an indexing method thereof.

Recently, CCTV (Closed Circuit Television) camera with built-in DVR (Digital Video Recorder) is installed in the entrance and perimeter of facility to prevent various accidents. Many CCTV systems are being installed and utilized to monitor through monitors and automatically save the images to video recording and storage devices.

Such systems are being installed around residential areas, playgrounds, etc., and the number of personal crimes, object crimes, and vehicle use crimes, which are increasing every year, is being reduced to some extent through CCTV systems. This can be checked directly or indirectly.

Such CCTV systems require a lot of searching time and are often not helpful in accurately arresting criminals or vehicles due to the mistakes of monitoring personnel, and people related to crimes in images taken from CCTV cameras stored in video storage devices. Searching for cars or vehicles is not easy. It is virtually impossible to find more accurate and quicker CCTV footage in real time, including limited CCTV systems, limited monitoring personnel, and increasing number of crimes, with many constraints, and reliable search results for criminals and vehicles. There is still no monitoring expertise required.

In addition, the management (working person) monitors the vast places where outsiders cannot invade, or the management (working person) continuously monitors the traffic accidents occurring on the road intermittently by manager breaks, shifts, and acquisitions. It is practically impossible due to handovers, and it is difficult for the administrator to continuously watch the video taken by CCTV on the monitor and watch outsiders invade the intrusion-prone border area.

Actual CCTV systems are increasing every year, but they do not show their full performance, so analyzing and indexing the images stored in the image storage device, searching for these indexed data, comparing the data to be compared with the data to be input in real time There is an urgent need for an algorithm that reports the results.

In view of this point, an object of the present invention is to analyze images collected from a plurality of surveillance cameras, to distinguish between human images and object images, and to add index information including color information and size information to each human image and object image. Records and manages videos that are indexed or face-to-face and facial recognition can be easily performed in a short time, thereby preventing crime and using video surveillance cameras to quickly and easily arrest criminals or vehicles using crime. It provides an indexing system and its indexing method.

In order to achieve the above object of the present invention, a video indexing system using a surveillance camera according to the present invention includes an image storage module for storing images captured by a plurality of surveillance cameras in chronological order, and the plurality of surveillance cameras. An image collection module for collecting an image and an event image transmitted from the image storage module, and using the GMM (Gaussian mixture modle) in the image collected by the image collection module to distinguish the foreground and detect edge information, An object extraction module for extracting a human image and an object image including a person or a vehicle by analyzing the edge information, an object index module for inserting index information into the human image and the object image detected by the object extraction module, and the object index Data that records and manages index data for object images or human images with index information added to the module. An image retrieval module for receiving an image corresponding to a recorded image retrieval condition range from the image storage module and extracting index data from a database according to a retrieval condition for a face or a vehicle, and an image for face recognition in the image retrieval module And a face recognition module which receives a human image from the object extraction module and compares the face image with the face recognition image information and transmits a face recognition result to the image retrieval module.

At this time, the image storage module is characterized in that for storing the image taken by the surveillance camera as a picture file, and designates the picture file name as the image acquisition time.

The object extracting module determines that the object is one when the size of the object extracted from the object image is less than a predetermined value (if the set value is Min), and when the size of the object is greater than or equal to the predetermined value (more than the set value). ) Is determined as two or more people or a vehicle image.

In addition, the object extraction module obtains an edge map by using a Sobel edge after performing the contour analysis of the object edge when the size of the object is a certain value or more, and horizontal projection method (horizontal) It is characterized by distinguishing vehicles / persons by analyzing peak and valley components through a projection result graph calculated by applying projection and vertical projection.

The object extraction module may identify the number of persons by referring to the number of vertices and lowest points of the horizontal projection method.

Meanwhile, the object index module obtains color information of the shortest distance by obtaining the RGB value of the desired pixel and the vector distance in each RGB direction by using the data of the RGB box in the human image, and color information on the human image in the database. It is characterized by adding to the index information and storing.

In addition, the object indexing module presets the skin color region learned in the RGB box in advance, and recognizes a region belonging to the skin color region in the human image as a face region, and the skin color region at the top when the skin color region is plural. It is characterized in that the recognition as a face area.

When the face region is recognized, the object indexing module detects eyes in the face region, and adds the index information after recognizing the face through a face recognition algorithm.

Here, the face recognition algorithm applies a principal component analysis (PCA) to feature point extraction and uses the eigen face and wavelet transform coefficients as features, and measures similarity with the feature vector in the database. It is characterized by applying the distance (Euclidean).

On the other hand, the video indexing method using the surveillance camera according to the present invention for achieving the above object of the present invention, the first step of storing the images captured by the plurality of surveillance cameras in the image storage module in chronological order, the image Analyzing the image stored in the storage module through a GMM to detect edge information and color information, and analyzing the edge information to extract a human image and an object image including a person or a vehicle; A third step of adding index information including color information to the extracted human image and the object image, indexing and storing the index information in a database, and receiving an image corresponding to a recorded image search condition range from the image storage module; Alternatively, the index information is extracted from the database based on the search condition for the vehicle and corresponds to the search condition. That consisting of a fourth step of detecting the image is characterized.

The extracting of the object image in the second step may include determining that the object is one when the size of the object extracted from the object image is less than a predetermined value, and the two or more when the size of the object is greater than or equal to the predetermined value. Or it is determined that the vehicle image.

When the size of the object is greater than or equal to a certain value, performing an edge analysis of an object edge and obtaining an edge map using a Sobel edge based on the result of the contour analysis. And applying a horizontal projection and a vertical projection to the edge map, and then analyzing peak and valley components through a projection result graph to distinguish between vehicles and people. Characterized in that made.

In addition, after applying a horizontal projection and a vertical projection to the edge map, the step of analyzing the peak and valley components through the projection result graph to distinguish the vehicle / person, the The number of people is determined by referring to the number of vertices and the lowest points of the horizontal projection method.

In the third step, the RGB image of the desired pixel and the vector distance in each RGB direction are obtained by using the data of the RGB box in the human image, and color information of the shortest distance is obtained. The information may be added as index information and stored.

On the other hand, after setting the skin color region learned in the RGB box in advance, the region belonging to the skin color region in the human image is recognized as a face region, and when the skin color region is plural, the skin color region at the top is recognized as the facial region. Characterized in that.

In the third step, when the face region is recognized, eyes are detected in the face region, and after recognizing the face through a face recognition algorithm, the third region is added as index information.

In addition, the face recognition algorithm applies a principal component analysis (PCA) to feature point extraction and uses the eigen face and wavelet transform coefficients as features, and measures the similarity with the feature vector in the database. It is characterized by applying the distance (Euclidean).

According to the video indexing system using the surveillance camera and the indexing method, the image collected from the surveillance camera is analyzed to extract the human image and the object image, and the index information including the color information and the size information is obtained from the human image and the object image. By indexing and storing them in a database, you can secure excellent face or vehicle index and search performance, and increase the reliability of face recognition.

In addition, the present invention can easily search for image data indexed at a desired date or time of day based on image data such as missing missing persons, criminals or criminal vehicles, and can prevent crimes in advance and prevent missing missing children. It can also save you money or time in finding or arresting criminals and criminals.

1 is a block diagram showing the overall configuration of a video indexing system using a surveillance camera according to an embodiment of the present invention;
2 is a view showing an RGB box applied to an embodiment of the present invention;
3 is a view of checking the position of both eyes after detecting the eyes in the face area
4 is a flowchart illustrating a video indexing method using a surveillance camera according to an embodiment of the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. All terms used herein, including technical or scientific terms, unless otherwise defined, are to be understood as not precluding the existence or addition of numbers or numbers, steps, operations, components, components or combinations thereof. These have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention.

1 is a block diagram showing the overall configuration of a video indexing system using a surveillance camera according to an embodiment of the present invention, Figure 2 is a view showing an RGB box applied to an embodiment of the present invention, Figure 3 FIG. 4 is a diagram illustrating the position of two eyes after detecting eyes in a face region, and FIG. 4 is a diagram illustrating a flow of a video index algorithm according to an exemplary embodiment of the present invention.

1 to 3, a video indexing system using a surveillance camera according to an embodiment of the present invention includes a local device 10 composed of a plurality of surveillance cameras 11 installed in various places around the surveillance camera, and the surveillance camera. The center apparatus 20 may be classified into a center apparatus 20 that stores and analyzes the captured image, extracts a human image and an object image, and finds an image that matches an external face and vehicle search condition.

In particular, the center apparatus 20 includes an image storage module 21 for storing images captured by the plurality of surveillance cameras 11 together with time information, and images stored by the plurality of surveillance cameras 11 and the image storage. An image collection module 22 collecting event images transmitted from the module 21 and an image collected by the image collection module 22 detect edge information, and a Gaussian mixture model (GMM) algorithm. The object extraction module 23 extracts a human image and an object image including a face or a vehicle by using the same, and index information such as color information and size information is applied to the human image and the object image detected by the object extraction module 23. An object index module 24 for inserting, a database 25 for recording and managing index data for an object image or a human image to which index information is added in the object index module 24, and the image storage module 21 In The image retrieval module 26 which receives an image corresponding to a recorded image retrieval condition range and extracts index data from a database according to a retrieval condition for a face or a vehicle, and the image retrieval module 26 may include image information for face recognition. When received, the object extraction module 23 includes a face recognition module 25 that receives a human image and compares the face recognition image information with the face recognition image information to the image retrieval module 26, but is not limited thereto. Does not.

In addition, the image storage module 21, the image collection module 22, the object extraction module 23, the object index module, the database 25, the image search module 26, and the face recognition of the center device 20. Module 25 may be modular, but may each be configured as a server.

Meanwhile, the image storage module 21 enlarges the image captured by the surveillance camera 11 and stores the image as a photo file, and designates the photo file name as the image acquisition time, thereby searching for the object in the image retrieval module 26. You can easily search for the time that appeared by using the file name.

The object extraction module 23 extracts an object image by using a Gaussian mixture model (GMM) algorithm, and determines that the object is one when the size of the object extracted from the object image is smaller than a predetermined value (less than a set value). When the size of the object is greater than or equal to a predetermined value (more than a set value), it is determined that the object is two or more or a vehicle image.

In this case, when the size of the object is more than a predetermined value, the object extraction module 23 obtains an edge map using a Sobel edge after performing contour analysis of the object edge, and horizontally The peak and valley components may be analyzed through a projection result graph obtained by applying a horizontal projection and a vertical projection to distinguish between a vehicle image and a human image.

In addition, when it is determined that the object extraction module 23 is a human image, the object extraction module 23 checks the number of persons by referring to the number of vertices and lowest points of the horizontal projection method.

Here, the object extraction module 23 checks the color information of the lower face pixel coordinates of the human face by using values from RGB 0 to 255 in the human image.

That is, as shown in FIG. 2, using the data of the RGB box, the RGB value of the desired pixel and the vector distance of each RGB direction are obtained to obtain color information of the shortest distance, and eight vertices (Red, Green, Blue) of the rectangle are obtained. , White, black, cyan, magenta, and yellow) and the vertex value closest to the distance of the current pixel becomes the color information of the pixel.

For example, black is the same as Equation 1, red is the same as Equation 2, and blue is the same as Equation 3, so that the remaining colors of cyan, red, and white are combined with white, black, blue, and green. After determining all eight vertices and distances in order of magenta (blue) and yellow (red and green), the shortest distance is determined by the color information of the pixel.

Then, the object index module 24 adds color information to the human image as index information in the database 25 and stores it.

That is, the object index module 24 recognizes a region belonging to the skin color region in the human image as a face region after presetting the skin color region learned in the RGB box, and is located at the top when there are a plurality of skin color regions. Recognizes the skin color area as a face area.

The object index module 24 detects eyes in the face area after the face area is recognized, and additionally stores the index information after recognizing the face through a face recognition algorithm.

Here, the face recognition algorithm applies a principal component analysis (PCA) to feature point extraction to use an eigen face and a wavelet transform coefficient as a feature, and measures the similarity in the database 25. Apply the distance from the vector (Euclidean).

In addition, as shown in FIG. 3, after detecting the eye in the face area, it is checked whether the positions of the two eyes are centered left and right.

That is, when the value of w- (a + b) / w is between the first and second thresholds, and the value of c / h is between the third and fourth thresholds, both eyes are centered left and right. I think it is.

Then, the object index module 24 adds color information as index information to the vehicle image and stores it in the database 25.

Hereinafter, an operation of a video indexing system using a surveillance camera according to an embodiment of the present invention will be described in more detail with reference to the accompanying drawings.

4 is a flowchart illustrating a video indexing method using a surveillance camera according to an embodiment of the present invention.

Referring to FIG. 4, the video indexing method using a surveillance camera according to an embodiment of the present invention stores images captured by the surveillance camera 11 of the local apparatus 10 in the image storage module 21. The image is enlarged and stored as a photo file, and the image acquisition time is set as the file name.

The object extraction module 23 performs a background image of the event image transmitted from the image storage module 21 and the image collected in real time from the plurality of surveillance cameras 11 in the image collection module 22 using a Gaussian Mixture Model (GMM). Distinguish from the foreground.

In this case, the object extraction module 23 determines that the object is one person when the size of the object extracted from the object image is smaller than a predetermined value, and determines that the object extraction module 23 is two or more or a vehicle image when the size of the object is greater than or equal to the predetermined value. do.

In this case, when the size of the object is greater than or equal to a certain value, after performing contour analysis of the object edge, the edge map is obtained using the Sobel edge, and the peak and peak values are obtained through a projection result graph applying the horizontal projection method and the vertical projection method. Valley components are analyzed to distinguish vehicle / person images.

In this case, the object extraction module 23 may check the number of persons by referring to the number of vertices and the lowest points of the vertical projection method in the case of the human image.

On the other hand, the object indexing module 24 obtains the shortest distance color information by obtaining the RGB value of the desired pixel and the vector distance in each RGB direction using the data of the RGB box in the human image, and stores the color information in the database. Add color information as index information and store it.

In this case, the object indexing module 24 presets the skin color region learned in the RGB box in advance, and recognizes a region belonging to the skin color region as the face region in the human image, detects the eye in the face region, and recognizes the face. The algorithm recognizes the face and adds it as index information.

Thereafter, the object indexing module 24 indexes and stores the size information such as skin color, impression wear (clothing color) information, height, and the like in the human image extracted by the object extraction module 23, and stores the size information in the database 25. Color and size information of the vehicle is indexed in the image and stored in the database 25.

When the image retrieval module 26 transmits face recognition image information, such as a picture of a person related to a missing child or other crime, to the face recognition module 25, the face recognition module 25 may perform real-time in the object extraction module 23. The human image is received and compared with the face recognition image information, and the person image corresponding to the face recognition image information is transmitted to the image retrieval module 26.

In addition, when performing a vehicle search on a specific date or time zone, the image search module 26 receives a search condition such as color or size information of a vehicle after receiving a recorded image of a search condition range from the image storage module 21. The database 25 is searched to extract index data. The time information is linked with the stored video by using the current time information.

In this case, since the image search module 26 includes image acquisition time information for each image, the image search module 26 may search for an image (face or vehicle) indexed at a date and time zone corresponding to a search condition or perform face recognition.

As described above, when the index information added by the object index module 24 transmits a person image and a vehicle image by distinguishing a person from a vehicle in the object extraction module 23, the color of the skin color or clothes of a person wearing the impression is displayed on the person image. Size information such as color information and keys is included, and color information of the vehicle is included in the vehicle image.

Therefore, the image retrieval module 26 outputs an image corresponding to any one of the index information from the database 25 on the screen based on the face of the person or the vehicle image information input from the outside.

In this case, the image search module 26 may simultaneously output the video stored in the DVR in connection with the DVR.

On the other hand, the image retrieval module 26 automatically performs real-time indexed image retrieval and face recognition when the administrator presets image information of missing missing persons, criminals, criminal vehicles, etc. as a search condition. A communication unit (not shown) may be installed to notify an administrator or a police station by SMS or MMS when an image corresponding to specific image information is captured among the search conditions.

 In addition, when the administrator sets the search condition, the risk level is set in connection with the crime information and the missing information in the police, and when an image meeting the search condition is captured, the risk level is determined in the SMS or MMS according to the risk level of the video. If the risk level is low, the supervisor can only be notified to prevent unnecessary police or fire fighting, but the higher the risk level, the higher the risk level can be directly linked to 112 or 119, so that the police or fire fighting can be made.

Although described with reference to the embodiments above, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

10: local device 11: surveillance camera
20: center device 21: video storage module
22: image acquisition module 23: object extraction module
24: object index module 25: database
26: Image Search Module 27: Face Recognition Module

Claims (9)

An image storage module for storing images captured by multiple surveillance cameras in chronological order;
An image collection module for collecting images captured by the plurality of surveillance cameras and event images transmitted from the image storage module;
An object extraction module for detecting edge information from the image collected by the image collection module and extracting a human image and an object image including a person or a vehicle by analyzing the edge information;
An object index module for inserting index information into the human image and the object image detected by the object extraction module;
A database for recording and managing index data on an object image or a human image to which index information is added in the object index module;
An image retrieval module for receiving an image corresponding to a recorded image retrieval condition range from the image storage module and extracting index data from a database according to a retrieval condition for a face or a vehicle; and
And a face recognition module configured to receive a human image from the object extraction module and to compare the face recognition image information with the face recognition image to the image retrieval module if the image recognizing image information is transmitted from the image retrieval module.
The object extraction module detects edge information by using Sobel Edge Detection, analyzes the edge information, analyzes RGB data and edge size of a color image inside the edge,
The image storage module stores the image taken by the surveillance camera as a photo file, and designates the photo file name as an image acquisition time.
The method of claim 1,
The object extracting module may determine that the object is one person when the size of the object extracted from the object image is less than a set value, and determine that the object is two or more or a vehicle image when the size of the object is greater than or equal to the set value. Video indexing system using camera
The method of claim 2,
The object extraction module obtains an edge map using a Sobel edge after performing contour analysis of the object edge when the size of the object is greater than or equal to a set value, and uses horizontal projection. And a video indexing system using a surveillance camera that distinguishes vehicles / persons by analyzing peak and valley components through a projection result graph obtained by applying a vertical projection method and a vertical projection method.
The method of claim 1,
The object index module obtains color information of the shortest distance by obtaining a RGB value of a desired pixel and a vector distance in each RGB direction using data of an RGB box in the human image, and indexes the color information to the human image in the database. Add it as information, save it,
The object index module presets the learned skin color region in the RGB box, and then recognizes a region belonging to the skin color region as a face region in the human image, and faces the skin color region at the top when the skin color region is multiple. Recognize it as an area
The face recognition algorithm applies a principal component analysis (PCA) to feature point extraction and uses the eigen face and wavelet transform coefficients as a feature, and measures the similarity to the distance from the feature vector in the database ( Video indexing system using surveillance camera, characterized by applying Euclidean).
A first step of storing the images captured by the plurality of surveillance cameras in the image storage module in chronological order;
A second step of analyzing edge images and color information by analyzing the images stored in the image storage module, extracting a human image and an object image including a person or a vehicle by analyzing the edge information;
A third step of adding and indexing index information including color information to the human image and the object image extracted in the second step and storing the index information in a database;
Receiving an image corresponding to a recorded image search condition range from the image storage module, extracting index information from the database based on a search condition for a face or a vehicle, and detecting an image corresponding to the search condition; ,
Extracting the object image in the second step,
When the size of the object extracted from the object image is less than the set value is determined to be one person, and when the size of the object is greater than the set value is determined to be two or more or a vehicle image video using a surveillance camera, characterized in that Index method.
The method of claim 5,
If the size of the object is more than the set value,
Performing contour analysis of the object edge,
Obtaining an edge map using a Sobel edge based on the result of the contour analysis;
After applying a horizontal projection (vertical) and a vertical projection (vertical projection) to the edge map characterized in that it comprises the step of analyzing the peak (valley) and the lowest (valley) component through the projection result graph to distinguish the vehicle / person Video indexing method using surveillance cameras.
The method of claim 6,
After applying a horizontal projection and a vertical projection to the edge map, analyzing a peak and a valley component through a projection result graph to classify vehicles / persons may include: The number of people is determined by referring to the number of vertices and the lowest point of,
In the third step, the RGB value of the desired pixel and the vector distance in each RGB direction are obtained by using the data of the RGB box in the human image to obtain color information of the shortest distance, and the color information is added to the human image in the database. Video indexing method using a surveillance camera, characterized in that the addition to the index information and storing.
The method of claim 7, wherein
After setting the skin color region learned in the RGB box in advance, a region belonging to the skin color region in the human image is recognized as a face region, and when the skin color region is plural, the skin color region at the top is recognized as a facial region.
In the third step,
When the face region is recognized, eyes are detected in the face region, the video indexing method using a surveillance camera, characterized in that after the face recognition through the face recognition algorithm added to the index information.
The method of claim 8,
The face recognition algorithm applies a principal component analysis (PCA) to feature point extraction and uses the eigen face and wavelet transform coefficients as a feature, and measures the similarity to the distance from the feature vector in the database ( A video indexing method using a surveillance camera characterized by applying Euclidean).

Images