KR20130094489A - System and method for monitoring emergency motion based image - Google Patents

Abstract

PURPOSE: An abnormal motion monitoring system and method based an image are provided to accurately detect the abnormal motion of a dynamic object by displaying the dynamic object existing in an image as an operation area bounding box and monitoring the abnormal motion of the dynamic object according to the size change of the operation area bounding box. CONSTITUTION: An abnormal motion monitoring system based an image comprises a photographing unit (10) and a monitoring device (20). The photographing unit photographs a photographing area. If a dynamic object exists in the photographed image, the control unit displays the dynamic object as an operation area bounding box and monitors the abnormal motion of the dynamic object according to the size change of the operation area bounding box. [Reference numerals] (10) Photographing unit; (20) Monitoring device; (21) Abnormal operation image storage unit; (22) Background image storage unit; (23) Background image managing unit; (24) Detection unit; (25) Extraction unit; (26) Setting unit; (27) Output unit; (28) Determining unit

Description

System and Method for Monitoring Emergency Motion based Image}

The present invention relates to an image-based abnormal motion monitoring system and method, and to an image-based abnormal motion monitoring system and method for detecting abnormal motion of a dynamic object existing in a captured image.

Today, the rapid aging of the population and the advancement of the form of living are combined to increase the number of the elderly living alone. In the case of elderly living alone or patients living alone, accidents that fall during everyday life frequently occur because of uncomfortable behavior.

Fall accidents occur more often indoors than outdoors, and are common enough for half of the elderly. Moreover, it can occur not only in the elderly but also in chronically ill or young people, such as stroke patients.

The elderly living alone or living alone are often living alone indoors, so even if a fall causes serious injuries such as a fracture or loss of mind, people who are unable to move can recognize the accident quickly. I can't. Moreover, it is not possible to take timely action in the event of an accident, often leading to serious situations.

In recent years, by monitoring the operation of the human body to quickly identify when the abnormal operation occurs, to solve the social neglect of the elderly living alone or patients living alone, a technology for protecting them has been developed. For example, Patent Document 1 discloses a technique for monitoring abnormal operation using a human body motion detection sensor, a call button, and a camera.

However, since the conventional abnormal operation monitoring method detects abnormal operation by detecting whether the human body is inoperative through a sensor, even if a fall accident occurs, if the human body has a slight movement, it is not judged as an abnormal operation or the human body. As there is no movement for a predetermined time as the user sleeps, there is a problem in that accuracy and reliability are inferior in detecting abnormal operation.

In addition, the conventional abnormal operation monitoring method has an inefficient problem because it is not possible to notify the emergency situation by using the call button when the fallen person cannot be moved due to loss of spirit or serious injury.

In addition, the conventional abnormal operation monitoring method has an inefficient problem in terms of manpower and labor costs because the administrator must always monitor the abnormal operation of the human body while displaying the camera photographed image output to the monitor.

KR1020050090947 A

The present invention has been made to solve the above problems, and provides an image-based abnormal motion monitoring system and method that can accurately detect the abnormal motion of the dynamic object by tracking the movement of the dynamic object present in the captured image. There is a purpose.

An image-based abnormal motion monitoring system according to an embodiment of the present invention for achieving the above object, the photographing unit for obtaining an image for the shooting area; And when a dynamic object appears in the image acquired through the photographing unit, expresses a dynamic object existing in the image as an operation area bounding box, and generates an abnormal operation of the dynamic object according to a change in the size of the operation area bounding box. It is preferable to include the monitoring apparatus to monitor.

On the other hand, the video-based abnormal motion monitoring method according to an embodiment of the present invention, the real-time transfer of the image of the photographing area to the monitoring device in the photographing unit; Obtaining a background image frame from an image transmitted from the photographing unit in the monitoring apparatus; Detecting, by the monitoring apparatus, dynamic object appearance image frames among image frames constituting an image transmitted from the photographing unit based on the background image frame; Recognizing, by the monitoring apparatus, a dynamic object present in each dynamic object appearance image frame; The monitoring apparatus sets a bounding box and a bounding box center point of the dynamic object for each dynamic object appearance image frame by using the motion region size and the motion position of the dynamic object present in each dynamic object appearance image frame. process; Tracking, by the monitoring apparatus, the size change of the operation area bounding box based on the bounding box center point set in each dynamic object appearance image frame, and determining whether an abnormal operation of the dynamic object occurs according to the size change of the operation area bounding box; And outputting an alarm for notifying occurrence of an abnormal operation when the abnormal result of the dynamic object is generated by the monitoring apparatus.

According to the video-based abnormal motion monitoring system and method according to the present invention, by accurately detecting the abnormal motion of the dynamic object by tracking the movement of the dynamic object present in the captured image, it is possible to build a reliable and efficient motion monitoring system There is.

1 is a block diagram schematically showing the configuration of a video-based abnormal motion monitoring system according to an embodiment of the present invention.
FIG. 2 is a reference diagram for explaining a determining unit in FIG. 1. FIG.
3 is a flowchart sequentially illustrating a video-based abnormal motion monitoring method according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the video-based abnormal operation monitoring system and method according to an embodiment of the present invention.

Referring to FIG. 1, an image-based abnormal motion monitoring system according to an embodiment of the present invention includes a photographing unit 10 and a monitoring device 20. The photographing unit 10 and the monitoring device 20 may communicate with each other through a network such as an internet network.

The photographing unit 10 acquires an image of a photographing area by capturing a specific area in the installation space and delivers the image to the monitoring device 20 in real time. In this case, the image may include a color image or an infrared image.

The photographing unit 10 is a color camera that operates during the day to acquire a color image, an infrared camera that operates at night to acquire an infrared image, and checks whether the light is illuminated or day / night by a light sensor or timer. It may include a controller for controlling the operation of the camera and the infrared camera.

Meanwhile, when the dynamic object appears in the image transmitted from the photographing unit 10, the monitoring apparatus 20 expresses the dynamic object existing in the image as an operation area bounding box and changes the size of the operation area bounding box. Monitors occurrences of abnormal behavior of dynamic objects. At this time, the abnormal operation preferably includes a sudden falling operation, that is, a falling operation and a falling operation.

The monitoring device 20 includes an abnormal operation image storage unit 21, a background image storage unit 22, a background image management unit 23, a detection unit 24, an extraction unit 25, a setting unit 26, and an output. The unit 27 and the determination unit 28 may be included.

Here, the abnormal operation image storage unit 21 stores an image corresponding to an abnormal operation occurrence among images transmitted from the photographing unit 10.

The abnormal operation image storage unit 21 may include a buffer module and a storage module.

The buffer module stores an image transmitted from the photographing unit 10, but stores only the latest image. That is, when the buffer module receives an image from the photographing unit 10, the buffer module stores the continuous image frames constituting the image in a playback order, and stores an image frame having a predetermined time elapsed from the initial storage time point among the stored image frames. As it is continuously removed, only the latest image frame is stored.

When an abnormal operation of the dynamic object occurs, the storage module obtains an image corresponding to a predetermined time before and after the abnormal operation occurs from the buffer module and stores the image as a corresponding image when the abnormal operation occurs.

Meanwhile, the background image storage unit 22 stores a background image frame.

The background image manager 23 obtains a background image frame from an image transmitted from the photographing unit 10 through a background subtraction technique and stores the background image frame in the background image storage 22. As a background separation technique, an adaptive Gaussian Mixture Model, which is a statistical background model, and an improved Double Frame Difference technique may be used.

That is, the background image manager 23 designates any one frame among the continuous image frames constituting the image transmitted from the photographing unit 10 as a temporary background image frame, and then the temporary background image frame and the photographing unit 10. By comparing pixel values (luminance, etc.) between successive image frames constituting the image transmitted from the image, the moving object is distinguished from the moving object in each image frame, and then the dynamic object is detected in the image frame. A background image frame including only the background is extracted and extracted, and the generated background image frame is stored in the background image storage unit 22.

In addition, the background image manager 23 compares the pixel values between the background image frames stored in the background image storage 22 and the continuous image frames constituting the image transmitted from the photographing unit 10 to each other. After recognizing the moving object and the moving background, the dynamic object is extracted from the image frame to generate a new background image frame including only the background, and the generated new background image frame is used as the background image storage unit 22. ), The background image frame stored in the background image storage unit 22 can be periodically updated.

In addition, when the background image manager 23 extracts the dynamic object from the image frame to generate a background image frame including only the background, the background image manager 23 performs a morphological operation and a labeling algorithm on the region from which the dynamic object is extracted. Noise such as small holes can be removed.

The detector 24 detects dynamic object appearance image frames among successive image frames constituting an image transmitted from the photographing unit 10 or 10 based on the background image frame stored in the background image storage unit 22.

For example, the detector 24 calculates a pixel value difference between the background image frame stored in the background image storage 22 and each image frame transferred from the photographing unit 10, and the calculated pixel value difference is a threshold. ) Is determined as the pixel in which the movement occurs, and the image frame having the pixel in which the movement occurs is detected as the dynamic object appearance image frame.

The extractor 25 includes positions, colors, pixel values, and the like of pixels in the dynamic object appearance image frame detected by the detector 24 based on the background image frame stored in the background image storage unit 22. Extract dynamic object feature data.

The setting unit 26 recognizes a dynamic object existing in each dynamic object appearance image frame detected by the detection unit 24 using the dynamic object feature point data extracted by the extractor 25, and recognizes the dynamic object. After extracting the motion region size and the motion position of, the motion region bounding box and the bounding box center point of the dynamic object for each dynamic object appearance image frame are set using the motion region size and the motion position of the extracted dynamic object. Here, it is preferable that the operation area bounding box has a minimum rectangular shape including a dynamic object.

The setting unit 26 preferably uses a statistical moment analysis method to extract the motion region size and the motion position of the dynamic object existing in each dynamic object appearance image frame.

For example, the setting unit 26 uses horizontal equations (x _c ) and vertical positions (y _c ) of the bounding box center point by using moment equations such as [Equation 1] and [Equation 2] below when analyzing the image moment. ), And the horizontal length and the vertical length of the operation area bounding box may be calculated using the obtained position of the bounding box center point.

In Equations 1 and 2, 'i' and 'j' represent 'x' and 'y' axis positions in the image, and 'M (i, j)' corresponds to a dynamic object. Represents an area. In addition, the denominator in [Equation 1] and [Equation 2] can be calculated using a zero-order moment analysis, the molecule can be calculated using a first-order moment analysis.

Meanwhile, the output unit 27 outputs an alarm for notifying occurrence of an abnormal operation when an abnormal operation of the dynamic object occurs.

The output unit 27 may include a sound module and a display module.

The sound module outputs an abnormal operation occurrence alarm in the form of an alarm voice or an alarm alarm when an abnormal operation of the dynamic object occurs.

When an abnormal operation of the dynamic object occurs, the display module displays at least one of the alarm message text / pattern and the abnormal operation occurrence image stored in the abnormal operation image storage unit 21.

The determination unit 28 tracks the size change of the motion area bounding box based on the bounding box center point set in each dynamic object appearance image frame, and determines whether an abnormal operation of the dynamic object occurs according to the size change of the motion area bounding box. The result of the determination is provided to the abnormal operation image storage unit 21 and the output unit 27.

When the location of the bounding box center point set in the adjacent frame among the dynamic object appearance image frames is located in the predetermined search area, the determination unit 28 determines the dynamic objects existing in the adjacent frame as the same object, and matches the template. Using the Template Matching technique, the similarity between dynamic objects that exist in adjacent frames and is determined to be the same object can be measured.

In addition, the determination unit 28 checks whether or not the amount of change in the aspect ratio of the operation area bounding box exceeds a predetermined reference value, and if it is determined that the reference value is exceeded, it is determined that an abnormal operation of the dynamic object occurs.

In addition, as shown in FIG. 2, the determination unit 28 includes a first line segment orthogonal to any one side of the bounding box center point and an operating region orthogonal to the first line segment from the bounding box center point. If the angle θ between the second line connecting one of the two end points of the bounding box exceeds the predetermined reference angle th _하여 , and if it is determined that the reference angle exceeds the reference angle, It is determined that an abnormal operation occurs. For example, the first line segment may be a line segment orthogonal to the lower side of the operating area bounding box from the bounding box center point, and the second line segment may be a line segment connecting any one of both endpoints of the lower side of the operating area bounding box from the bounding box center point. .

At this time, the determination unit 28 may obtain an angle between the first line segment and the second line segment through Equation 3 below.

Here, 'a' is the length of a portion corresponding to a point intersecting the first line segment and the second line segment of the lower side of the operating area bounding box, and 'b' represents the length of the first line segment.

In the above-described configuration, a video-based abnormal operation monitoring method according to an embodiment of the present invention will be described with reference to FIG. 3 as follows.

First, the photographing unit 10 transmits the image of the photographing area to the monitoring device 20 in real time by photographing a specific area in the installation space (S100).

The image transmitted to the monitoring device 20 through the above step S100 may be a color image or an infrared image according to the presence or absence of illumination or day / night.

After the above step S100, the monitoring device 20 obtains a background image frame from the image through a background separation technique (S200).

In operation S200, the monitoring apparatus 20 designates one of the continuous image frames constituting the image as a temporary background image frame, and then configures the temporary background image frame and the image transmitted through the operation S100. Through the comparison of pixel values between successive image frames, the moving object and the moving background are distinguished from each other in each image frame, and the dynamic object is extracted from the image frame. The generated background image frame is stored therein.

In addition, in operation S200, the monitoring apparatus 20 may perform a motion within each image frame by comparing pixel values between a background image frame stored therein and successive image frames constituting an image transmitted through the operation S100. After recognizing the dynamic object and the motionless background, the dynamic object is extracted from the image frame to create a new background image frame consisting of only the background, and the generated new background image frame is stored internally. The background image frame may be periodically updated.

Next to step S200, the monitoring apparatus 20 detects dynamic object appearance image frames from successive image frames constituting the image transmitted through step S100 based on the background image frame (S300).

In operation S300, the monitoring apparatus 20 calculates a pixel value difference between the background image frame acquired through step S200 and each image frame transmitted through step S100, and the calculated pixel value difference is equal to or greater than a threshold. The pixel is determined to be a motion pixel, and an image frame having the motion pixel is detected as a dynamic object appearance image frame.

After the above step S300, the monitoring device 20 recognizes a dynamic object present in each dynamic object appearance image frame (S400).

In operation S400, the monitoring apparatus 20 extracts dynamic object feature point data including positions, colors, and pixel values of pixels in which a motion occurs in the dynamic object appearance image frame detected through operation S300, and extracts the same. The dynamic object feature point data may be used to recognize a dynamic object existing in the dynamic object appearance image frame.

After the above step S400, the monitoring apparatus 20 extracts an operation region size and an operation position of the dynamic object existing in each dynamic object appearance image frame, and then uses the extracted operation region size and operation position of the dynamic object. An operation region bounding box and a bounding box center point of the dynamic object for each dynamic object appearance image frame are set (S500).

In the above step S500, the operation area bounding box is preferably made of a minimum rectangular shape or the like containing a dynamic object.

In operation S500, the monitoring apparatus 20 may use a statistical moment analysis method to extract an operation region size and an operation position of a dynamic object existing in each binary image frame.

After the above step S500, the monitoring apparatus 20 tracks the size change of the operation area bounding box based on the bounding box center point set in each dynamic object appearance image frame, and the abnormality of the dynamic object according to the size change of the operation area bounding box. It is determined whether the operation occurs (S600).

In operation S600, when the position of the bounding box center point set in the adjacent frame among the dynamic object appearance image frames is located in the predetermined search area, the monitoring apparatus 20 determines that the dynamic objects existing in the adjacent frame are the same object. Using the template matching technique, the similarity between dynamic objects existing in adjacent frames and determined to be the same object can be measured.

In operation S600, the monitoring apparatus 20 checks whether the aspect ratio change amount of the operation area bounding box exceeds a predetermined reference value, and if it exceeds the reference value, determines that an abnormal operation of the dynamic object occurs. do.

In operation S600, the monitoring apparatus 20 may further include a first line segment orthogonal to any one side of the bounding box center point and the first line segment orthogonal to the first line segment from the bounding box center point. When the angle θ between the second line connecting any one of the two end points of the lateral side exceeds a predetermined reference angle (th _하여 ), and if the result of the check exceeds the reference angle, an abnormal operation of the dynamic object occurs. Will be judged.

As a result of the determination in step S600, when an abnormal operation of the dynamic object occurs, the monitoring device 20 outputs an alarm for notifying occurrence of the abnormal operation (S700).

In operation S700, when the abnormal operation of the dynamic object occurs, the monitoring apparatus 20 outputs an alarm voice or an alarm alarm, and stores the abnormal operation occurrence image therein, or generates an alarm message phrase / pattern and an internal stored abnormal operation. At least one of the images may be displayed.

The video-based abnormal motion monitoring system and method according to the present invention are not limited to the above-described embodiments, and may be variously modified and implemented within the range allowed by the technical idea of the present invention.

10: photographing unit 20: monitoring device
21: abnormal operation image storage unit 22: background image storage unit
23: background image management unit 24: detection unit
25: extraction section 26: setting section
27: output unit 28: determination unit

Claims (11)

A photographing unit obtaining an image of a photographing area; And
When the dynamic object appears in the image acquired through the photographing unit, the dynamic object existing in the image is represented as an operation area bounding box, and the occurrence of abnormal motion of the dynamic object is monitored according to the change in the size of the operation area bounding box. An image-based abnormal motion monitoring system comprising a monitoring device.
The method of claim 1,
The monitoring device includes:
A detector for detecting dynamic object appearance image frames among successive color image frames constituting a color image transmitted from the multi-viewer based on a background image frame;
An extraction unit for extracting dynamic object feature point data in the dynamic object appearance image frame detected by the detection unit;
Recognizing dynamic objects existing in each dynamic object appearance image frame detected by the detector using the dynamic object feature point data extracted through the extractor, and using the motion region size and the motion position of the recognized dynamic object A setting unit configured to set an operating area bounding box and a bounding box center point of the dynamic object for each of the dynamic object appearance image frames; And
A determination unit which tracks the change in the size of the motion area bounding box based on the bounding box center point set in each dynamic object appearance image frame through the setting unit, and determines whether an abnormal operation of the dynamic object occurs according to the size change of the motion area bounding box. Image-based abnormal motion monitoring system, characterized in that made.
The method of claim 2,
The determination unit,
When the location of the bounding box center point set in the adjacent frame among each dynamic object appearance frame is located within a certain search area, the dynamic objects existing in the adjacent frame are regarded as the same object, and the template matching method is used. An image-based abnormal motion monitoring system, characterized in that for measuring the similarity between the dynamic objects present in the adjacent frame and determined to be the same object.
The method of claim 2,
The determination unit,
And determining whether an abnormal motion of the dynamic object occurs according to whether the change in the aspect ratio of the motion area bounding box exceeds a reference value.
The method of claim 2,
The determination unit,
Any one of a first line segment orthogonal to any one side of the bounding box center point from the bounding box center point, and either end point of a side of the side bounding the operating area bounding box orthogonal to the first line segment from the bounding box center point And determining whether an abnormal motion of the dynamic object occurs according to whether the angle between the second line segments exceeds the reference angle.
6. The method according to any one of claims 1 to 5,
Wherein,
A color camera for obtaining a color image;
An infrared camera for acquiring an infrared image; And
An image-based abnormal motion monitoring system comprising a controller for controlling the operation of the color camera and the infrared camera by checking the presence or absence of illumination by a light sensor or timer.
The method according to claim 6,
An abnormal operation image storage unit which stores an image corresponding to an abnormal operation occurrence among images transmitted from the photographing unit; And
And an output unit configured to output an alarm for notifying occurrence of an abnormal operation when the abnormal operation of the dynamic object occurs.
Real-time delivery of the image of the photographing area to the monitoring apparatus by the photographing unit;
Obtaining a background image frame from an image transmitted from the photographing unit in the monitoring apparatus;
Detecting, by the monitoring apparatus, dynamic object appearance image frames among image frames constituting an image transmitted from the photographing unit based on the background image frame;
Recognizing, by the monitoring apparatus, a dynamic object present in each dynamic object appearance image frame;
The monitoring apparatus sets a bounding box and a bounding box center point of the dynamic object for each dynamic object appearance image frame by using the motion region size and the motion position of the dynamic object present in each dynamic object appearance image frame. process;
Tracking, by the monitoring apparatus, the size change of the operation area bounding box based on the bounding box center point set in each dynamic object appearance image frame, and determining whether an abnormal operation of the dynamic object occurs according to the size change of the operation area bounding box; And
And outputting an alarm for notifying occurrence of an abnormal operation when an abnormal operation of a dynamic object occurs as a result of the determination by the monitoring apparatus.
9. The method of claim 8,
The image transmitted from the photographing unit to the monitoring device,
Image-based abnormal motion monitoring method characterized in that the color image or infrared image.
10. The method of claim 9,
The process of determining whether the abnormal operation occurs,
Confirming whether a change in an aspect ratio of the operating area bounding box exceeds a reference value; And
And determining that an abnormal motion of the dynamic object occurs when a change in the aspect ratio of the motion area bounding box exceeds a reference value.
10. The method of claim 9,
The process of determining whether the abnormal operation occurs,
A first line segment orthogonal to any one side forming the operating area bounding box from the bounding box center point, and one end point of either side of the side forming the operating area bounding box orthogonal to the first line segment from the bounding box center point; Checking whether the angle between the second line segments exceeds the reference angle; And
And determining that an abnormal motion of the dynamic object occurs when the angle between the first line segment and the second line segment exceeds a reference angle.

Images