JPH06294808A - Method and device for moving body detection - Google Patents

Abstract

PURPOSE:To detect a moving body as an object by extracting characteristic parameters in image information on the moving object and extracting limitedly the image of the object in the parameter values. CONSTITUTION:By comparing the first picture taken with a monitor camera in a monitoring object region and the second picture taken with the same camera for the same region a few second later, the variation amount is obtained. The calculation of the absolute value of the difference in the two pictures are repeated plural times, the pictures are processed for binarization with a certain threshold value and a certain image in the resulting picture is labeled one by one. For the each labeled binarized image of the object, parameters such as the inclination of the inertia axis, the ratio of length and breadth of circumscribed quadrilateral of the object image, etc., are detected to classify the object. In the case the object is judged as aiming object, the presumed aiming object in the labeled object in the image data is selected by one and it is indicated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving body detecting method and apparatus, and more particularly to a moving body detecting method and apparatus when there is a moving body other than a moving body to be detected in image information.

[0002]

2. Description of the Related Art In thermal power plants, there is an increasing demand for automation of equipment monitoring due to demands for longer life of equipment and labor saving of patrols. Therefore, a method of intensively monitoring important devices (for example, a fuel supply device, a burner section, valves of main pipes, etc.) on a monitor TV by using a monitor TV camera is widely used. In addition, with the recent progress of image processing technology, FA (Factory Automati) is often used to determine whether there is an abnormality by performing image processing on an image of a surveillance television camera.
on) and it tends to increase as a spillover from the field.

However, the television camera has a limited monitoring area and cannot cover the entire boiler plant. Actually, patrols perform visual inspection 6 to 7 times a day or visual inspection by hearing. This inspection inspection work requires a great deal of skill to detect anomalies, which is a considerable burden. In addition, centralization of monitoring as a need for power plants,
Or, there is a human limitation due to labor saving, and currently, a robot that performs a site inspection on behalf of a patrol officer is being developed.

When analyzing the function of human patrol,
(1) situation grasp, (2) movement, (3) situation judgment (normal or abnormal), (4) processing function, system functions corresponding to this, (1) sensor technology, (2) movement Functions, (3) computer technology, and (4) driving guidance display. When these systems are embodied and a robot inspection system is realized, this outline mainly shows the following three parts as shown in FIG. 4, that is, the monitoring robot 1 that patrols and inspects the site, and the information from the monitoring robot is processed. , An anomaly judgment device 4 for judging the presence or absence of an anomaly, a raw image and a live sound from the site are presented, and if the anomaly judgment device judges that there is an anomaly, the man-machine interface (the man-machine interface) For example, an image display device) 5.

The monitoring robot 1 includes an ITV camera 2 which has a sense of sight, a sense of hearing, a sense of smell, and a sense of touch among the five human senses.
A microphone 3, a gas sensor and a non-contact type thermometer are mounted. These sensors are used to move around the site and collect various information. This information is processed by the abnormality determination device 4 such as image processing for image data and acoustic processing for acoustic data to determine the presence or absence of abnormality. In FIG. 4, 6 is a robot control unit, 7 is a communication control unit, 8 is a robot traveling axis, 9 is a sensor unit turning axis, and 10 is a sensor unit vertical axis.

Here, an example of detecting an abnormality will be described. The abnormal items differ depending on the monitoring target and the monitoring location, but in the case of a boiler plant, it is mainly limited to oil leaks and steam leaks. These leaks lead to serious accidents if left unchecked, and early detection is required. The oil leak is accompanied by only the image change, and the image processing focusing only on the image change is performed to extract the abnormality. An abnormality detection image processing procedure that is currently performed will be described below by taking oil leak detection as an example.

An image processing flowchart is shown in FIG. As shown in the figure, the robot stores the normal state of the target device or place as the first screen in the memory as the reference image A. Speaking of a boiler, it is an image at a normal time at a place where an oil leak is likely to occur (a flange portion of an oil pipe, an oil pan in front of a burner (a pan for leaking oil), etc.). The binarization process is a process in which an image is captured in pixel units, and one having a high luminance is black and one having a low luminance is white based on a certain threshold value, and is generally performed as a preprocessing of the image processing. It is a thing.

When the robot makes a patrol and stops again when the first screen is stored, the processing shown in the flowchart in the figure is performed. That is, the same target captured as the first screen is captured as the second screen B. The second screen is captured by capturing a plurality of screens at short time intervals and averaging the images as a second screen. This averaging process has an effect of thinning a stain or the like due to noise in one screen among a plurality of screens. Then the first
The difference between the image and the second image is subtracted to extract the difference as a difference image, then the difference image is binarized to remove noise, and then the window is set up so that only the necessary part of the difference image is extracted. Create a histogram that represents the area ratio of the black and white parts in the image. In FIG. 3, whether or not there is an abnormality is determined by the area ratio W% of the white portion. The reason for this area calculation is as follows.

[0009] Generally, in image-to-image matching, a shift of about one pixel always occurs even between images captured at the same location with a fixed camera. This is because a minute change, such as the vibration of the target device, is not visible to humans. In addition, in the current surveillance system, the images are obtained by using a moving camera by a robot instead of a fixed camera, and positioning reproducibility is generally limited.
In order to prevent misjudgment of the amount of change due to this deviation, a threshold value is provided as a criterion. This standard is currently set at 3%. That is, the area is calculated, and if the change amount is within 3%, it is considered as an image change due to the deviation, it is determined that there is no abnormality, and if it is 3% or more, it is determined that it is abnormal.

This kind of abnormality detection method is general, and is used as a product in abnormality determination logic such as an unmanned monitoring digital image sensor (however, the camera is fixed). The example of abnormality detection determination using the configuration of the robot monitoring system in the plant and the image processing method has been described above. However, as described above, the reference image must always be held as a memory, which causes the following problems. (1) It is easily affected by changes in sunshine.

When a reference image is taken in the morning and inspected in the daytime, there is a change in the position of the shadow and the illuminance, which may be judged as abnormal. (2) The monitoring points are highly limited. The reference image must be provided for each inspection location and monitoring target, and the robot memory becomes huge.

When the above problems are sorted out, it is not possible to classify the fact that a reference image is required, and those that should be judged as abnormal (water, oil, steam) and those that are not (human, shadow, etc.). It will be summarized. In addition, there is an intruder / suspicious person detection system using a camera outdoors or indoors, which is also being developed. Conventionally, this system was performed by humans monitoring the images taken by surveillance cameras, but in the past, humans performed image processing to detect intruders and suspicious persons and maintain facilities. The demand for a system that promotes labor saving in safety and crime prevention is increasing.

As the image processing method applied to this system, as in the above-mentioned method, the main method is subtraction between the reference image and the image with respect to the target screen. When considering, there are the following problems. (1) The movement of the natural environment such as the sway of plants and trees, the wave, the ebb and flow of the tide, etc. is also judged to be abnormal due to the difference from the reference image. (2) The appearance of small animals and insects is also considered abnormal. (3) If the setting is all-day monitoring, the image to be monitored by a standard visible camera, infrared camera, night-vision camera, etc. against changes in temperature and illuminance in the field of view due to the effects of day and night, weather, etc. Amount of change other than

Attempts to follow this change often result in less unmanned systems. When the above problems are sorted out, it is due to the fact that a reference image is provided as in the method described above.

[0015]

The above-mentioned prior art requires a reference image in a normal state in performing the abnormality determination method in the image processing method, and the quantification and model for many parameters in a normal state are required. There is a problem such as a low limit of misjudgment or abnormality detection because no consideration is given to proper removal of disturbance that is difficult to realize. In short, there are an infinite number of reference images under normal conditions, so if the difference between the internal change width and the normal / abnormal change width is not so large, the Differentiation is easy to become difficult.

Further, since it is not possible to recognize each object due to the temporal change of the image, there is a problem that "irregular normal change" is erroneously recognized and judged as abnormal. An object of the present invention is to improve a conventional image processing method for a detection target such as an intruder or a suspicious person outdoors or outdoors, and process a target image including various disturbances and ambiguous information. It is also for constructing a moving body detection method and apparatus capable of appropriately determining the situation.

[0017]

In order to achieve the above object, the first invention of the present application is to provide a method for detecting a moving object in a monitoring area based on image information from an image information extracting device for the monitoring area. A first step of obtaining a difference image between the first image and the second image taken by the image information extracting device at a predetermined time, and a difference image obtained in the first step by repeating the same operation as the above step. A step of collectively obtaining a plurality of difference images; a step of binarizing the difference images;
A step of obtaining a feature amount as an object shape for each binarized image of the object in the difference image; and a step of detecting a monitoring target moving body based on the feature amount. And a moving object detection method.

A second invention of the present application is a method for detecting a moving object in a surveillance area based on image information from the image information retrieval apparatus for the surveillance area, in which the surveillance area is photographed by the image information retrieval apparatus at predetermined intervals. A step of repeatedly obtaining the difference image between the first image and the second image to obtain a plurality of difference images to obtain a moving body image in the monitoring area; and a step of binarizing the difference image with a threshold value. And a step of labeling a moving body image in the difference image, a step of detecting a feature amount as an object shape of a binarized image of each labeled object, and the feature amount Based on the feature amount and the degree of correlation between the object images, the object that is determined as the monitoring target object is detected from among the labeled objects. It relates to a mobile detection method to be.

A third invention of the present application is a device for detecting a moving object in a surveillance area based on image information from the surveillance camera in the surveillance area, wherein the surveillance camera takes a short time image of the surveillance area. A moving object image detecting means for detecting a difference image between the image and the second image, a means for binarizing the difference image by a threshold value, and an object for each moving object image in the difference image Means for detecting a characteristic amount as a shape, means for detecting a moving object to be monitored based on the characteristic amount, means for detecting a moving speed of the monitored object, and means for displaying the monitored object and its moving speed The present invention relates to a moving body detection device including:

[0020]

The image processing method according to the present invention extracts characteristic parameters from image information of a moving object outdoors or indoors, and extracts a monitoring target object, for example, a human image from the value of the parameter. However, it can be detected as an intruder or suspicious person, and for other environmental disturbances such as shaking of plants due to wind, tides, changes in sunlight, changes in images due to the presence of small animals, etc. , As well as with characteristic parameters, it can be removed as a disturbance. 6 and 7 show the difference in the feature amount between images of humans and images of animals.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the configuration of the entire system showing an embodiment of the present invention. An image captured from an external information device such as a camera is sent to a moving body detection device that performs image processing. The moving body detection device extracts an image of the moving body from the image data by an image processing method, and extracts a feature amount of each moving body. This feature amount will be described later.

For the parameters of the feature quantity of the moving body extracted here, the one corresponding to an intruder or a suspicious person is selected from the parameter of the feature quantity of the image data of the moving body in the moving body detecting device. . After that, the moving amount of the image is analyzed and sent to the display system so as to collectively display the "intruder / suspicious person" obtained so far. Next, an image processing method performed in the moving body detection device will be described. The outline of the flowchart of this image processing method is shown in FIG. Like the conventional method, this image processing method also uses the difference component between the reference image and the target image. However, in the present invention, the same area as the first screen taken by the surveillance camera for the surveillance target area after a predetermined time has passed. The second screens taken by the same camera are compared and the amount of change (difference image) is determined. The time difference between the capture of these two screens is within a few seconds, and since there is almost no change in illuminance within this time, it is planned according to the average brightness in the screen, and there is almost no change in the brightness of the screen. . Then, the absolute value of the difference between these two screens is calculated. By this calculation, the amount of change between the reference image (first screen) and the target image (second screen) can be calculated in pixel units. The above-described capture of the first image and the second screen and calculation of the absolute value of the difference between the two images are repeatedly executed a plurality of times. Through the steps up to this point, only the image of the moved object is in a state of being obtained as the difference image. Here, this screen is binarized by a certain threshold value, and the resulting images in the screen are each labeled. This labeling is to number the object images in the image one by one, and is performed in order to speed up the subsequent processing and to confirm the correlation for each label.

Next, the feature amount of the binarized image of each labeled object is detected. This feature amount includes the following items: (1) inclination of the principal axis of inertia (the axis at which the moment of inertia is minimum) (inclination angle with respect to the horizontal or vertical direction), (2) aspect ratio of the circumscribed rectangle of the object image. , (3) number of holes in the binarized image, (4)
The area on the image (5) The ratio of the perimeter to the area is performed.

With respect to the above items, the shape of the object is analyzed by an image processing method, and the object is classified based on the parameter amount (feature amount). If the object is determined to be an intruder or a suspicious person, one of the labeled objects in the image data that is considered to be an intruder (excluding the object already determined to be the intruder) is selected. This selection criterion is performed by analyzing the degree of correlation between images in addition to the above five items. This correlation is performed for the first screen and the second screen corresponding to the two labeled objects, and analyzed for the coordinate system and the shading of pixels,
From the degree of correlation, it is determined whether two objects (intruders) are the same, and the detection of the intruder and the amount of movement thereof are sent to the display system and displayed. An image processing flowchart of this speed detection and an outline of the correlation coefficient are shown in FIG.

Although the system configuration of the present invention is camera → moving object detecting device → display system, it is also possible to use a generalized image processing device as the moving object detecting device. However, in order to maintain high speed, a sequential arithmetic unit (CPU)
It is necessary that the number of bits of is quite large. In addition, the specifications of the camera that is the external spatial information device are currently an infrared camera, a night-vision (starlight) camera, and a visible camera, but specifications such as a fish-eye camera and a wide-angle camera can be considered as necessary. .

[0026]

As described above, the present invention does not require a reference image in a normal state of a monitoring target area, and appropriately removes various disturbances to make an accurate situation judgment for a detection target. it can.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of an embodiment of the present invention.

FIG. 2 is a flowchart of image processing according to an embodiment of the present invention.

FIG. 3 is a flowchart of image processing according to a conventional technique.

FIG. 4 is a diagram showing a conventional robot inspection system.

FIG. 5 is a flowchart of speed detection of a moving object by image processing according to the present invention.

FIG. 6 and

FIG. 7 is a diagram showing a difference in feature amount between an image of a person image and an image of an animal according to the present invention.

Claims (3)

[Claims] 1. A method for detecting a moving object in a surveillance area based on image information from the image information retrieval apparatus for the surveillance area, wherein a first image and a first image taken by the image information retrieval apparatus for the surveillance area at a predetermined time interval are provided. The first step of obtaining the difference image between the two images and the same operation as the above step are repeated.
Step of obtaining a plurality of difference images together with the difference image obtained in the step of, the step of binarizing the difference image, and each of the binarized images of the object in the difference image The method for detecting a moving object, which comprises: a step of obtaining a characteristic value as an object shape of the above item; 2. A method of detecting a moving object in a surveillance area based on image information from the image information retrieval apparatus for the surveillance area, wherein the first image and the first image taken by the image information retrieval apparatus at the surveillance area are separated by a predetermined time. Repeating the operation of obtaining the difference image of the two images to obtain a plurality of difference images to obtain a moving object image in the monitoring area; binarizing the difference image with a threshold value; A step of performing labeling on a moving object image, a step of detecting a feature amount as an object shape in a binarized image of each labeled object, and a classification of the object based on the feature amount. And a step of detecting an object determined to be a monitoring target object from the labeled objects based on the feature amount and the degree of correlation between the object images. Law. 3. An apparatus for detecting a moving object in a surveillance area based on image information from the surveillance camera in the surveillance area, the first image and the second image taken by the surveillance camera in the surveillance area at short intervals. A moving object image detecting means in the monitoring area for obtaining a difference image, a means for binarizing the difference image by a threshold value, and a feature amount as an object shape for each moving object image in the difference image. A means for detecting, a means for detecting a moving object to be monitored based on the characteristic amount, a means for detecting the moving speed of the object to be monitored, and a means for displaying the monitored object and its moving speed. Characteristic moving object detection device.