JP2002262282A - Image supervisory method and image supervisory device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and a device that enable automatic detection of a fault from an image of a supervisory object area in a short processing time with sufficient reliability. SOLUTION: The required number of observation windows 21 are placed in an area desired to detect changes generated therein from an image obtained imaging a supervisory object and the observation windows corresponding to a plurality of images different on time base are compared so as to automatically detect a long time residence, appearance or missing of part of an object.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a monitoring method and a monitoring apparatus using a series of images photographed by an image pickup apparatus, and more particularly to a monitoring time of an object in a predetermined monitoring target area and a monitoring method of the monitoring apparatus. The present invention relates to a monitoring method and a monitoring device suitable for detecting the appearance and disappearance of an object.

[0002]

2. Description of the Related Art In recent years, from the viewpoint of preventing illegal parking and crime prevention, a monitoring system using a television camera has been widely used. Because reliability depends mainly on the ability of the person monitoring, it requires a lot of effort and
Since human attention and concentration are limited, there is a problem in maintaining sufficient reliability. Therefore, in recent years, various methods for analyzing image data and automatically detecting occurrence of abnormality have been proposed and put into practical use.

[0003] Here, an abnormal condition is an illegal parking or crime described above, such as the presence of a specific subject image in a monitored area screen for a long time, or the appearance of a specific subject not existing in the relevant area. This is a change in image data that has a high possibility of leading to the occurrence of image data. For this reason, in the related art, image data of the entire screen or a part thereof at a different imaging time is overlapped, and a part of the subject in the screen is overlaid. By checking the degree of overlap, for example, in the case of a television image, an abnormality such as stagnation of a subject is detected by calculating a difference between a previous frame and a subsequent frame.

[0004]

The above prior art does not take into consideration the point that the shape of the subject to be monitored (monitoring target) is not specified and the amount of data of the image to be processed is not considered. There is a problem that a large amount of data processing is required for analyzing the abnormality. That is, in the prior art, since the entire screen is compared, the amount of image data becomes large, not only takes a long time to process, but also becomes insensitive to a small change in a part, and the shape differs individually. In order to check the temporal change of the subject, it is necessary to determine each comparison range corresponding to the subject having a different shape, so that preparation and image processing are troublesome.

An object of the present invention is to provide an image monitoring method and an image monitoring apparatus capable of automatically detecting an abnormality from an image of a monitoring target area in a short processing time and with sufficient reliability. That is, the present invention automatically issues an alarm when a initially set screen change occurs, even when a person is not always watching the screen when monitoring with a camera image. This is for judging the situation by looking at the screen of the camera, or for automatically determining a predetermined treatment.

[0006]

First, an object of the present invention is to set at least two observation windows on a screen taken at a different time by the same video camera, and to set an image for each of the at least two corresponding observation windows. This is achieved by performing a logical operation on the information indicating the presence or absence of a change in the data to detect whether or not a certain event has occurred.

[0007] In addition, the above object is also to set at least two observation windows on a screen taken at the same time by an individual video camera, and to set in advance from among these at least two observation windows in accordance with the observation purpose. Set multiple observation window groups together,
This is achieved by performing a logical operation on information representing the presence or absence of a change in the image of the observation window in the plurality of observation window groups, thereby detecting whether or not a predetermined specific event has occurred.

[0008] Further, the above object is to capture at least two images of the monitoring target area before and after in time, and to detect the movement of the object in the monitoring target area by comparing these at least two images. In the image monitoring method of the type, at least two observation windows are set on the image plane, and the comparison of the at least two images is performed by the observation windows between the corresponding observation windows on each of the image planes. This is achieved by limiting the comparison to the cut image.

At this time, the object is achieved even if the at least two observation windows are each smaller than the object on the image plane and are arranged according to the shape of the object on the image plane. can do. At this time, the above object is achieved even if any one of the shape, the number, and the position of the at least two observation windows can be arbitrarily set on the display image plane of the monitoring target area.

Further, a window for normalizing brightness is set on the image plane, and a signal for normalizing brightness is generated from an image cut out by the window for normalizing brightness. The above object can be achieved even if the level of the image signal is normalized by the following.

Similarly, the brightness normalization signal is generated independently for each color signal component from the result of measuring and correcting the R, G, and B color signal components constituting the image signal. Even so, the above object is achieved. Also, at this time, the shape and number of the brightness normalizing windows,
The above object can be achieved even if any of the positions can be arbitrarily set on the display image plane of the monitoring target area.

Next, the above-mentioned object can be achieved even if the image of the monitored area is taken in through a moving object removing filter. Means for multiplying the signal by a coefficient α, adding the result to the multiplication of each pixel of the immediately preceding image by a coefficient (1−α), and outputting the result as a current image, wherein 0 <α≪ The above object is also achieved by setting the value to 1.

According to the present invention, there is provided an image capturing apparatus for capturing a general landscape as an image and determining whether or not a part of the landscape has changed within a predetermined time.
Without paying attention to the change of the entire screen, a single or an appropriate number and an appropriate number may be placed at an arbitrary position of a single or a plurality of parts or an entire part in an area where it is desired to observe whether there is a change in the screen. An observation window having a different shape is provided, and only the image change for each observation window is compared to determine whether or not a part or all of the subject reflected on the screen has changed.

At this time, a plurality of windows smaller than the screen are set in an area to be detected on the imaging screen of the video camera,
At least one of the positions for the image cropped by this window
One makes a comparison. At this time, the determination as to whether or not the specific subject is continuously staying on the screen temporally is not limited to the above-described situation of one window, and the target subject is covered with a plurality of windows. The judgment is made based on the comparison result of the plural windows.

Here, if the observation window of the subject is made smaller in order to judge the difference in the image over time, the proportion of the change of the subject in the observation window becomes larger, and the size and position of the subject become larger. It is easy to maintain a sufficient sensitivity coefficient even when it is not constant. Also, even when comparing images in a relatively large range, if the target is viewed as a set of multiple observation windows,
If the detection for each observation window is properly performed, the comparison result of a small area can be obtained well.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image monitoring method and an image monitoring apparatus according to the present invention will be described in detail with reference to the illustrated embodiments. Here, as one embodiment, the present invention
Description will be given using an embodiment in which the present invention is applied to a system for monitoring the parking status of a car. FIG. 1 is an example of a monitoring target area in this embodiment. Here, a landscape centering on a roadside zone of a road is shown.

FIG. 1 shows a state where there is no object to be monitored, including an automobile, in the roadside portion of the road, so that the image is continuously monitored as it is. No change on the screen. Therefore, this screen is hereinafter referred to as a reference screen. Next, FIG. 2 shows a situation in which a plurality of (24 in the figure) observation windows 21 of a predetermined shape (same rectangle) are set on the image processing apparatus on the reference screen of FIG. 1 in this embodiment. It is a thing.

Here, these observation windows 21 are arranged so as to cover a portion where the change of the image is to be monitored, that is, in this case, a place where a roadside vehicle is likely to park and stop. The size of each observation window 21 is set such that two to four observation windows cover one vehicle and are arranged along the roadside zone. This is in order to cope with the fact that the position and size of the car will change each time.

Next, FIG. 3 shows a situation in which a car is parked on this roadside zone.
When comparing the corresponding observation windows of the images taken at intervals of (T), it can be confirmed that the specific observation window is different from the reference screen, and the image 3T before and the 2T time
If we can confirm that the image before time and the image before time T have not changed from the current image, the cars covered by these observation windows have been parked since 3T ago. I understand.

Therefore, if the degree of change in the image of the observation window at this time is confirmed by comparing the corresponding observation windows, the contents of the observation window do not change for a predetermined time or longer, and the observation window on the reference screen is not changed. Observation windows with different contents can be extracted and an alarm necessary for detecting parking can be issued. As for the alarm at this time, not only the screen display but also the alarm notification by sound such as voice or beep sound may be used together.

More specifically, for example, images are taken every 5 minutes (T = 5 minutes), and among these images, an image different from the reference screen in the specific observation window is the same. If four screens are continuous, it means that something that was not in the reference screen continues to exist in the portion covered by the observation window for at least 15 minutes or more.

In this case, an alarm can be automatically issued by setting an alarm to be generated when four screens are consecutive. At this time,
At the same time, as described later, if a display such as flickering the corresponding observation window is performed on a predetermined display screen, a visual warning can be issued at the same time. We just need to take action.

FIG. 4 shows an image when the monitoring result at this time is displayed on a predetermined monitor device.
As a result of comparing the images in the observation window as described above, it is found that the observation window has changed from the reference screen, and that the images before time 3T, time 2T, and time T have not changed from the current image. The confirmed observation windows 41 (12 in FIG. 4) are displayed in solid black, so that the person who is monitoring can easily recognize the occurrence of the alarm.

Here, FIG. 4 shows a case where the observation window 41 is displayed in solid black, but by changing image processing contents, only the frame of the observation window 41 is set in advance.
The observation window 41 can be alternately displayed in colors, and the entire observation window 41 can be flickered as it is or in a complementary color.

In this embodiment, by using a pointing device such as a mouse or a man-machine interface such as a keyboard,
An arbitrary value can be set for the time T, that is, the image capturing time interval.

Here, the time T is related to the movement of the monitored object, and as the time T is shortened, the fast-moving monitored object can be detected. It can be applied to the detection of (suspicious person).

Next, the abnormality detection in this embodiment will be described more specifically. First, as described with reference to FIG. 1, a plurality of observation windows 21 are set in the image plane, that is, at least two observation windows 21. At this time, the size of each observation window 21 The size of the object is smaller than the size of the object on the image plane, and these observation windows 21 are arranged in the direction of their arrangement or in such a manner that two or more objects are always superimposed on the object to be monitored (object to be monitored). Set according to the shape.

For example, in the case of the above embodiment, since the monitoring target is an automobile, the size of each observation window 21 is set to be sufficiently smaller than the image of the automobile, as shown in FIGS. In this case, the predicted parking state of the vehicle is arranged along the roadside zone of the road. Therefore, as shown in the drawing, each observation window 21 also extends along the roadside zone. They are arranged.

More specifically, assuming that the object to be detected at this time is, for example, a car that has been parked for a long time as described above, in this case, the car itself and the position on the screen where the car appears will be, for example, It changes every time the car is replaced, so that one observation window does not cover two or more cars.
Regardless of its position, the smallest assumed object must be covered by two or more observation windows.

This is illustrated in FIG. First, in FIG. 8, broken-line frames indicate the respective observation windows, and the check pattern indicates a long-time vehicle (a vehicle that has been parked for a long time).
Here, the light black (gray) is a short-time vehicle (a vehicle that has been replaced in a short time), and the observation window shaded by a halftone dot indicates an observation window in which the vehicle is detected for a long time.

In FIG. 8, first, the case is:
When each observation window is set to the length of two cars, and the size of each observation window is too large compared to the detection target, the long-time vehicle and the short-time vehicle Indistinguishable. Here, the case is when the break of the observation window is between the vehicles, and the case is when the break is over the vehicle.

Next, the case is the case where each observation window is set to be substantially equal to the length of the vehicle pedestal. In this case, as shown in FIG. Although there is no problem, as shown in the case, it is inevitable that the vehicle is separated, and in this case, it is not possible to distinguish between a long-time vehicle and a short-time vehicle.

Also, the case where each observation window is set to be longer than 1/2 of the length of the car stand, also in this case, as shown in the figure, the observation window is divided between the vehicles. There is no problem if it just fits, but at this time, as shown in the case, it is inevitable that the vehicle will be separated.

In this case, if the long-time vehicle and the short-time vehicle alternate, as shown in the figure, the observation window corresponding to the short-time vehicle becomes one, and this observation window If an error occurs due to a disturbance applied to the image data, the error will increase, and the determination accuracy will decrease.

On the other hand, in each case, a plurality of observation windows can correspond to each monitored object, so that the determination can be made clearly and definitely. Here, in the case, the size of the observation window is approximately を of the length of the car.
If set smaller, the case is also 1/3
In this case, three observation windows correspond to the case, and four cases correspond to the case.

As described above, by setting a plurality of observation windows to correspond to one monitored object, for example, sunlight is specularly reflected on a roof or window glass of a car, and observation is performed for a short time. Even when a large level change occurs in the image signal of the window, it is compensated by the image signal of the observation window that has not caused another level change, so that the possibility of malfunction can be suppressed.

At this time, from another viewpoint, the detection sensitivity of the change can be increased by reducing the observation window. This is because detecting the change as a whole, rather than detecting the change in a part of the observation window, improves the sensitivity because the change occurs in many parts.

Therefore, in general, objects such as the above-mentioned monitored objects such as automobiles which appear at positions expected on the screen but cannot appear at small positions cannot be predicted. To detect, it is better to do as follows. (a) The size of the observation window is set to be approximately １ ／ or less of the size of the monitoring target on the screen. (b) The number of observation windows should be two or more.

Hereinafter, the operation of the present invention will be described in detail with reference to FIG. 9. In FIG. 9, the image of the video camera 601 is taken into the computer 324 from the video capture board 603 via the moving object removing filter 602. . The reference image memory 604, the image memory A 605, and the image memory B 6
In step 06, image data of the entire screen is sequentially captured.

More specifically, the reference image memory 604
Has an initial screen, image memory A605 and image memory B
The full-screen data 606 is fetched alternately at a fetch timing every fixed time. That is, the image memory A 605 and the image memory B 606 always store the current image and the image one timing before. Here, one timing before is appropriately set according to the purpose of use of the present invention, such as 5 minutes before, 10 minutes before, or 30 minutes before.

Next, the observation window cutout processing circuit 608 takes in pixel data corresponding to the observation window at a predetermined timing. Specifically, for example, the screen information at the initial time of the five observation windows A, B, C, D, and E that are sequentially adjacent to each other, the screen information one timing before, and the current screen information are the observation windows. The data is captured by the cutout processing circuit 608.

The screen information is compared for each observation window in accordance with a predetermined combination in the observation window comparison determination processing circuit 619. For example, for each of the observation windows A to E, a comparison is made between the screen at the initial time and the current screen, and the screen one timing before and the current screen. As described above, there are various methods for comparison, and one of them is used.

The result is stored in the counter 620. The counter 620 stores those results and sends them to the observation window state combination logic operation circuit 630 and the observation window alarm generation circuit 621.

The observation window alarm generation circuit 621 generates an alarm signal for each observation window according to a predetermined basic combination. Specifically, for example, for each observation window, the screen at the initial time and the current screen are different, and if the screen one timing before and the current screen are the same, there will be an object staying for a long time. Information to display the alarm signal on the screen.

In the observation window state combination logical operation circuit 630,
A logical operation is performed by a predetermined combination. For example, for each observation window, the screen at the initial time and the current screen are different, and if the screen one timing before and the current screen are the same, an object stays for a long time.

As described above, when the presence / absence of the change of the two observation windows A and B, B and C, and C and D is calculated sequentially, the long time spans the two adjacent observation windows. There will be stagnant objects (eg, parked vehicles). In addition, if the screen one timing before and the current screen are the same for the same observation window, there is an intruder (person) at the position of the observation window, and one timing of the specific observation window for a continuous certain period By taking the logical product of the comparison result before and after, the intruder (person)
Can be specified whether or not has stayed for a long time.

The superimposition circuit 622 of the captured image and the alarm generates a signal in which the alarm signal of each observation window and the video signal of the current full screen are superimposed, and the image is displayed on the screen display device 623.
To be displayed. In this way, for example, a parking violation vehicle can be automatically monitored, and can be recorded on a recording device as needed.

Next, regarding the comparison for each corresponding observation window, a method for comparing the brightness of the entire observation window, a method for comparing the brightness of three colors of the video signal of the entire observation window, and a method for comparing the difference for each pixel are described. The method of calculating the sum of the differences over the entire observation window and comparing it with a threshold, or the method of taking the difference for each pixel for each of the three colors and calculating the sum of the differences over the entire observation window and comparing the difference with the threshold for each color Yes, these comparison results are integrated to determine a match or a difference for each combination of observation windows compared.

Then, regarding the setting of the threshold value at this time,
To explain with specific numbers, in the case of a standard TV screen composed of 640 × 480 pixels, one of the practical choices is to set the size of the observation window 21 to 16 × 16 pixels.
In this case, the observation window becomes 256 pixels and 1
If the brightness of a pixel is processed by 8 bits, 25
There are six gradations.

Therefore, taking the difference in the brightness of the observation window, in the experimental example, the same subject is averaged by 1 due to the dispersion of various factors in the image processing system including the camera.
There is about 2% gradation variation per pixel. The total number of gradations and the number of pixels 256 × 256 = 65,536 fluctuates by about 2%. That is, 5 × 2 in total for 256 pixels
About 56 gradations, a change occurs even in an image of the same object.

On the other hand, in the experimental example, if there is a difference of 10.times.256 gradations, it can be considered that the object has changed sufficiently. This is an example.

In the case of this embodiment, the size of the object to be monitored on the screen changes depending on the angle of view of the video camera used for capturing the image or the size of the subject. The size of 21 also changes according to the use condition of the device. Therefore, in this embodiment, the position, size, and shape of the observation window can be set on a monitor screen of a computer using a pointing device such as a mouse. Therefore, according to this embodiment, the specifications such as the number, position, size, shape, and arrangement direction of the observation windows 21 can be easily set while watching the image displayed on the monitor screen.

Next, as in the above-described embodiment, particularly when an image of a subject is taken under natural light, the illuminance in the field of view of the image may vary depending on the position of the sun, the state of clouds, or a difference in weather such as rainy weather. When it changes, the brightness of the screen also changes.
Therefore, in this case, in the imaging field of view, for example, a predetermined background portion whose brightness is not affected by replacement or movement of objects, such as a roof or a grove of a building that does not reflect specularly as viewed from a video camera. And set a window for brightness measurement, that is, a window for brightness normalization.

That is, when the illuminance by natural light increases by k times, the portion corresponding to that portion also increases by k times the brightness of the screen, and the portion is enclosed as a brightness normalizing window. Then, the screen in which the brightness of the image of the brightness normalization window is increased by m times is obtained by multiplying the pixel signal of the observation window by (1 / m) and comparing it with the observation windows of other screens. Even if the brightness changes, accurate abnormality detection can always be obtained.

At this time, the position, size and shape of the brightness normalizing window can be set on the monitor screen of the computer by using a pointing device according to the monitoring target. Next, if the subject is irradiated with a strong light for a short time, or the sunlight is specularly reflected on a part of the subject and enters the video camera, the input light of the video camera is temporarily There is a possibility that the output becomes excessively large, the output of the corresponding portion of the screen is saturated, and a correct image cannot be obtained.

Therefore, in this embodiment, when the image signal of the observation window continuously captured at a fixed time interval is saturated due to the above-described cause, another image captured before time T is used. To compare. When the light is specularly reflected, it only saturates a part of the observation window, so it is possible to cope with the output saturation of the video camera by making a comparison with the image signal of the adjacent observation window. .

At this time, the time width of the gate signal for reading an image from the image pickup device of the video camera is changed in accordance with the change in the amount of light from the subject so that the exposure time of the image pickup device can be increased or decreased. Good.

Next, the image monitoring apparatus according to this embodiment will be described with reference to the block diagram of FIG. The video camera 601 is installed such that a monitoring target area such as a roadside zone of the road described with reference to FIG. 1 becomes an imaging visual field, and an image signal captured by the video camera 601 is input to a moving object removal filter 602, Here, a moving object image that is a hindrance to original subject imaging is removed. The details of the moving object removing filter 602 will be described later.

The image signal output from the moving object elimination filter 602 is then input to the computer 624 sequentially for each T time in frame units, using the video capture board 603 as an inlet. Therefore, the subsequent processing is
It is executed by software using a program stored in the computer 624, and firstly, the image signal switch 625
Thus, the first image signal is taken into the reference image memory 604 and stored as a reference image.

Here, as described with reference to FIG. 1, when the reference image is in a state where no object including a car exists in the roadside part of the road, the video camera 601 is used.
Therefore, the processing up to this point is a preparation processing, and thereafter, the processing shifts to a monitoring operation. When the monitoring operation is started in this manner, the time T
The image input every time is alternately input to the image memory A605 and the image memory B606. Therefore, the images before and after the lapse of the time T are stored in the image memory A605 and the image memory B in a different order each time. Will be done.

First, from the reference image read from the reference image memory 604, pixel information of the brightness normalization window is cut out by the brightness normalization window cutout processing circuit 607, and this is normalized by the normalization coefficient calculators 613 and 614. Supplied as a divisor. The image read out from the image memory A 605 and the image memory B 606 is used to extract the pixel information of the brightness normalization window of each screen by the brightness normalization window cutout processing circuits 609 and 611, and the normalization coefficient calculator 613 and 614 are supplied as dividends.

Then, these normalization coefficient calculators 613,
The output of 614 is supplied to multipliers 615 and 616, respectively, and becomes a multiplier of multipliers 615 and 616 as a brightness normalization coefficient. On the other hand, the observation window cutout processing circuits 610 and 612 extract pixel information of a plurality of observation windows set in advance, and obtain the multiplicands of the multipliers 615 and 616. As a result, the outputs of the multipliers 615 and 616 become , The pixel information of the observation window whose brightness is normalized.

The outputs of the multipliers 615 and 616 are supplied to the image signal switch 626, and the pixel information of the observation window corresponding to the newer screen is supplied to the difference calculator 617. Then, the output of the difference calculator 617 is supplied as a difference for each of the plurality of observation windows to the comparison and judgment processing circuit 619 for each observation window, where the presence or absence of a difference between each of the observation windows on the latest screen and the reference screen is determined. Is determined.

The outputs of the multipliers 615 and 616 are also supplied to a difference calculator 618, from which the difference for each of the plurality of observation windows is supplied to a comparison / judgment processing circuit 619 for each observation window. It is determined whether there is a difference between the contents of the observation windows before and after. Then, in this observation window-by-observation comparison processing circuit 619, for each of the observation windows of the screen taken in at each time T, "is there any difference from the reference screen?"
Is determined as to whether there is a difference from the image before time T, and the result is supplied to the counter 620 each time.

In this counter 620, “n consecutive numbers
The contents are the same for the specific observation windows (for example, three),
And the information is different from the reference screen "is taken out and supplied to the alarm generation processing circuit 621 for the observation window. Therefore,
The alarm generation processing circuit 621 for the observation window performs a visible alarm generation process for the indicated observation window, sends information to the superimposed circuit 622 of the captured image and the alarm, and issues the current captured image and the visible alarm. The image of the observation window is superimposed and sent to the display 623 to be displayed. At this time, as described above, a sound alarm may be used together.

Here, when a personal computer is used as the computer 624, the display 623 is used.
Is the same personal computer display.
Accordingly, a flowchart of the processing by the computer 624 is as shown in FIG. The flow of the abnormality detection processing shown in FIG. 6 will be described below. This processing starts with a start (701). Then, the reference screen is first stored in the reference screen memory.
(702).

At this time, the images to be taken every time T are taken into the image memory A and the image memory B alternately. The memory order determination processing (703) is executed by the observation window comparison judgment processing 619 in FIG. And the image signal switch 6
25. The selection processing (704) of the image memory A or B at this time is executed by the image signal switch 625 in FIG.

The observation window clipping process (705) from the image fetched into the image memory is executed by the observation window clipping processing circuits 610 and 612 from the image in FIG. The comparison processing (706) between the observation window image of the reference screen and the observation window image at a certain point in time Ti is performed by the difference calculator 617 in FIG. 5, and the difference is sent to the observation window comparison / judgment processing circuit 619. Image difference determination processing (707) is executed.

On the other hand, the comparison processing (709) between the observation window image at the time point Ti and the observation window image at the time point Ti-1 is performed by the difference calculator 618 in FIG.
9, where a difference determination process between the two images is performed (710). Then, as a result of comparing the image at the time point Ti with the observation window image of the reference image and the image at the time point Ti-1, the observation window of the reference screen differs from the observation window of the screen at the time point Ti, and the screen at the time point Ti and the screen image of the time point Ti- If the observation windows on the one-time screen are not different (that is, the same without changing), 1 is added to the count value for the observation window by the logical product of two conditions (708).

On the other hand, the image at the time of Ti, the reference image and T
As a result of comparing the observation window image with the image at the time point i-1, the observation window of the reference screen and the observation window of the screen at the time point of Ti do not differ, or Ti
If the observation window of the screen at the time point is different from the observation window of the screen at the time point of Ti-1, the count value for the observation window is reset.
(711). Then, as a result of adding 1 to the count value of the observation window by the logical product of two conditions in the addition process (708), if the count number becomes n (n = 3 in the above description) for any of the observation windows, , ID of observation window with count n or more
No. (identification number) is output (712).

By the way, in one embodiment of the present invention, a well-known normalized correlation process is adopted in the comparison processes (706) and (709). At this time, even if the size of the observation window changes, A normalized correlation process corresponding to each size is performed. Here, a detailed description is given to, for example, the following publication.
This technique allows comparison by absorbing changes in the density of the observation window image, such as fluctuations in brightness due to natural light.

[0072] Fuji Techno System, published by Jan. 1994, edited by Masatoshi Ejiri, "Image Processing Industry Application Review", p.51. However, in this case, the observation window image becomes a so-called solid image with little change in brightness and color tone. In this case, normalization correlation processing is not performed, and processing is switched to processing based on comparison of average values, for the following reason.

That is, when the average value of the observation window image is a small variation from the average value of each pixel, such as a solid image, the divisor and dividend (both image data) during the processing calculation become smaller. This is because the correlation coefficient (coincidence) in the normalized correlation processing is likely to be out of order. Here, specifically, the variation of the average value of each pixel is, for example, 5
When it is less than%, the comparison is performed only by the average value, and the determination may be made based on the magnitude of the difference between the average values.

Next, details of the moving object removing filter 602 in this embodiment will be described with reference to FIG. In the case of the present invention, as described in the above embodiment, it is necessary to capture an image at every time T. At this time, if a moving object such as a person or a car crosses the field of view of the video camera 601, Since a shielding object appears in the captured image at each time T, an error occurs in the above-described processing of comparing observation windows. Therefore, in this embodiment, the moving object removal filter 602 is inserted so that the reflected image of the moving object can be ignored.

In FIG. 7, an image signal 51 supplied from a video camera 601 is output as an image signal 56 via a multiplication element 52 and an addition element 54 for a coefficient α, and is taken into a computer 624 from a video capture board 603. At this time, it is also supplied to the frame delay element 55 in parallel. Then, the frame delay element 55
Output from the adder 5 through the coefficient (1-α) multiplying element 53.
4 and is added to the image signal 51.

Therefore, the image signal 56 obtained at the current output is a signal obtained by multiplying the pixel signal one frame before delayed by the frame delay element 55 by a factor (1−α),
This is the result of the addition of the current pixel signal 51 multiplied by α with the addition element 54. As a result, the current video camera 6
The output signal of 01 is output only α times as it is. Therefore, if the subject moves in the next frame and the position on the screen changes, the image is simply multiplied by α in the next frame.

For example, assuming that the coefficient α is 1/255, even if an image of a signal corresponding to all amplitudes is input, only about one tone of the 256-level image of the current image is presently used. Is not reflected on the image signal 56 obtained at the output.

Therefore, an image of a moving object appearing in only a few frames at most from one frame is not reflected in the image signal 56, so that moving object elimination can be obtained.

At this time, depending on the size and speed of the moving object on the screen, if an automobile or a pedestrian is a moving object, by setting the α value to the above-mentioned level, the above-described abnormality detection can be performed. Is hardly affected, and therefore, according to the embodiment of FIG. 7, a sufficient function as a moving object removing filter can be obtained.

In the above embodiment, as an example,
Although the present invention has been described for a case where the present invention is applied to a system for monitoring a parking situation, the present invention can also be applied to detection of "suspicious objects left" in the sense that extraneous things have appeared on the screen, It can be applied to monitoring such as "the disappearance of facilities and exhibits" to detect that the original thing disappeared from the screen, and can greatly contribute to crime prevention.

Next, another embodiment of the present invention will be described. In the above embodiments, the use of images from a plurality of observation windows is for specifying a monitoring target such as a car. However, in the embodiment described below, monitoring using images from a plurality of observation windows is performed. The size, shape, and direction of movement of the object can be monitored. In this case, in short,
This is to compare images of different observation windows at the same time, not at different times.

FIG. 13 is a block diagram of another embodiment of the present invention. In the figure, reference numeral 632 denotes an observation window state combination logic calculation process. The logic calculation processing circuit 630 is replaced with an observation window state combination logic operation processing circuit 632 having a different function, and the other configuration is the same as that of the embodiment described with reference to FIG. Therefore, the operation is the same as that of the embodiment of FIG. 9 except that the function obtained in this embodiment is added.

The observation window state combination logic calculation processing circuit 632 in the embodiment shown in FIG.
n, sequentially updating and storing only the latest change state of each image, performing a logical operation on the image state for a predetermined combination among the plurality of stored observation windows, and outputting a calculation result. Work.

The following describes the combination of observation windows at this time and the monitoring mode obtained based on the calculation results based on the combination. Here, for each observation window image, “0” indicates that there is no change, and “1” indicates that the change appears.
FIG. 10A shows a case where a plurality of observation windows to be combined are arranged in the vertical direction in the imaging screen of the video camera 601. In this case, N observation windows A to N in the vertical direction are used. Logical AND (AND) is performed on the image state, and (A, B, C, D,
... N) = 1 at this time indicates that the monitoring object is simultaneously displayed in all of the five observation windows A to N on the monitoring screen.

Therefore, in this case, it means that an object having a height corresponding to at least the distance between the uppermost observation window A and the lowest observation window N has appeared on the monitoring screen. Thus, by setting the distance of each of the observation windows A to N to a predetermined value, the height of the monitoring target in the imaging visual field can be determined based on how many observation windows have simultaneously changed. When the logical product is established for the observation window of, it can be estimated that a person having a certain height or more, that is, an adult has invaded, and it can be determined whether the intruder is a minor or not.

At this time, instead of providing only one observation window arranged in the vertical direction, as shown in FIG. 10B, a plurality of (n) columns are arranged in the horizontal direction and the observation windows are arranged in different columns. By taking the logical product of the combination of observation windows, the size of the object to be monitored in the horizontal direction on the monitoring screen can also be identified, and if the combination in the vertical direction is also used, the size of both the vertical and horizontal directions Can be identified, and in this case, the shape of the monitoring target object can be determined to some extent.

For example, when the object to be monitored is an automobile and only a large vehicle is to be monitored, it is sufficient to perform a logical AND operation on a plurality of observation windows at positions separated according to the length to be detected. If it is determined how long the detected state continues for a long time, monitoring of parking for a long time can be obtained.

However, in the case of this embodiment, when images of different objects appear simultaneously in each of the observation windows, there is a possibility that the image may be erroneously recognized as a large object.
At this time, by further increasing the number of observation windows or shortening the image update cycle, that is, the time T described above,
The probability of false recognition can be reduced.

Next, still another embodiment of the present invention will be described. Here, the embodiment described below uses a plurality of, for example, two video cameras, and moves these video cameras toward the field of view. By disposing them on the left and right, parallax appearing in the same subject can be used to three-dimensionally identify the monitoring target.

That is, in this embodiment, FIG.
As shown in (2), two video cameras 601-1 and 601-2 are used, and they are installed left and right apart such that their imaging optical axes intersect at a certain distance L. Then, the image of the object A at the position where the imaging optical axes of both cameras coincide with each other at the distance L is shown in FIG.
As shown in (b) and (c), any camera is located at the center of the imaging screen, but the image of the object B at a position shorter than the distance L is the imaging screen of the left camera 601-1. As shown in the figure, it shifts to the left, and shifts to the right on the imaging screen of the camera 601-2 on the right. On the other hand, in the case of a position farther than the distance L, the shift direction is reversed left and right.

Therefore, in this case, the position of the object to be monitored in the distance L direction (depth direction) can be determined based on the position of the observation window on the imaging screen of both cameras. Therefore, in this case, observation windows are respectively set on the screens of both cameras 601-2 and 601-2 as shown in FIGS. 11B and 11C, and a logical operation is performed on the image state of these observation windows. By taking (logical product), the objects A and B having different positions in the depth direction can be individually identified and independently detected.

Here, FIGS. 11 (b) and 11 (c) show a case where only two observation windows are set for each imaging screen, but a plurality of observation windows are arranged on each imaging screen side by side. By setting windows and arbitrarily setting a combination of observation windows on both imaging screens, it is possible to arbitrarily select the position of the object to be detected in both the depth direction and the left and right direction.

In this case as well, if images of different objects appear simultaneously in each observation window, there is a risk of erroneous recognition. The probability of erroneous recognition can be reduced by further increasing the number or shortening the image update cycle, that is, the time T described above.

By the way, in the embodiment described above,
As is clear from FIG. 5 and FIG. 9, the system is configured to display a monitoring image or the like on the display 623 and to perform monitoring. However, according to the embodiment of the present invention, necessary information such as the monitoring image is stored in another place. It may be transmitted so that monitoring can be performed at an arbitrary position. In the case of this embodiment, the computer 624 is provided with a communication function so that the computer 624 can be connected to a terminal such as a mobile phone with a screen via a public telephone line or the Internet.

Regarding the line connection at this time, there are a case where the monitoring system calls the terminal and a case where the terminal calls the monitoring system. First, when a terminal is called from the monitoring system side, when a monitoring alarm occurs, a necessary terminal is called and an alarm signal or the like is transmitted to the terminal. When a call is made from the terminal side, it is uncertain whether or not an alarm signal has been generated at the time of the call, so that a monitoring screen is transmitted to the terminal for the time being.

Next, at this time, in the embodiment of the present invention, the transmission mode of the alarm signal to the terminal is shown in FIG.
This will be described below. Here, first, FIG. 12A shows a case where the image monitoring system according to the present invention is applied as a system for monitoring the approaching of a suspicious person to the fence F for enclosing. In this embodiment, as shown in FIG. A plurality of observation windows are set in front of the passage leading to the fence F, as shown in column A, and when an abnormality is detected in any of the observation windows in column A, an alarm signal is sent to the terminal. In this case, when a return request is received from the terminal, the entire scene image, that is, the image of the monitoring target area is transmitted to the terminal.

Next, FIG. 12B shows a system for monitoring a suspicious person approaching the fence F.
Here, in one embodiment, in addition to the observation window in column A, an observation window in column B is set near the fence F. In this case, first, an abnormality is detected in any of the observation windows in column A. Then, an alarm signal is transmitted to the terminal, and if an abnormality is detected in any of the observation windows in column B, an entire scene image is automatically transmitted to the terminal. is there.

Therefore, according to the embodiment described here, monitoring can be performed from an arbitrary location remote from the monitoring target area, and when necessary, the terminal can be called, so that monitoring is easy. At this time, as described above, it is needless to say that the alarm notification by voice or beep sound may be obtained. Here, when the terminal is a mobile phone, it can be said that the configuration is rather natural.

Further, the present invention is not limited to the above embodiment, but can be implemented in the following modes. Embodiment A When comparing two screens, a difference between the brightness of the corresponding pixels of the corresponding observation window (in a color image, the brightness of any one color) is obtained, and this difference is summed up for each observation window. Then, by comparing the total value between the corresponding observation windows, the difference in the subject shape between the corresponding observation window images,
Alternatively, an image monitoring method and apparatus that captures a change in the presence or absence of an object, a positional shift, and the like, and uses the information to determine the similarity of the object reflected in each observation window.

Embodiment B When calculating the difference in brightness between the corresponding pixels of the corresponding observation windows, the difference between the signals of the three colors is obtained, and the total value of the differences of the signals of the three colors is obtained for each observation window. An image monitoring method and apparatus which individually compares the three colors and determines whether or not there is a change in each of the three colors for each corresponding observation window.

Embodiment C An image monitoring system characterized in that it is possible to detect whether the current situation is different from the standard state by targeting the screen in the standard state and the current screen as the two screens to be compared. Methods and equipment.

Embodiment D In addition to the reason that the general landscape captured as an image has a different brightness level for each captured screen, strong light shines on a part of the subject captured on the screen. When the level of the image signal of the observation window corresponding to this is extremely high, only the image signal of the observation window at that point in time or only the image signals of the observation windows at a plurality of successive points in time is excluded from the series of data, and the excluded data. An image monitoring method and apparatus, characterized in that the method is operated by using data immediately before or after the time.

Embodiment E When a change occurs in a plurality of predetermined observation windows among a plurality of preset observation windows, a change state of the plurality of observation windows is logically calculated, and only the result is obtained. An image monitoring method and apparatus, wherein the amount of transmission to a place monitored through a line at a remote place is reduced by outputting a signal.

Embodiment F An image monitoring system characterized in that the amount of transmission by the line is reduced by transmitting the image information through the line only when the result of the logical calculation of the change condition satisfies a preset condition. Methods and equipment. Embodiment G An image monitoring method and apparatus wherein the positions and sizes of a plurality of observation windows can be set from a location distant from the image information extraction device through a line.

[0105]

According to the present invention, the occurrence of an abnormality in a monitored area can be automatically detected from an image signal picked up by a video camera. It is possible to always reliably detect the disappearance of what should originally be present, such as an exhibit, without manual intervention.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram illustrating an example of a monitoring target area according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating an example of a setting state of an observation window according to an embodiment of the present invention.

FIG. 3 is an explanatory diagram showing a relationship between an object to be monitored and an observation window in one embodiment of the present invention.

FIG. 4 is an explanatory diagram illustrating an example of an alarm display according to an embodiment of the present invention.

FIG. 5 is a block diagram showing an embodiment of the image monitoring apparatus according to the present invention.

FIG. 6 is a flowchart illustrating an abnormality detection process according to an embodiment of the image monitoring device according to the present invention.

FIG. 7 is a block diagram illustrating an example of a moving object removing filter according to an embodiment of the present invention.

FIG. 8 is a schematic diagram illustrating a relationship between the number of observation windows and a monitoring target according to an embodiment of the present invention.

FIG. 9 is a block diagram showing another embodiment of the image monitoring apparatus according to the present invention.

FIG. 10 is an explanatory diagram of an example of setting an observation window according to another embodiment of the present invention.

FIG. 11 is an explanatory diagram of a camera arrangement example according to another embodiment of the present invention.

FIG. 12 is an explanatory diagram showing an example of a monitoring screen according to still another embodiment of the present invention.

FIG. 13 is a block diagram showing still another embodiment of the image monitoring apparatus according to the present invention.

[Explanation of symbols]

Reference Signs List 51 Pixel output of image sensor 52 Multiplier of coefficient α 53 Multiplier of coefficient (1-α) 54 Adder 55 Frame delay element 56 Pixel output 601 Video camera 602 Moving object removal filter 603 Video capture board 604 Standard image memory 605 Image memory A 606 Image memory B 607 Brightness normalized window cutout processing circuit from reference image 608 Observation window cutout processing circuit from reference image 609 Brightness normalized window cutout processing circuit 610 From image Observation window cutout processing circuit 611 Image normalization window extraction processing circuit 612 from image Observation window extraction processing circuit from image 613, 614 Normalization coefficient calculator (divider) 615, 616 Multiplier 617, 618 Difference calculator 619 Comparison judgment processing for each observation window Circuit 620 Counter 62 Alarm generation processing circuit for observation window 622 Superimposition circuit of captured image and alarm 623 Display 624 Computer 625 Image signal switch 626 Image signal switch 630 Observation window state logic calculation processing circuit 631 Combination alarm generation processing circuit 701 Start 702 Standard screen Memory 703 Determination of memory order 704 Selection of image memory A or B 705 Cutout of observation window (several times for each observation window) 706 Comparison between observation window image of standard screen and observation window image at Ti time 707 Difference? 708 The count of the observation window is 1 by the logical product of two conditions.
709 Comparison between observation window image at Ti and observation window image at Ti-1 710 Difference? 711 Reset the count for the observation window 712 Output the ID number of the observation window with count n or more

Continued on the front page F-term (reference) 5B057 AA16 AA19 BA11 BA19 BA24 CE09 CE11 CH01 CH11 DA06 DC32 5C054 AA01 AA05 CA04 CC02 CE15 CH01 EA01 EE00 FA09 FB03 FC01 FC13 FC16 FE09 HA18 HA26 5C084 AA02 AA12 BB33 DD01 BA02 BA04 CA04 CA24 EA12 EA35 GA08 GA40 GA41 HA03

Claims (19)

[Claims] At least two observation windows are set in a predetermined monitoring portion on an image plane of a video camera, and video information is collected at least at different times from the observation windows, and obtained from the at least two observation windows. An image monitoring method, wherein a change in the predetermined monitoring portion is detected by calculating a change in video information at different times. 2. The image monitoring system according to claim 1, wherein the size of the observation window is set to be about half or less of the size of the monitoring target on the image plane. Method. 3. The image monitoring method according to claim 2, wherein the observation windows are arranged according to the shape of the monitoring target. 4. The image monitoring system according to claim 1, wherein any one of the shape, the number, and the position of the at least two observation windows can be arbitrarily set on a display image surface of the monitoring target area. Method. 5. The invention according to claim 1, wherein a window for normalizing brightness is set on the image plane, and a signal for normalizing brightness is obtained from an image cut out by the window for normalizing brightness. , And the level of the image signal is normalized by this signal. 6. The image processing apparatus according to claim 5, wherein the brightness normalization signal includes R,
An image monitoring method, wherein each of the color signal components is independently generated from the measurement result of each of the G and B color signal components. 7. The invention according to claim 5, wherein at least one of the shape, the number, and the position of the brightness normalizing window can be arbitrarily set on a display image surface of the monitoring target area. Image monitoring method. 8. The image monitoring method according to claim 1, wherein the image of the monitoring target area is captured via a moving object removal filter. 9. The invention according to claim 8, wherein the moving object elimination filter multiplies each pixel signal of the latest image by a coefficient α, and multiplies this by a coefficient (1−α) for each pixel of the immediately preceding image. An image monitoring method, comprising: means for adding a product of the multiplication to output as a current image, wherein the coefficient α is set so that 0 <α≪1. 10. An image monitoring method of capturing at least two images of a monitoring target area before and after in time, and detecting a movement of an object in the monitoring target area by comparing the at least two images. Means for setting at least two observation windows in the image plane; and comparing the at least two images with the observation window between the corresponding observation windows in each of the image planes. An image monitoring apparatus, comprising means for performing a comparison only by comparing images. 11. The monitoring object according to claim 10, wherein the at least two observation windows each have a size of the observation window smaller than the object on the image plane, and the monitoring target object on the image plane. An image monitoring apparatus characterized in that the size is set to approximately 1/2 or less of the size of. 12. The image monitoring apparatus according to claim 11, wherein the observation windows are arranged according to a shape of the monitoring target. 13. The image according to claim 11, wherein at least one of the shape, the number, and the position of the at least two observation windows can be arbitrarily set on a display image surface of the monitoring target area. Monitoring device. 14. The invention according to claim 11, wherein: a means for setting a window for normalizing brightness on the image plane; and an image for normalizing brightness from an image cut out by the window for normalizing brightness. And a means for normalizing the level of the image signal based on the signal. 15. The apparatus according to claim 14, wherein said means for generating a signal for normalizing brightness includes a color signal component based on a measurement result of each color signal component of R, G, and B constituting an image signal. An image monitoring apparatus configured to generate the brightness normalization signal independently for each image. 16. The invention according to claim 14, wherein at least one of the shape, the number, and the position of the brightness normalizing window can be arbitrarily set on a display image surface of the monitoring target area. Image monitoring device. 17. The image monitoring apparatus according to claim 11, further comprising a moving object removing filter, wherein an image of the monitoring target area is captured via the moving object removing filter. 18. The invention according to claim 17, wherein the moving object elimination filter multiplies each pixel signal of the latest image by a coefficient α, and assigns a coefficient (1−α) to each pixel of the immediately preceding image. An image monitoring apparatus, comprising: means for adding the multiplied result to a current image and outputting the current image, wherein the coefficient α is set so that 0 <α≪1. 19. At least two observation windows are set on a screen taken by an individual video camera at the same time, and a plurality of observation windows combined in advance according to the observation purpose from among these at least two observation windows are set. A window group is set, and a logical operation is performed on information representing the presence or absence of a change in the image of the observation window in the plurality of observation window groups, thereby detecting whether or not a predetermined specific event has occurred. Image monitoring method.