JP2005277726A - Image monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image monitoring device capable of obtaining the magnified image of a face by which characteristics of a person can be further easily obtained. <P>SOLUTION: Detection and tracing of a person are executed. It is managed whether a zoom shot of a face for each person is photographed or not. Zoom photography of a face of a person is executed whose zoom shot is not obtained. A representative face image is obtained. When it is predicted that a face image with a larger evaluation value can be obtained even regarding a person to whom zoom photography is executed once, the zoom photography is executed to the person again. Then, the representative face image is updated. It is determined whether a representative face image with a larger evaluation value can be obtained or not from the moving direction of a person. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image monitoring apparatus that includes a plurality of monitoring cameras and monitors a person who enters a monitoring area while detecting and tracking the person.

This type of image monitoring apparatus detects a person in the monitoring area and the face of the person by capturing an image from a monitoring camera installed to capture the monitoring area and processing the captured image with a computer. When a person and a face are detected, an alarm is output on the display device to notify the monitor, thereby preventing an image of the monitor from being overlooked.
In addition, the person and the face image are stored in a storage device such as a hard disk for the purpose of referring to the information about the person appearing in the monitoring area later.

For example, Patent Document 1 and Patent Document 2 disclose conventional techniques related to such an image monitoring apparatus.
The technology of Patent Document 1 is to monitor one monitoring area with two monitoring cameras. One monitoring camera is a fixed-field camera that captures the entire monitoring area, and the other monitoring camera is a zoom shot of a person. This is a zoom camera for obtaining (enlarged image). When a person who has entered the surveillance area is detected from an image from the fixed-field camera, the zoom camera performs zoom photography (enlarged photography) of the person.

  The purpose of Patent Document 1 is an invention for solving the problem that when a wide monitoring area is photographed by a single camera, the person is small on the screen and the characteristics of the person are difficult to grasp. By zooming in on a person, an image that makes it easier to capture the characteristics of the person can be obtained.

  The technique of Patent Document 2 is to monitor one monitoring area using two cameras, as in Patent Document 1, and the zoom target of the zoom camera is not the whole person but the face of the person.

FIG. 11 shows a conventional processing method for determining a zoom target. In this processing method, when there is a person who is already subject to zoom shooting, zoom shooting is performed until the target person leaves the monitoring area or a zoom shot of the target person is obtained. When there is no person to be zoomed and there is a person who has not yet performed zoom shooting, or when a new person is detected by a fixed-field camera, zoom shooting is performed on that person.
JP-A-6-217316 JP 2003-284053 A

  However, the conventional technique has a problem in that zoom photography is not performed even when a zoom shot of a face is once taken and then a better face image is obtained for the person. Here, a better face image is a good image in that it captures the characteristics of a person. For example, an image with a face facing front is better than a face facing sideways.

  This will be described in detail with reference to FIG. FIG. 12 shows a series of images from the fixed-field camera in the monitoring area, and it is assumed that time elapses in the order of FIGS. 12 (a), (b),..., (F). The person A appears in the monitoring area in FIG. 12A and moves from the right to the left with respect to the drawing. At this time, the face of the person A is sideways with respect to the camera. The person A faces in front of the camera in FIG. 12E and moves toward the camera in FIG. The person B appears in the monitoring area in FIG. 12C, moves toward the camera, and disappears in FIG. 12E.

  In the prior art, in FIG. 12A, a person A is detected from a fixed-field camera, and a zoom camera is used to zoom in on the face of the person A. In this case, the zoom shot for the person A is a face image only in landscape orientation. In FIG. 12C, the person B is detected, and the zoom camera is used to zoom in on the face of the person B.

  In FIGS. 12E to 12F, since the person A has already performed zoom photography of the face, zoom photography of the face of the person A is not performed. As a result, the finally obtained zoom shot of the face is only the side face as shown in FIG. 13, although the person A is facing the front in FIGS. 12 (e) and 12 (f).

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image monitoring device that can obtain an enlarged image of a face that can more easily capture the characteristics of a person.

  An image monitoring apparatus of the present invention includes a first monitoring camera that captures and inputs an image in a monitoring area, and a second monitoring camera that can change a shooting magnification for capturing and inputting an image in the monitoring area. A person detection and tracking means for detecting and tracking a person in the monitoring area based on an image obtained from the first monitoring camera; and a person's face in the monitoring area based on an image obtained from the second monitoring camera. Face detection means for detecting an image; face image evaluation means for evaluating the face image detected by the face detection means to obtain an evaluation value; and when the person is detected by the person detection tracking means, the face image evaluation First control means for controlling the second surveillance camera so as to magnify the face image of the person until the evaluation value obtained from the means becomes larger than a first threshold value set in advance; 2 surveillance turtles Whether or not there is a person in the monitoring area whose evaluation value obtained from the face image evaluation means is smaller than a second threshold value set in advance based on the fact that the enlargement of the face image of the person by And a person who is smaller than a second threshold value set in advance as the evaluation value obtained from the face image evaluation means as a result of the determination by the first determination means. If present, the second monitoring camera is again controlled to magnify the face image of the person until the evaluation value obtained from the face image evaluation means becomes larger than the first threshold. Control means.

  ADVANTAGE OF THE INVENTION According to this invention, the image monitoring apparatus from which the enlarged image of the face which can catch a person's feature more easily can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 schematically shows a configuration of an image monitoring apparatus according to the present embodiment. This image monitoring apparatus includes a fixed monitoring camera 11 as a first monitoring camera, a zoom camera 12 as a second monitoring camera, a person detection tracking unit 13 as a person detection tracking unit, and a face detection unit 14 as a face detection unit. A person management unit 15 as a control unit and a determination unit, a face image evaluation unit 16 as a face image evaluation unit, a zoom camera control unit 17 as a control unit, a detection result display unit 18 as a display unit, and a storage unit The detection result storage unit 19 is provided, and the function of each unit will be described in detail below.

As shown in FIG. 2, the fixed monitoring camera 11 and the zoom camera 12 are installed so as to capture an image in a predetermined monitoring area E. That is, one fixed-field camera 11 captures the entire monitoring area E and is used for the purpose of detecting and tracking the person M in the monitoring area E. Another zoom camera 12 is used for the purpose of detecting a face image of a person existing in the monitoring area E and performing zoom photography. The zoom camera 12 can control the shooting direction vertically and horizontally and can also control the zoom rate (shooting magnification).
Note that the zoom camera 12 shoots with a setting of a direction and a zoom ratio of the lens so that substantially the same image as the fixed-field camera 11 is obtained when the person M is not detected in the monitoring area E. .

  The person detection tracking unit 13 and the face detection unit 14 sequentially capture and process image signals from the fixed-field camera 11 and the zoom camera 12 as image information at regular time intervals, respectively. Here, images that are sequentially captured are also referred to as frames.

  The person detection / tracking unit 13 detects and tracks a person region (person) based on an image obtained from the fixed-field camera 11. A unique ID (identification information) is assigned to the detected person area. The detected person area is then tracked until the person area no longer exists from within the monitoring area. FIG. 3 shows a flow of processing in the person detection tracking unit 13.

  First, an outline of the processing flow in the person detection tracking unit 13 will be described. The image signal of the fixed field camera 11 is taken in as an image, and a change area is detected from the obtained image. For example, the change area is detected by using a background image as described in the literature 1 (Nakai, “A moving object detection method using posterior probabilities”, Information Processing Research Reports, 94-CV90, pp. 1-8, 1994). It detects from the difference. This has an image when no person is present in the monitoring area E as a background image, and detects an area that has changed from the difference between the background image and the current frame as a change area. An area determined to be human is detected as a person area from the size and aspect ratio of the detected change area.

  The person area of the current frame is associated with the person area being tracked. The association is performed, for example, by associating objects that are close in size and position with the person area in the previous frame. Of the person areas detected in the current frame, those that have not been associated with the person area being tracked are assigned new IDs and added as persons to be tracked. If the person being tracked finds no more than a certain frame Tf, the tracking is stopped because the person has left the monitoring area E.

  Hereinafter, the flow of processing in the person detection tracking unit 13 will be specifically described with reference to FIG. First, the image signal of the fixed field camera 11 is captured as an image (step S1), and a change area is detected from the obtained image (step S2). Next, an area determined to be human is detected as a person area from the size and aspect ratio of the detected change area (step S3).

  Next, the person area detected from the current frame is associated with the person area detected in the past (the person area being tracked) (step S4). Next, it is checked whether or not there is a person ID for which a person area corresponding to a certain frame Tf or more is not found (step S5). If not, the ID is discarded as if the person has left the monitoring area E. Tracking is stopped (step S6).

As a result of the check in step S5, if there is a person ID for which a person area corresponding to a certain frame Tf or more cannot be found, it is checked whether there is any person area in the current frame that cannot be associated (step S7). If there is, the person area is generated as a new person area, and a new ID is added (step S8).
As a result of the check in step S7, if there is no person region in the current frame that cannot be associated, or when the process in step S8 is completed, the process returns to step S1 and the same process as described above is repeated.

FIG. 4 shows a processing example of the person detection tracking unit 13. A rectangle indicated by a broken line in the figure is a person area detected in each frame. In FIG. 4, it is assumed that time has passed in the order of (a), (b),..., (H). The person area ME1 appears in the monitoring area E in FIG. 4A and moves from the right to the left with respect to the drawing. At this time, the face of the person area ME1 is lateral to the camera. The person area ME1 faces in front of the camera in FIG. 4 (e) and moves toward the camera in FIG. 4 (f). The person area ME1 moves from the left to the right in FIG. At this time, the face of the person area ME1 is lateral to the camera. The person area ME1 moves in a direction away from the camera with the face facing backward with respect to the camera in FIG. 4 (h). The person area ME2 appears in the monitoring area E in FIG. 4 (c) and disappears in FIG. 4 (e).
In the example of FIG. 4, “ID = 1” is assigned to the person area ME1, and “ID = 2” is assigned to the person area ME2.

  The face detection unit 14 continuously captures a plurality of images from the zoom camera 12 at regular time intervals, and detects a face area for each of the captured plurality of images. For example, reference 2 (Mita, Kaneko, Hori “Proposal of Spatial Difference Probability Template Suitable for Matching Images Containing Minute Differences”, Proceedings of the 9th Meeting of Image Sensing Symposium, SSII03, 2003. ) Method can be used.

  The person management unit 15 manages information about the person detected by the person detection tracking unit 13. Specifically, a person area, a face area, a representative face image, and information on whether or not zoom shooting has been performed are stored for each person ID detected by the person detection tracking unit 13. FIG. 5 shows a flow of processing in the person management unit 15.

  First, an outline of the processing flow in the person management unit 15 will be described. For each person's ID, a person region is obtained from the person detection tracking unit 13 and the information is updated. Next, a face area (an image including at least a face, hereinafter referred to as a face image) and a person ID are obtained from the face detection unit 14. A plurality of face images are obtained for one person ID. The obtained set of face images is sent to the face image evaluation unit 16, and the one with the highest evaluation value is set as the representative face image.

  For a person who has not been zoomed, the zoom camera control unit 17 is instructed to make a subject for zoom shooting, and the evaluation value Ee of the representative face image of the person to be zoomed is larger than the first threshold Te1. Continue zooming until

  If a person in the monitoring area E has taken a zoom shot for all persons and there is a person whose representative face image evaluation value Ee is lower than the second threshold value Te2, the person is zoomed in again for that person. Take a photo of However, it is assumed that the evaluation value Eb indicating the possibility of obtaining a face image having a larger evaluation value is larger than the third threshold value Tb.

  Hereinafter, the flow of processing in the person management unit 15 will be specifically described with reference to FIG. First, for each person's ID, a person region is obtained from the person detection tracking unit 13 and the information is updated (step S11). Next, a face area (face image) and a person ID are obtained from the face detection unit 14, and the information is updated (step S12). A plurality of face images are obtained for one person ID. The obtained set of face images is sent to the face image evaluation unit 16, and the one with the highest evaluation value is set as the representative face image.

Next, it is checked whether or not there is a person (IDa) who is performing zoom shooting (step S13). If there is, the evaluation value Ee obtained from the face image evaluation unit 16 is greater than the first threshold Te1. It is checked whether or not there is a person (IDa) to be enlarged (step S14). If there is no person (IDa), the process returns to step S11 and the same operation as described above is repeated.
If there is a person (IDa) whose evaluation value Ee is larger than the first threshold value Te1 as a result of the check in step S14, zoom shooting for the person (IDa) is terminated, and the process returns to step S11 and performs the same operation as above. repeat.

  If the result of the check in step S13 is that there is no person (IDa) performing zoom shooting, it is checked whether there is a person (IDb) who has not yet taken a zoom shot (step S16). In this case, the zoom target is set to the person (IDb), and the process returns to step S11 to repeat the same operation.

  As a result of the check in step S16, if there is no person (IDb) who has not yet taken a zoom shot, a person (IDc) whose evaluation value Ee obtained from the face image evaluation unit 16 is smaller than the second threshold Te2 is obtained. It is checked whether or not it exists (step S18), and if it does not exist, the process returns to step S11 and the same operation is repeated.

If there is a person (IDc) whose evaluation value Ee is smaller than the second threshold value Te2 as a result of the check in step S18, this represents the possibility that a face image with a higher evaluation value will be obtained for that person (IDc). It is checked whether or not the evaluation value Eb is larger than the third threshold value Tb (step S19). If the evaluation value Eb is not larger than the third threshold value Tb, the process returns to step S11 and the same operation is repeated.
As a result of the check in step S19, if the evaluation value Eb is larger than the third threshold value Tb, the zoom target is set to the person (IDc) (step S20), the process returns to step S11, and the same operation is repeated.

  Here, the evaluation value Eb is calculated based on the moving direction of the person in the image obtained from the fixed-field camera 11. For example, it defines so that it may become a high evaluation value, when moving toward the direction of a camera. This will be described with reference to the example of FIG. As shown in FIG. 6A, when the person M moves toward the camera, a high evaluation value Eb is given. Also, as shown in FIG. 6B, when the person M moves sideways with respect to the camera, an evaluation value Eb lower than that in FIG. 6A is given. When the person M moves away from the camera as shown in FIG. 6C, an evaluation value Eb smaller than those in FIGS. 6A and 6B is given.

  However, when there are a plurality of persons satisfying Ee <Te2 and Eb> Tb, a time until the person M disappears from the monitoring area E is predicted, and the person M with the shortest time is set as a zoom target. The time until it disappears from the monitoring region E is calculated as, for example, the time required to calculate the moving speed from the tracking result of the person M and reach the end of the image.

  The face image evaluation unit 16 calculates an evaluation value for the given face image. In this case, the evaluation value is calculated from the direction of the face area, the area of the face area, the face area contrast, and the like as disclosed in, for example, Japanese Patent Application Laid-Open No. 2002-191044. Specifically, the evaluation value is set higher as the direction of the face region is closer to the front. Also, the evaluation value is set higher as the area of the face region is larger. As for the contrast of the face area, the higher the contrast is, the higher the evaluation value is.

  The zoom camera control unit 17 zooms toward the face of the person to be zoomed to obtain a zoom shot of the face. FIG. 7 shows the flow of processing in the zoom camera control unit 17.

  First, an outline of the flow of processing in the zoom camera control unit 17 will be described. If the person to be zoomed is not set, the zoom camera 12 is set to the neutral position. Here, the setting at the neutral position is the setting of the direction and the zoom ratio of the lens so that substantially the same image as that of the fixed-field camera 11 can be obtained.

  When a person to be zoomed is set, zooming is performed toward the face area of the person. The person to be zoomed is given as an area on the image of the fixed field camera 11 from the person management unit 15. At the time when the zoom target is set, the zoom camera 12 is set to a field of view so that substantially the same image as that of the fixed field camera 11 can be obtained, so the head of the person area given from the person management unit 15 Zoom in toward.

  If the face detection unit 14 detects a face area in the given person area, the face area is targeted, and if not, zooming is performed on the upper part of the person area. In this case, the orientation of the zoom camera 12 is controlled so that the face area is located at the center of the image obtained from the zoom camera 12. When the face area is located at the center of the image, zoom in is performed so that the face area becomes larger. The zoom-in is performed until the size of the face area exceeds a certain value. If the face area becomes too large, the zoom camera 12 is zoomed out.

  Hereinafter, the flow of processing in the zoom camera control unit 17 will be specifically described with reference to FIG. First, it is checked whether or not a person to be zoomed is set (step S31). If not set, the zoom camera 12 is returned to the neutral position (step S32), and the process returns to step S31 and the same as above. Repeat the operation.

  If the person to be zoomed is set as a result of the check in step S31, the face area of the person is acquired from the person management unit 15 (step S33), and the face area is the center of the image obtained from the zoom camera 12 It is checked whether it is located at (step S34). If the result of this check is that it is not located at the center of the image, the orientation of the zoom camera 12 is controlled so that the face area is located at the center of the image obtained from the zoom camera 12 (step S35). Return and repeat the same operation as above.

  If the result of the check in step S34 is that the face area is located at the center of the image obtained from the zoom camera 12, it is checked whether the face area is too small (step S36). If the size of the face area is too small as a result of this check, zoom-in control is performed on the zoom camera 12 (step S37). The zoom-in control is performed until the size of the face area becomes a certain value or more. Then, it returns to step S31 and repeats the same operation as described above.

  If it is determined in step S36 that the face area is not too small, it is checked whether the face area is too large (step S38). If the result of this check is that the face area is too large, zoom-out control is performed on the zoom camera 12 (step S39). The zoom-out control is performed until the size of the face area becomes a certain value or more.

  If the result of the check in step S38 is that the size of the face area is not too large, or when the processing in step S39 is completed, the process returns to step S31 and repeats the same operation as described above.

  The detection result display unit 18 adds a representative face image in the vicinity of the person area (person) being tracked in the image obtained from the fixed-field camera 11, and changes the position of the representative face image according to the movement of the person area. To display. When the representative face image is updated, the displayed representative face image is also changed.

  FIG. 8 shows a display example of the detection result display unit 18. In the figure, ME indicates a person area (person) being detected and tracked, and FE indicates a representative face image detected in the person area. 8A corresponds to the processing examples of FIGS. 4A to 4G, and FIG. 8B corresponds to the processing examples of FIGS. 4A to 4H.

The detection result accumulating unit 19 includes an image of the fixed-field camera 11 when a person is present in the monitoring area E, an entry time of each person, an existence time, an area of each person, and a representative face of each person. Save images etc. as history information.
The information stored in the detection result storage unit 19 can be displayed on the detection result display unit 18 as a list image as shown in FIG. 9, for example. In the example of FIG. 9, the face image, the image of the whole body of the person, the intrusion time, and the existence time are shown in order from the left with respect to the drawing.

As described above, according to the above-described embodiment, a zoom shot of a face that makes it easier to capture the characteristics of a person can be obtained as compared with the conventional zoom imaging control method described above.
That is, the person management unit 15 manages whether or not a zoom shot of a face has been taken for each person, and it is predicted that a face image having a larger evaluation value can be obtained even for a person who has once performed zoom photography. This is because the zoom photographing is performed again and the representative face image is updated.

  According to the embodiment, for example, the person A is detected in FIG. 12A and the face of the person A is zoomed. At this time, the representative face image of the person A is a side face image. In FIG. 12C, the person B is detected, and a zoom shot of the face of the person B is taken. 12E to 12F, a zoom shot of the face has already been taken for the person A, but a representative face image having a sufficiently large evaluation value has not been obtained, and the person A is a camera. It is determined that a face image having a larger evaluation value can be obtained, and the image is to be zoomed. As a result, a zoom shot of the front face as shown in FIG. 10 finally becomes the representative face image of the person A.

In the detection result display, a zoom shot of a face that easily captures the characteristics of a person is displayed so that the correspondence with the person can be easily understood.
In other words, the detection result display unit 18 displays a representative face image that is sequentially updated in the vicinity of the person, and always displays the face in front of the person. In the case of the example shown in FIG. 8, the person (person area) ME is facing sideways with respect to the camera (FIG. 8A) or facing back (FIG. 8B). The face image FE that most easily captures the characteristics of the person at that time is displayed in the vicinity of the person ME in an easy-to-understand manner together with the correspondence relationship of the person ME.

1 is a block diagram schematically showing a configuration of an image monitoring apparatus according to an embodiment of the present invention. The schematic diagram which shows the arrangement | positioning state of a fixed surveillance camera and a zoom camera. The flowchart which demonstrates the flow of a process in a person detection tracking part concretely. The schematic diagram for demonstrating the process example of a person detection tracking part. The flowchart which demonstrates the flow of a process in a person management part concretely. The schematic diagram for demonstrating the example of a moving direction of the person in the monitoring area | region. 5 is a flowchart for specifically explaining the flow of processing in a zoom camera control unit. The schematic diagram which shows the example of a display of the detection result in a detection result display part. The schematic diagram which shows the example of a list display of the information accumulate | stored in the detection result storage part. The schematic diagram which shows the zoom shot of the front face finally obtained. The flowchart explaining the conventional processing method which determines a zoom object. The schematic diagram which shows the example of the input image series from a fixed visual field camera. The schematic diagram which shows the zoom shot of the face in a prior art.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Fixed surveillance camera (1st surveillance camera), 12 ... Zoom camera (2nd surveillance camera), 13 ... Person detection tracking part (person detection tracking means), 14 ... Face detection part (face detection means), 15 ... person management unit (control means, determination means), 16 ... face image evaluation section (face image evaluation means), 17 ... zoom camera control section (control means), 18 ... detection result display section (display means), 19 ... detection Result accumulating unit (storage means), E ... monitoring area, M, A, B ... person, ME ... person area, FE ... face image.

Claims (5)

A first surveillance camera that captures and inputs an image within the surveillance area;
A second surveillance camera capable of changing a photographing magnification for photographing and inputting an image in the surveillance area;
Human detection tracking means for detecting and tracking a person in the monitoring area based on an image obtained from the first monitoring camera;
Face detection means for detecting a face image of a person in the monitoring area based on an image obtained from the second monitoring camera;
Face image evaluation means for evaluating the face image detected by the face detection means to obtain an evaluation value;
When a person is detected by the person detection and tracking means, the face image of the person is magnified until the evaluation value obtained from the face image evaluation means becomes larger than a first threshold value set in advance. First control means for controlling the second surveillance camera;
A person whose evaluation value obtained from the face image evaluation means is smaller than a preset second threshold value is within the monitoring area based on the end of the enlarged photographing of the person's face image by the second monitoring camera. First determination means for determining whether or not the
As a result of the determination by the first determination unit, when a person whose evaluation value obtained from the face image evaluation unit is smaller than a second threshold value set in advance is present in the monitoring area, from the face image evaluation unit Second control means for controlling the second monitoring camera again so as to magnify the face image of the person until the obtained evaluation value is greater than the first threshold;
An image monitoring apparatus comprising:   In the image obtained from the first monitoring camera, add a face image of the person magnified by the second monitoring camera in the vicinity of the person detected and tracked by the person detection and tracking means; and The image monitoring apparatus according to claim 1, further comprising display means for displaying the face image by changing the position according to the movement of the person. The face detection means continuously receives a plurality of images from the second monitoring camera, and performs face image detection for each of the plurality of images.
The image monitoring apparatus according to claim 1, wherein the face image evaluation unit evaluates each of the plurality of face images obtained from the face detection unit, and outputs the highest evaluation value as an evaluation result. As a result of the determination by the first determination unit, when a person whose evaluation value obtained from the face image evaluation unit is smaller than a second threshold value set in advance is present in the monitoring area, the person has a higher evaluation. Second determining means for determining whether or not the face image of the value is predicted to be obtained;
The second control means evaluates from the face image evaluation means if the person is predicted to obtain a face image having a larger evaluation value as a result of the determination by the second determination means. 2. The image monitoring apparatus according to claim 1, wherein the second monitoring camera is controlled again so that the face image of the person is magnified until the value becomes larger than the first threshold value.   The second determination means determines whether or not the person is predicted to obtain a face image having a larger evaluation value based on the moving direction of the person in the image obtained from the first monitoring camera. The image monitoring apparatus according to claim 4, wherein the image monitoring apparatus determines whether or not.

Images