JP5358851B2 - Suspicious behavior detection method and suspicious behavior detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for detecting a suspicious action, capable of achieving more highly reliable detection of an suspicious action, based on an objective reference even in an object of monitoring having various and complicate action patterns. <P>SOLUTION: The suspicious action detection device includes: a tracking unit 21 for tracking a person in an image input from a camera 11; a reliability deriving unit 25a for repeatedly deriving reliability showing how likely the operation of the person under tracking is a predetermined basic operation, for each of a plurality of various basic operations; a memory 22 for storing basic operation reliability time series data in which the derived reliability of each basic operation is integrated into time series data; and a suspiciousness determination unit 25c for determining whether the action of the person under tracking is a suspicious action based on the stored basic operation reliability time series data. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a suspicious behavior detection method and a suspicious behavior detection device that detect suspicious behavior in a monitoring area based on a camera image.

  As this type of monitoring system, there is a system that detects abnormal behavior by comparing the behavior of a person to be monitored with predefined behavior patterns and determination rules (see Patent Documents 1 and 2). For example, there is a monitoring system in which a place and an action are defined as an abnormal action pattern, and an action similar to the defined action is detected as an abnormal action (see Patent Document 3). In this monitoring system, motions such as staggering walking and staggered walking in a dangerous area (such as a sidewalk or a platform end of a station) are detected as abnormal behavior (abnormal walking).

  In addition, the monitoring target is analyzed in units of elements (movement of each part of the human body, appearance identification, number of visual recognition actions), and the monitoring items of the monitoring target are determined based on predetermined determination rules for each monitoring element. However, there is a system that outputs an alarm based on the comprehensive judgment (see, for example, Patent Document 4). Also, analyze the general behavior of people such as walking and bending, and identify whether or not a specific dedicated behavior is based on the time-series data on this general behavior (for example, transition from walking to bending, After that, there is an image monitoring system that determines that the patient is sick if the bending action continues for N seconds or longer (see Patent Document 5).

JP 2004-328622 A JP 2006-285399 A JP 2007-89105 A JP 2006-28784 A JP 2006-272488 A

  The method of detecting suspicious (abnormal) behavior by matching with predefined behavior patterns and judgment rules in the prior art documents can handle monitoring of behavior patterns that are relatively simple and limited in place, time, and action. It is difficult to cope with cases where various behavior patterns occur in the monitored area. For example, in a monitoring area such as a parking lot, there are multiple types of objects such as people and vehicles, the degree of freedom of entering and exiting the monitoring target and the freedom of movement are high, the possible behavior patterns are varied, and there are individual differences, It is difficult to deal with complex and various types of suspicious behaviors by matching with specific behavior patterns and judgment rules that have been defined.

  In addition, in the invention described in the prior art document, the definition of the behavior pattern to be monitored and the setting of the determination rule are performed by the system designer, etc. There is a problem that it is difficult to detect suspicious behavior objectively and with high reliability because it tends to be based on it, and there are many control parameters for determination and variations in the setting conditions are likely to occur.

  The present invention is intended to solve the above problems, and is objective in the monitoring target of various and complex behavior patterns in a wide area where it is difficult to define suspicious (abnormal) behavior by location, time, and action. It is an object of the present invention to provide a suspicious behavior detection method and a suspicious behavior detection device that can detect suspicious behavior with higher reliability on the basis of standards.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] The processing unit tracks a person in an image input from the camera, and has a plurality of types of reliability indicating the degree of basic motion likelihood determined in advance during the tracking. Repetitively deriving for each of the basic actions, and determining whether or not the action of the person is a suspicious action based on the basic action reliability time-series data in which the reliability for each basic action is time-series. Suspicious behavior detection method characterized by this.

  In the invention of [1] and [8] to be described later, at each time point during tracking, a reliability indicating the degree of basic motion likelihood determined in advance by the motion of the person being tracked is obtained for each of the plurality of basic motions. Basic operation reliability time-series data is generated in time series, and it is determined whether or not the person's action is suspicious based on the basic operation reliability time-series data. The basic motions are motions that become elements of human behavior patterns such as “grab”, “walk”, “run”, “bend”, and the like. A person's behavior pattern is formed by connecting basic actions in time series.

  When the human action at each time is specified as any one basic action, the basic action specifying accuracy is lowered during a transition from one basic action to another. In the present invention, instead of specifying any one basic motion as a human motion at each time, the reliability for each basic motion is obtained and based on the time series data of the basic motion reliability. Furthermore, since the behavior pattern is analyzed to determine whether or not the behavior is suspicious, the suspicious behavior can be detected without being influenced by the influence of the basic motion identification error at the time of the basic motion transition.

[2] having a database in which the basic operation reliability time series data is registered for a plurality of people;
Whether the action of the person is suspicious by a statistical method based on the basic action reliability time-series data related to the person to be determined and the basic action reliability time-series data registered in the database. The suspicious behavior detection method according to [1], characterized in that it is determined whether or not.

  In the invention of [2] and [9] described later, the basic operation reliability time series data acquired during tracking of the person to be determined and the basic operation reliability time series data stored in advance in the database are statistically calculated. Thus, it is determined whether or not the action of the person is a suspicious action. Since the determination is made by comparison with the sample data stored in the database by a statistical method, objective determination can be made as compared with the case where the behavior pattern of suspicious behavior is defined by a system designer or the like. Moreover, the trouble of defining a suspicious behavior pattern becomes unnecessary.

[3] The suspicious behavior detection method according to [2], wherein the database is obtained by registering the basic motion reliability time series data regarding normal behavior.

  In the invention of [3] and [10] to be described later, basic action reliability time series data relating to normal behavior (for example, human daily behavior) capable of collecting a large number of samples is registered in a database, and this normal behavior is recorded. The behavior pattern of the person and the behavior of the person being tracked are compared by a statistical method to determine whether or not the behavior pattern is suspicious.

[4] The processing unit derives a degree of deviation of the basic motion reliability time-series data related to the determination target person from the basic motion reliability time-series data related to normal behavior registered in the database by a statistical method. And determining whether or not the action of the person is a suspicious action based on the degree of detachment. The suspicious action detection method according to [3].

  In the invention of [4] and [11] described later, based on the degree of deviation from normal behavior, for example, when the degree of deviation is equal to or greater than a threshold, it is determined that the action is suspicious.

[5] The suspicious behavior detection method according to [2], wherein the database is registered with the basic operation reliability time series data regarding suspicious behavior.

  In the invention of [5] and [12] described later, basic operation reliability time series data regarding suspicious behavior is registered in a database, and the behavior pattern of the suspicious behavior is compared with the behavior of the person being tracked by a statistical method. It is then determined whether or not the action is suspicious. For example, when the degree of similarity with the suspicious behavior pattern is equal to or greater than a threshold value, the behavior of the person being tracked is determined to be suspicious behavior.

[6] The database includes a first database in which the basic operation reliability time-series data regarding normal behavior is registered, and a second database in which the basic operation reliability time-series data regarding suspicious behavior is registered,
The processing unit indicates a degree to which the action of the person is a normal action based on the basic action reliability time series data regarding the person to be determined and the basic action reliability time series data registered in the first database. Degree of normality, and the degree to which the action of the person is suspicious based on the basic action reliability time series data related to the person to be determined and the basic action reliability time series data registered in the second database The suspicious behavior detection method according to [2], wherein a suspicious degree indicating suspiciousness is derived, and whether or not the behavior of the person is suspicious behavior is determined based on the normality level and the suspicious degree.

In the invention of [6] and [13] described later, a first database in which basic action reliability time-series data related to normal behavior is registered and a second database in which basic action reliability time-series data related to suspicious behavior are registered. It is prepared and it is determined whether it is suspicious action using these together.

[7] If the normality is less than the first threshold, or if at least one of the suspiciousness is greater than or equal to the second threshold is true, the person's action is determined to be suspicious. [6] The suspicious behavior detection method according to [6].

  In the invention of [7] and [14] to be described later, the suspicious degree indicating the degree of similarity between the case where the normality indicating the degree of similarity with the normal action is less than the first threshold (when not normal) and the suspicious action. If any of the cases is equal to or greater than the second threshold value (when it is a suspicious action), it is possible to reduce the detection failure of the suspicious action.

[8] A tracking unit that tracks a person in an image input from the camera;
A derivation unit that repeatedly derives the reliability indicating the degree of the basic behavior that the person's motion during the tracking is determined in advance for each of a plurality of types of the basic operations during the tracking by the tracking unit;
A storage unit for storing basic operation reliability time-series data in which the reliability of each basic operation derived by the derivation unit is time-series;
A suspicious determination unit that determines whether or not the action of the person is a suspicious action based on the basic operation reliability time-series data stored in the storage unit;
A suspicious behavior detection device characterized by comprising:

[9] A database in which the basic operation reliability time-series data is registered for a plurality of people,
The suspicious determination unit is a statistical method based on basic operation reliability time-series data related to a determination target person stored in the storage unit and basic operation reliability time-series data registered in the database. The suspicious behavior detection device according to [8], wherein it is determined whether or not a person's behavior is suspicious behavior.

[10] The suspicious behavior detection device according to [9], wherein the database registers the basic motion reliability time-series data regarding normal behavior.

[11] The suspicious determination unit may calculate a degree of deviation of the basic operation reliability time-series data related to the determination target person from the basic operation reliability time-series data registered in the database by a statistical method. The suspicious behavior detection device according to [10], wherein the suspicious behavior is derived and determined based on the degree of detachment whether or not the behavior of the person is suspicious behavior.

[12] The suspicious behavior detection device according to [9], wherein the database registers the basic operation reliability time-series data regarding suspicious behavior.

[13] The database includes a first database in which the basic operation reliability time-series data regarding normal behavior is registered, and a second database in which the basic operation reliability time-series data regarding suspicious behavior is registered,
The suspicious determination unit is configured to store the person based on the basic operation reliability time-series data regarding the determination target person stored in the storage unit and the basic operation reliability time-series data registered in the first database. Normality indicating the degree of behavior being normal behavior, basic motion reliability time-series data related to the determination target person stored in the storage unit, and basic motion reliability time-series registered in the second database Deriving a suspicious degree indicating a degree of suspicious behavior of the person based on the data and determining whether the behavior of the person is suspicious based on the normality and the suspicious degree. The suspicious behavior detection device according to [9], characterized by:

[14] The suspicious determination unit determines that the behavior of the person is suspicious if the normality is less than a first threshold or if at least one of the suspiciousness is greater than or equal to a second threshold is true. The suspicious behavior detection device according to [13], wherein the suspicious behavior detection device is determined.

  According to the suspicious behavior detection method and the suspicious behavior detection device according to the present invention, it is possible to detect the suspicious behavior with higher reliability on an objective basis.

It is a block diagram which shows schematic structure of the suspicious action detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process which a suspicious action detection apparatus performs. It is explanatory drawing which shows an object data table. It is explanatory drawing which enumerated the basic operation | movement of the person in a parking lot. It is explanatory drawing which shows a basic operation | movement reliability matrix. It is a block diagram which shows the structural example of the reliability deriving part classified by basic operation which a reliability deriving part has. It is a flowchart which shows the process in a reliability derivation | leading-out part. It is explanatory drawing which illustrated the accumulation | storage state of the feature-value time series data. It is explanatory drawing which shows the method of taking out feature-value time series data. It is a flowchart which shows the detail of a basic operation | movement reliability matrix update process (step S14 of FIG. 2). It is explanatory drawing which shows the outline | summary of action suspicious degree derivation | leading-out determination processing (step S16, S17 of FIG. 2). It is explanatory drawing which shows the other example of the outline | summary of action suspicious degree derivation | leading-out determination processing (step S16, S17 of FIG. 2). It is explanatory drawing which shows the determination logic table used for determination of suspicious action. It is explanatory drawing which shows an example of the suspicious degree derivation process which a behavior suspicious degree derivation part performs. It is a flowchart which shows the detail of the process of the part which calculates | requires a suspicious degree among action suspicious degree derivation processes (step S16 of FIG. 2). It is a flowchart which shows the detail of the process of the part which calculates | requires normality among action suspiciousness derivation | leading-out processes (step S16 of FIG. 2). It is a flowchart which shows the detail of action suspicious degree derivation | leading-out determination processing which performs suspicious determination from suspicious degree. It is a flowchart which shows the detail of the suspicious degree derivation | leading-out determination process which performs suspicious determination from normality. It is explanatory drawing which illustrated the transition of the normality derived | led-out with respect to a human behavior pattern, and the behavior pattern, and a suspicious degree. It is a state transition diagram of the object which the tracking part of the processing part of a suspicious action detection device tracks. It is explanatory drawing which shows each process phase of the tracking determination process of the object which the process part of a suspicious action detection apparatus performs. It is a flowchart which shows a tracking discrimination | determination process. It is explanatory drawing which shows the outline of the object search by a search block. It is explanatory drawing which shows an example of the state which divided | segmented the image (image for 1 frame) of a process target into the some division block. It is a flowchart which shows an object tracking and search process (FIG. 22; step S105). It is a flowchart which shows search block generation processing (FIG. 25; step S122). It is explanatory drawing which shows the relationship between the origin (px0, py0) and center position (px, py) of a search block. It is explanatory drawing which shows a search block data table. It is explanatory drawing which shows an example of an HSV color space accumulation histogram. It is a flowchart which shows the detail of a fitness update process (FIG. 25: step S123). It is explanatory drawing which shows the relationship between the distance between search block center and the object gravity center of tracking object, and a fitness. It is a flowchart which shows the detail of a division | segmentation block area ID allocation process (FIG. 25: step S124). It is a flowchart which shows the detail of search block erasure | elimination process (FIG. 25: step S125). It is a flowchart which shows the detail of a search block resampling process (FIG. 25: step S128). It is explanatory drawing which shows an example of the roulette produced | generated by the search block resampling process. It is explanatory drawing which shows an example of the replacement condition of the search block by search block resampling processing. It is a flowchart which shows the detail of search block update process (FIG. 25: step S129). It is explanatory drawing which shows an example of the tracking and search condition of the object by the search block before and behind an intersection. It is a flowchart which shows the detail of a person and vehicle discrimination | determination process (FIG. 22: step S111).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a schematic configuration of a suspicious behavior detection apparatus 10 according to an embodiment of the present invention. The suspicious behavior detection apparatus 10 of the present embodiment is a surveillance camera system that detects a suspicious person who acts in a wide area where a plurality of objects and persons exist. In this embodiment, a surveillance camera system that monitors a suspicious person in a large parking lot will be described as an example. That is, if the moving object detected from within the camera image is a person or a vehicle, if it is determined that it is a person, the behavior is analyzed to determine whether it is a suspicious action, and when a suspicious action is detected Performs functions such as alerting observers. The camera captures a wide range, and the “view” of a person passing through the imaging area can change variously.

  The suspicious behavior detection apparatus 10 detects a target object by analyzing the image captured by the camera unit 11 and capturing the image by capturing the monitoring target area, tracking the object, and searching for the object. The processing unit 12 that performs processing such as type discrimination and human behavior analysis, the output unit 14 that outputs the processing result of the processing unit 12, and inputs of various settings and instructions regarding the suspicious behavior detection device 10 are received. The input unit 13 is configured to include a database unit 15 in which reference data for discriminating object types and analyzing human behavior is stored. The camera unit 11, the output unit 14, the input unit 13, and the database unit 15 are each connected to the processing unit 12. Hereinafter, an object recognized in the image data is referred to as an object.

  The camera unit 11 captures a moving image. The camera unit 11 functions to capture a moving image by capturing several tens of frames of images per frame (a still image for one screen) per second. The processing unit 12 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) in which programs and data executed by the CPU are stored, a RAM (Random Access Memory), and the like as main parts.

  The processing unit 12 extracts feature information of the object during tracking of the object by the tracking unit 21 that tracks and searches for an object in the image input from the camera unit 11, the storage unit 22, and the tracking unit 21, The accumulation control unit 23 that repeatedly stores the feature amount data indicating the feature information in the storage unit 22, and the object type based on the feature amount data of the object that is stored in the storage unit 22 for a predetermined time length or more. A function of a person / vehicle discriminating unit 24 for discriminating and a basic operation / behavior analyzing unit 25 for analyzing the behavior of the person based on the feature amount data of the object discriminated as a person is provided.

  The feature amount data is, for example, the shape, size, aspect ratio, movement, etc. of the object. The feature amount data is stored in the storage unit 22 in time series. The accumulation control unit 23 stores the feature data in the storage unit 22 at predetermined time intervals during tracking of the object. Here, it is performed for each frame. Note that the time-series feature amount data is referred to as feature amount time-series data.

  The tracking unit 21 has a function of tracking the same object as the same object before and after the occurrence of a phenomenon such as the intersection of the tracking target objects or the temporary hiding of the tracking target object behind the obstacle. I have.

  The person / vehicle discrimination unit 24 has a function of discriminating whether the tracking target object is a person or a vehicle. What is necessary is just to perform discrimination | determination of a person and a vehicle by well-known arbitrary methods.

  The basic motion / behavior analysis unit 25 performs a function of determining whether or not the behavior of the person being tracked is suspicious behavior. Specifically, it functions as a reliability deriving unit 25a, a behavior suspicious degree deriving unit 25b, and a suspicious determination unit 25c. The reliability deriving unit 25a predetermines a degree (reliability) of the basic behavior (“walking”, “bend”, etc.) that the person's action during the tracking is predetermined at a plurality of time points during the person's tracking. Each of the multiple types of basic operations is derived. The behavior suspiciousness deriving unit 25b is a basic operation reliability time-series data in which the reliability for each basic operation derived by the reliability deriving unit 25a is time-sequentially stored in the behavior pattern reliability time-series DB unit 15d. From the operation reliability time series data, the degree that the person's action is a daily action (normal action) (normal degree), or the degree that the person's action is suspicious action (suspicious degree) in addition to the normal degree Is derived. The suspicious determination unit 25c determines whether or not the person's action is suspicious based on the normality or the suspicious degree derived by the behavior suspicious degree deriving unit 25b.

  The database unit 15 includes a human feature DB unit 15a in which feature amount data such as a shape related to a person is stored, a vehicle feature DB unit 15b in which feature amount data such as a shape related to a vehicle is stored, and a feature amount related to a basic human motion. A basic motion feature amount time series DB unit 15c in which time series data and the like are stored and an action pattern reliability time series DB unit 15d are configured. The behavior pattern reliability time series DB unit 15d is a daily behavior pattern related reliability time series DB 15d1 in which behavior patterns related to daily activities of humans (basic motion reliability time series data related to daily activities) are accumulated, and behaviors related to suspicious behavior of people. A suspicious behavior pattern related reliability time series DB 15d2 in which patterns (basic motion reliability time series data related to suspicious behavior) are stored is configured.

  For example, behavior patterns of daily behavior in a parking lot include passing near the vehicle, getting off the vehicle, leaving, and getting on the vehicle as soon as possible. The behavior pattern of suspicious behavior is a behavior pattern when a criminal act such as vandalism or vehicle theft is performed. For example, the behavior pattern of suspicious behavior is to get on after turning around the vehicle, or to get on after getting bent for more than a certain time near the vehicle.

  The person / vehicle discriminating unit 24 refers to the feature amount data stored in the human feature DB unit 15a to determine whether the object is a person, and the feature amount data stored in the vehicle feature DB unit 15b. To determine whether or not the object is a vehicle.

  The reliability deriving unit 25a includes feature time series data such as movements of an object determined to be a person in a past predetermined time and a feature amount time series relating to a predetermined basic motion accumulated in the basic motion feature time series DB unit 15c. The reliability for each basic motion is derived from the data using statistical methods. The reliability for each basic operation derived by the basic operation / behavior analysis unit 25 is stored in the storage unit 22 or the like as basic operation reliability time-series data in which these are time-sequential.

  The behavior suspiciousness deriving unit 25b is accumulated and stored in the basic motion reliability time series data derived by the reliability deriving unit 25a and the daily behavior pattern related reliability time series DB 15d1 of the behavior pattern reliability time series DB unit 15d. The degree of normality indicating the degree to which the person's action is a normal action is derived from the basic action reliability time series data regarding the daily action. Alternatively, in addition to the normality, the basic operation reliability time series data derived by the reliability deriving unit 25a and the suspicious behavior pattern related reliability time series DB 15d2 of the behavior pattern reliability time series DB unit 15d are stored and stored. A suspicious degree indicating a degree of the suspicious behavior of the person is derived from the basic motion reliability time series data regarding the suspicious behavior by a statistical method. Alternatively, regarding the suspicious degree, the basic motion reliability time-series data derived by the reliability deriving unit 25a and the daily behavior stored and stored in the daily behavior pattern related reliability time-series DB 15d1 of the behavior pattern reliability time-series DB unit 15d. The suspicious degree is derived as a degree of deviation from the daily action of the person's action from the basic action reliability time series data regarding the action by a statistical method.

  The output unit 14 includes an interface circuit for outputting a processing result data signal to the outside, various setting screens and operation screens, a display device for visually displaying processing results, and the like. The input unit 13 includes various operation switches in addition to an interface circuit for inputting signals relating to various settings and processing instructions.

  Next, the process which the suspicious action detection apparatus 10 performs is demonstrated.

  FIG. 2 shows a flow of processing performed by the suspicious behavior detection apparatus 10. A detailed example of processing relating to the tracking unit 21, the storage unit 22, the accumulation control unit 23, and the person / vehicle determination unit 24 will be described later. Here, it is assumed that the image input from the camera unit 11 is analyzed to track an object, and it is determined whether each object is a person or a vehicle. The processing to be performed will be mainly described.

  An object data table 30 shown in FIG. 3 is created for each object being tracked. The object data table 30 is created every time a new object is detected, and is updated every time one frame of image is processed until the object disappears.

The object data table 30 includes an object ID (obj_id), an object type (obj_type ^obj_id ), an object state (obj_state ^obj_id ), an object suspicious determination flag (suspicious_flag ^obj_id ), an object existence duration (obj_time ^obj_id ), Object area left edge position (obj_lx ^obj_id ), object area right edge position (obj_rx ^obj_id ), object area top edge position (obj_ty ^obj_id ), object area bottom edge position (obj_by ^obj_id ), object area gravity center position (horizontal direction) (obj_gx ^obj_id ), object area centroid position (vertical direction) (obj_gy ^obj_id ), object size (obj_size ^obj_id ), object aspect ratio (obj_aspect ^obj_id ), and object motion vector (horizontal direction) (obj_dx ^obj_id ) And the motion vector of the object (vertical And direction) (obj_dy ^obj_id), Object - nearest vehicle distance and (obj_car_dist ^obj_id) is registered.

The object ID (obj_id) is an identifier uniquely assigned to the object. Whether the object type (obj_type ^obj_id ) is an object that has been determined to be a person (hereinafter also referred to as a human object) or an object that has been determined to be a vehicle (hereinafter also referred to as a vehicle object) is not determined. Indicates whether it is an object. The object state (obj_state ^obj_id ) indicates whether the state of the object is being tracked (ACTIVE) or temporarily lost (LOST). The suspicious determination flag (suspicious_flag ^obj_id ) indicates whether the object is a human object that is set when the object is a human and is determined as a suspicious person (person who has performed suspicious behavior). The distance between the object and the nearest vehicle (obj_car_dist ^obj_id ) is set when the object is a person, and indicates a distance to the nearest vehicle.

  Returning to FIG. 2, the description will be continued. After initializing the frame time t to 0 (step S1), the processing unit 12 checks whether or not an object (person object) that is determined to be a person exists in the tracked object (step S2). . If there is no tracking person object (tracking person) (step S3; No), the frame time t is incremented by 1 (step S22), the process proceeds to step S2, and the process is continued. Note that the processing unit 12 executes the processes in steps S2 to S22 every time one frame image (or a predetermined plurality of frames) is captured from the camera unit 11.

  If there is a tracking human object (tracking person) (step S3; Yes), it is checked whether there is a disappearing human object (disappearing person) (step S4). If there is no disappeared person object (step S4; No), the process proceeds to step S9. If there is a disappeared person object (step S4; Yes), the object data table 30 of all the disappeared person objects is cleared (steps S5 to S8), and the process proceeds to step S9.

In step S9, the variable i indicating the number of the human object to be processed is cleared to 0, and the processes of steps S10 to S21 are performed for each of the tracking human objects. That is, it is checked whether or not the suspicious determination flag (suspicious_flag ^{obj_id (i)} ) is in an ON state (ON indicates a suspicious person) (step S10). The time series data is updated (step S11). Here, the contents of the object data table 30 for each frame (all or a part (top / bottom / left / right end position, center of gravity position, motion vector, size, aspect ratio, object-nearest vehicle distance, etc.)) The feature amount data of the object is stored in the storage unit 22 in time series (as feature amount time series data).

Next, it is checked whether or not the feature amount time-series data has been accumulated for _Tb time or more (step S12). If the accumulation amount is less than _Tb time (step S12; No), the process proceeds to step S20. To do. When accumulated for _Tb time or more (step S12; Yes), the reliability of the human object for each predetermined basic motion is determined with respect to the motion of the human object based on the feature amount time-series data accumulated for _Tb time or more. A basic motion reliability deriving process for deriving the degree is performed (step S13).

FIG. 4 lists the basic actions of people in the parking lot. In this case, “basking”, “walking”, “running”, “bending”, “getting on (disappearing)”, and “getting off (appearing)” are the basic operations. R _stand , the reliability of the basic operation “walk”, R _walk , the reliability of the basic operation “run”, R _run , the reliability of the basic operation “run”, R _bend , the reliability of the basic operation “bend”, the basic operation The reliability of “ride (disappearance)” is R _{into_car} , and the reliability of the basic operation “get off (appearance)” is R _{outof_car} .

Subsequently, the basic operation reliability time-series data is updated with the reliability for each basic operation obtained (FIG. 2, step S14). Here, the reliability for the latest _Ta time of each target basic operation is stored and updated as a basic operation reliability matrix as shown in FIG. Note that T _a > T _b . For example, _{T a} is 10 seconds, _{T b} is set to 1 second.

Then it checks whether feature time-series data (Basic operation reliability time-series data) is accumulated in T _a more time min (step S15), and if the accumulation amount is less than T _a time duration is (step S15 No), the process proceeds to step S20. If the accumulated T _a more time minutes, i.e., if the basic operation reliability data T _a time period is stored in the basic operational reliability matrix; stored as (step S15 Yes), the basic operational reliability matrix Based on the obtained basic motion reliability time series data, behavior suspiciousness derivation processing for deriving a suspicious degree (or normality and suspiciousness) related to the behavior of the person object is performed (step S16).

If obtained suspiciousness degree, or to determine suspicious behavior or not on the basis of the normal level and the suspiciousness degree it was determined to be a suspicious behavior; a (step S17 Yes), suspicious judgment flag ^{(suspicious_flag obj_id (i))} Turn on (step S18), display an alarm (step S19), and proceed to step S20. Here, if the determination based on the suspiciousness degree, the suspiciousness degree is judged to be suspicious behavior when a predetermined threshold Th ₂ or more. In the determination based on the normality and the suspicious degree, it is determined that the suspicious behavior is true when either the normality is less than the predetermined threshold Th ₁ or the suspicious degree is greater than or equal to the predetermined threshold Th _2. .

When the suspicious determination flag (suspicious_flag ^{obj_id (i)} ) is already ON (step S10; Yes), an alarm is displayed (step S19), and the process proceeds to step S20.

  In step S20, the variable i is incremented by 1 and it is checked whether or not the variable i after addition is equal to the number of all-person objects being tracked (step S21). If they are not equal (step S21; No), the process proceeds to step S10. Return and process the next person object. If equal (step S21; Yes), the frame time t is incremented by 1 (step S22), and the process returns to step S2 to process the next frame image.

  Next, the basic operation reliability deriving process (step S13 in FIG. 2) will be described.

  FIG. 6 shows a configuration example of the reliability derivation unit 26 by basic operation that the reliability derivation unit 25a has. The reliability derivation unit 26 for each basic operation in FIG. 6 has a configuration for obtaining the reliability related to one basic operation, and the reliability derivation unit 25a has the reliability derivation unit 26 for each basic operation for the number of target basic operations. It has.

The reliability derivation unit 26 for each basic operation includes a plurality (in this case, Mf) discriminators 27 and an addition for adding (load sum) a value obtained by multiplying the output values of the Mf discriminators 27 by weighting factors. A container 28 is provided. The output R ^j of the adder 28 is a reliability with respect to a predetermined basic operation. Each of the discriminators 27 includes a feature amount for a predetermined time with respect to any feature amount data that contributes to the determination of whether or not the basic motion is a basic motion (for example, “bend”). time series data set (e.g., T _b time duration (feature data sets of T _b number) which is the time series) entered as to output a numerical value indicating the degree of the basic operation likeness applicable as a reliability R ^j. For example, the discriminator 27 is provided for each feature amount data such as the shape, size, moving speed, and movement pattern of an object, and each discriminator 27 is based on a time-series data set of feature amounts contributing to the discriminator 27. Judge the basic behavior. Each discriminator 27 is tuned in advance (adjusts the weighting factors α ^{1 to} α ^Mf ) with a large number of samples of the movement of the person who performs the target basic motion. Here, the more likely a basic operation of the subject, weighted sum R ^j of the output of Mf number of classifiers 27 is to be a large value.

FIG. 7 is a flowchart showing the processing in the reliability deriving unit 25a. In the figure, i is an identification number of the basic operation for obtaining the reliability, and m is an identification number of each discriminator 27. In FIG. 7, the process of obtaining the reliability for one basic operation (steps S34 to S37) is repeated for the number of basic operations. Specifically, it initializes a variable i to 1 (step S31), the i-th of the reliability _R ^{i j} Basic operation is initialized to 0 (step S32), and initializes a variable m (step S33). In S34 to S37, values obtained by multiplying the output values of the Mf discriminators 27 by weighting factors are added.

FIG. 8 illustrates an accumulation state of feature amount time-series data. The feature amount time series data storage area F stores feature amount data up to Lmax time before the current time in time series. Note that Lmax> T _a > T _b . The storage area F is provided in the storage unit 22, for example. FIG. 5A shows a state in which feature amount time-series data having a _Tb time length necessary for deriving the basic operation reliability is accumulated. The storage area F is not clogged and has a free area. FIG. 5B shows a state where feature amount time-series data having _{a Ta} time length necessary for suspicious behavior pattern determination is accumulated. The storage area F is not yet filled and there is a free area. FIG. 5C shows a state where the storage area F is filled with feature amount time-series data. Thereafter, the data is shifted one by one to the left, the oldest data is discarded, and the latest data is stored in a free area at the right end. That is, the stored contents are updated in a FIFO (First in First Out) format.

Figure 9 is input to the reliability deriving unit 25a (discriminator 27 of the basic operation by the reliability deriving unit 26), the feature time-series data of the time length T _b min, during removal from the storage area F in FIG. 8 The method of taking out is illustrated. 9 (a) is a case of inputting separated for each time series interval time length T _b, FIG. 9 (b), a case of inputting by sliding the time-series interval (window) of time length T _b Show.

FIG. 10 shows details of the basic operation reliability matrix update process (step S14 in FIG. 2). First, to confirm the number _{N Ta_data} time data of the time length _{T a} stored in the storage area F (step S51). Here, “time data” refers to data corresponding to the past _Ta time length from the reference time t with reference to the frame time t described in FIG. 2 (basic motion reliability matrix data at the reference time t). It is. After confirming the number _{N Ta_data} time data in step S51, and updates the basic operational reliability matrix data of the _{N Ta_data} pieces worth of time length _{T a} (step S52~S62).

FIG. 11 shows an overview of processing related to behavior suspicious degree derivation determination processing (steps S16 and S17 in FIG. 2). In FIG. 11, the behavior suspiciousness degree deriving unit 25b performs behavior pattern data (basic motion reliability time series data (basic motion reliability matrix)) related to a number of daily behaviors stored in the daily behavior pattern related reliability time series DB 15d1. A typical daily behavior pattern (representative daily behavior pattern) is derived by performing predetermined statistical processing on the basis of this, and this is compared with the basic motion reliability matrix at the time t of the processing target, and the basic motion of the processing target Processing such as deriving a degree of suspiciousness from the representative daily behavior pattern of the behavior pattern indicated by the reliability matrix is performed. Suspicious determination unit 25c, when the threshold value Th ₂ or more suspiciousness degree outputting action suspiciousness degree deriving unit 25b has a predetermined, determines that suspicious behavior.

  FIG. 12 shows another example of the outline of the process related to the behavior suspicious degree derivation determination process (steps S16 and S17 in FIG. 2). In FIG. 12, the behavior suspicious degree deriving unit 25b includes a normality deriving processing unit 25b1 and a suspicious degree deriving processing unit 25b2. The normal motion derivation processing unit 25b1 receives the basic motion reliability matrix at time t and the basic motion reliability time series data (basic motion reliability matrix) stored in the daily behavior pattern related reliability time series DB 15d1, and is suspicious. The basic motion reliability matrix at time t and the basic motion reliability time series data (basic motion reliability matrix) stored in the suspicious behavior pattern related reliability time series DB 15d2 are input to the degree derivation processing unit 25b2.

  The normality derivation processing unit 25b1 is based on behavior pattern data (basic motion reliability time series data (basic motion reliability matrix)) related to a large number of daily behaviors stored in the daily behavior pattern related reliability time series DB 15d1. Perform a predetermined statistical process to derive a representative daily action pattern (representative daily action pattern), and compare this with the basic motion reliability matrix at the time t to be processed, and the basic of the processing target for the representative daily action pattern A process of deriving the similarity of the behavior pattern indicated by the motion reliability matrix as the normality is performed.

  The suspicious degree derivation processing unit 25b2 performs predetermined processing based on behavior pattern data (basic motion reliability time series data (basic motion reliability matrix)) related to suspicious behavior stored in the suspicious behavior pattern related reliability time series DB 15d2. By performing statistical processing, a representative suspicious behavior pattern (representative suspicious behavior pattern) is derived, and the basic motion reliability of the processing target for the representative suspicious behavior pattern is compared with the basic motion reliability matrix at the time t of the processing target. A process of deriving the similarity of the action pattern indicated by the degree matrix as the suspicious degree is performed.

The suspicious determination unit 25c determines whether or not the action of the person to be determined is a suspicious action based on the normality output from the normality derivation processing unit 25b1 and the suspicious degree output from the suspicious degree derivation processing unit 25b2. FIG. 13 shows a decision logic table 40 used for this decision. If at least one of the condition that the suspicious degree is equal to or greater than the threshold Th ₂ and the condition that the normality is less than the threshold Th ₁ is satisfied, the suspicious action is determined.

FIG. 14 shows an example of the suspicious degree derivation process performed by the behavior suspicious degree deriving unit 25b. Of these, FIG. 14A shows an outline of the method described in FIG. That is, a predetermined statistical process is performed on behavior pattern data related to a large number of daily behaviors stored in the daily behavior pattern related reliability time series DB 15d1 to obtain representative daily behavior patterns (u _normal : daily behavior pattern DB representative matrix). The degree of deviation (distance: dist (u _normal , mtx_rel ^ref_t )) of the behavior pattern indicated by the basic behavior reliability matrix (mtx_rel ^ref_t : basic motion reliability matrix at time t) from the representative daily behavior pattern is suspicious. Derived as (S ^ref_t _suspicious ). FIG. 14B shows an overview of the method described in FIG. That is, a representative matrix indicating representative suspicious behavior patterns derived by performing predetermined statistical processing on the suspicious behavior pattern data stored in the suspicious behavior pattern related reliability time series DB 15d2, and the basic motion reliability at time t Similarity with time series data (basic motion reliability matrix) is obtained as suspicious degree. These methods are merely examples, and the suspicious degree and normality may be obtained by other methods.

15 and 16 show details of the behavior suspicious degree derivation process (step S16 in FIG. 2). FIG. 15 is a process for deriving the suspicious degree, and FIG. 16 is a process for deriving the normality. In either process, first checks the number _{N Ta_data} time data of the time length _{T a} stored in the storage area F (Figure 15: Step S71, FIG. 16: step S81). _Then, to derive the suspiciousness degree-health of _{N Ta_data} pieces worth of basic operational reliability behavioral pattern matrix data represents the time length _{T a} (FIG. 15: step S72～S78, Figure 16: Step S82~S88). As for the suspicious degree, the maximum value among N _{ta_data} pieces of data is searched, and the suspicious degree maximum value is derived as the final suspicious degree. For Normality searches the minimum value among the N _{Ta_data} pieces of data, and derive the health of minimum value as a final normality.

  In this case, the suspicious degree maximum value and the normality minimum value derived in the processes of FIGS. 15 and 16 are used in the behavior suspicious determination process (step S17 of FIG. 2) based on the determination logic table of FIG. Make suspicious / normal judgments of human behavior.

FIGS. 17 and 18 show behavior suspicious degree derivation determination processing in which the action suspicious degree derivation processing (step S16) and behavior suspicious degree determination processing (step S17) of FIG. 2 are integrated to increase the processing speed. . FIG. 17 shows a process for making a suspicious determination from the degree of suspiciousness, and FIG. The same step numbers are assigned to the same parts as those shown in FIGS. The difference from the processing shown in FIGS. 15 and 16 is that when there is time data that satisfies even one of the conditions based on the suspicious judgment thresholds Th ₁ and Th ₂ (FIG. 17: Step S75A; Yes, FIG. 18: Step S85A; Yes), it is determined as “suspicious” (FIG. 17: step S75B, FIG. 18: step S85B), and the process is terminated. In the case of following the determination logic table 40 of FIG. 13, if only one of the suspiciousness level and the normality level satisfies the suspicious determination threshold condition, it is determined as “suspicious” and both the processes in FIGS. 17 and 18 are ended. What should I do?

  In the processing shown in FIGS. 17 and 18, the processing speed can be increased compared to the processing methods shown in FIGS. 15 and 16 in which all time data stored in the storage area F is confirmed. On the other hand, since the processing shown in FIGS. 15 and 16 is performed for all the stored time data, the reliability of the suspicious behavior / normality determination can be improved as compared with the processing methods shown in FIGS.

  FIG. 19 exemplifies human behavior patterns and transitions of normality and suspiciousness derived for the behavior patterns. FIG. 6A shows a state in which a person appears in the camera shooting area (in the detection target area scene). At this time, the normality is high and the suspiciousness is low (time T1). FIG. 6B shows a case where a tracking person is circling the vehicle, and the suspicious degree is high at this time (time T2). FIG. 6C shows a case where a person being tracked is bending in front of the door of the vehicle and trying to open the door key. At this time, the degree of suspiciousness is high (time T3). FIG. 4D shows a case where the person being tracked enters the vehicle after opening the door. At this time, the degree of suspiciousness is high (time T4).

For example, by setting the T _a to 10 seconds, whether suspicious behavior is derived normal degree-suspiciousness degree from the comparison between the behavior pattern and daily behavior pattern DB or suspicious behavior pattern DB during that 10-second decision Is done.

  If it is determined that the action is suspicious, for example, a process of displaying a warning mark on the portion of the person on the monitoring image displayed on the monitor is performed.

  In this way, paying attention to the basic movements of people (such as “walking”, “running”, “bending”, and “crunching”) in a parking lot, etc., the human movement at each time is changed to one basic movement. Rather than specifying, the reliability of each basic action is obtained, and the suspiciousness or normality of the action is derived based on the time series data (basic action reliability matrix) of the basic action reliability. Therefore, it is possible to detect suspicious behavior with higher reliability. In addition, when specifying a human action as any one basic action at each time, the basic action specifying accuracy becomes low during the transition from the basic action to another basic action. Specifying the basic action reduces the normal / suspicious judgment accuracy of the subsequent action pattern.

  In the present invention, the suspiciousness and normality are derived from the accumulated daily behavior pattern data and suspicious behavior pattern data (sample data as facts) and the behavior pattern data of the person being tracked by a statistical method. Since it is determined whether or not the behavior is suspicious, unlike the case where the system designer sets the determination rule, it is difficult to include subjective elements and objective determination is possible. In particular, since a large number of behavior pattern data relating to daily behavior can be collected, the reliability of the determination by comparison with the daily behavior pattern is high.

Further, unlike the case where determination rules and the like are determined, it is not necessary to set many conditions (control parameters), and the labor related to the setting is reduced. Moreover, since there are few conditions to set, the dispersion | variation in the determination result based on condition setting reduces, and the objectivity of determination increases. In the present embodiment, the control parameters to be set are only T _a , T _{b, and} Th ₂ when the determination of FIG. 11 is performed, and T _a , T _b, Th ₁ , and Th when the determination of FIG. 12 is performed. There are only _two .

  Next, an object tracking method by the tracking unit 21, the storage unit 22, and the accumulation control unit 23 will be described.

  In this tracking method, even if a phenomenon such as the intersection of the tracking target objects or the temporary hiding of the tracking target object behind the obstacle occurs, the same object can be tracked as the same object before and after that. It can be done. Note that the following tracking method is an example, and the tracking method is not limited to this.

  FIG. 20 illustrates a state transition of an object tracked by the tracking unit 21 of the suspicious behavior detection device 10. The object status is undetected UNDETECTED, PENDING determining whether the object detected in the UNDETECTED state is an object that can be tracked, ACTIVE being tracked and successfully tracked, and tracking It is managed in four states: LOST, which is the target object, but tracking has failed (disappearing or searching).

  While the object is not detected, the UNDETECTED state continues (E1 in the figure). When an object is detected in the UNDETECTED state, the state shifts to the PENDING state (E2). In the PENDING state, the duration is counted (E4), and when a predetermined time (PENDING_TIME) elapses in the PENDING state, the object transitions to the ACTIVE state (E5). An object that disappears from the image before transitioning from the PENDING state to the ACTIVE state transitions to the UNDETECTED state (E3), and is excluded from the tracking target.

  In the ACTIVE state, the duration is counted (E6). If tracking of an object in the ACTIVE state fails (the object disappears), the object transitions to the LOST state, and the duration of the ACTIVE state is reset (E7). Even in the LOST state, the duration is measured (E9), and when the predetermined duration (LOST_TIME) elapses, the object is removed from the tracking target and becomes the UNDETECTED state (E10). An object found (detected) by the search by the search block within a predetermined time (LOST_TIME) after entering the LOST state returns to the ACTIVE state (E8).

  FIG. 21 shows each processing phase of object tracking determination processing performed by the processing unit 12 of the suspicious behavior detection apparatus 10. FIG. 22 shows the flow of the tracking discrimination process. In the tracking determination process, object tracking and whether each object is a vehicle or a person are determined. As shown in FIG. 21, the tracking determination processing includes an object detection phase P1, an object tracking / search phase P2, a person / vehicle determination phase P3, and a human object extraction phase P4. The object detection phase P1 corresponds to steps S102 to S104 in the flowchart of FIG. 22, the object tracking / search phase P2 corresponds to steps S105 to S110, the person / vehicle discrimination phase P3 corresponds to step S111, and the human object extraction phase P4 corresponds to steps S112 and S113. In the process SA indicated by the broken line in FIG. 22, if the process from step S2 to step S21 in FIG. 2 is performed, tracking and detection of suspicious behavior are performed for each frame.

<Object detection phase P1>
First, the frame time t is initialized to 0 (FIG. 22; step S101), and then an object candidate area is detected from the processing target image captured by the camera unit 11 (step S102). The object candidate area is detected by a known method such as a difference process between frames or a binarization process.

  Next, it is confirmed whether or not one or more object candidate areas have been detected in the processing target image (step S103). When one or more object candidate areas are detected (step S103; Yes), the process proceeds to step S105. If no object candidate area is detected (step S103; No), it is confirmed whether or not there is an object whose state is LOST (hereinafter referred to as a LOST object) (step S104). The LOST object is an object in which an object detected as a tracking target in the past disappears (tracking failure), and the duration of the disappearance state is less than a predetermined time (LOST_TIME), and is an object being searched by a search block described later.

  When one or more LOST objects exist (step S104; Yes), the process proceeds to step S110. If no LOST object exists (step S104; No), the frame time t = t + 1 is set (step S114), the process returns to step S102, and the same processing is performed on the image of the next frame captured by the camera unit 11. Do. As described above, the object detection phase P1 is a state in which it is confirmed whether or not one or more object candidate areas exist, and when there is no object candidate area, the detection of the object candidate area is awaited.

<Object Tracking / Search Phase P2>
In step S105, a tracking / search process is performed on the object candidate area detected in the object detection phase P1. In the object tracking / search process, a tracking process for associating an object in the image at the previous time with an object in the image at the current time and a search process for searching for an object that disappears due to hiding or crossing are performed. The tracking process is performed by a known method such as a motion vector.

  The search process for searching for the lost object is to generate a plurality of search blocks around the center of gravity of the newly detected tracking target object, and then cause each search block to follow the tracking target object. The tracking is performed by focusing on the local area (here, the area of the search block unit) in the object area. The search block is a unit area for tracking and searching for an object. If the search for the lost object is not successful within a predetermined time (LOST_TIME), the searched object is regarded as having been lost and is set to the UNDETECTED state, and is excluded from the tracking target. Details of the object tracking / search process will be described later.

  For objects that have been successfully tracked / searched in the object tracking / search process (objects in the active state), feature information such as shape, motion, texture, etc. is extracted (calculated) (step S106). Extraction of object feature information such as shape and movement may be performed by any known method. The feature amount data indicating the extracted feature information of the object is sequentially accumulated in the storage unit 22, and updated or newly generated as time series data (step S107). The feature amount data accumulated in time series for each object is used to determine whether the object is a person or a vehicle.

  Thereafter, if the object to be tracked is an object that has already been identified as a vehicle (step S108; Yes), the frame time t = t + 1 is set (step S114), the process returns to step S102, and the next frame captured by the camera unit 11 is obtained. Similar processing is performed on the image. When the object to be tracked is already determined to be a person and the object exists in the image (tracking / searching has succeeded) (step S109; Yes), the human object extraction process in step S113 Migrate to

  If neither the vehicle nor the person is discriminating the object being tracked (step S109; No), the state (status) of each object is updated according to the state transition of FIG. obj_time is incremented and updated (step S110), and the process proceeds to a person / vehicle discrimination process (step S111).

<Person / vehicle discrimination phase P3>
In the person / vehicle discrimination process, based on the time-series feature amount data (feature amount time-series data) of the tracking target object accumulated and stored in the object tracking / search phase P2, whether the tracking target object is a person, It is determined whether the vehicle is a vehicle (step S111). The person / vehicle discrimination process is performed by an appropriate known method.

<Human object extraction phase P4>
If it is determined in the person / vehicle determination phase P3 that the object to be tracked is a person (step S112; Yes), the process proceeds to a human object extraction process (step S113). (S112; No), the frame time t is incremented and updated (step S114), the process returns to step S102, and the same processing is performed on the image of the next frame captured by the camera unit 11.

  In the human object extraction process, the location of the person is communicated to the monitoring person in an easy-to-understand format, and monitoring attention is urged (step S113). For example, when an image from the camera unit 11 is displayed on the display device of the output unit 14, the detected person's location is displayed in a red frame. Thereafter, the frame time t is incremented and updated (step S114), and the process returns to step S102 and continues. Suspicious behavior detection processing (steps S2 to S21 in FIG. 2) is performed on the extracted human object.

  Next, details of the object tracking / searching process in step S105 of FIG. 22 will be described.

  In the object tracking / search process, tracking of an object and searching for an object (LOST object) that has failed (disappeared) in tracking are performed by a known method. The search is performed by a plurality of search blocks that move so as to track the disappeared tracking target object.

  FIG. 23 shows an outline of the search by the search block. For a newly detected object A to be tracked (person A in the example of FIG. 23), a plurality of search blocks D having a predetermined size for which the object A is tracked are generated, and each of these search blocks D is The characteristic information of the tracking target object A is associated and stored (t1). In FIG. 23, each rectangular area shaded is a search block D. Each object to be tracked is assigned a unique object ID as identification information for uniquely identifying the object, and the object ID of the object to be tracked is registered by registering the object ID of the object to be tracked in each search block. Is associated with the search block.

  After that, the object is tracked by an existing method such as matching of the image of the frame at the previous time and the image of the frame at the current time, and each search block D is tracked to the object to be tracked (the object's object). Move so as to follow the motion vector (FIG. 23; t2).

  If tracking of an object fails (disappears) due to crossing or hiding (FIG. 23; t3), the movement of the search block D whose tracking target is the object is continued according to a predetermined movement rule. For example, the movement is continued at random, or the movement is continued by combining the movement of the object before disappearance with the random movement. In FIG. 23, the object A is hidden behind the obstacle 61 at time t3, and each search block D is moving with a motion that combines the motion of the object A up to the current time and a random motion. The influence of the motion before the disappearance of the object A gradually decreases, and the random motion becomes dominant, and the search block gradually moves so as to be spread over a wide range (t4). In FIG. 23, the character “t3” is attached to each search block at time t3, and the character “t4” is attached to each search block at time t4.

  As described above, when an object exists at the position (existing region) of the search block continuously moved so as to search for the lost object, the feature information of the object is stored in association with each search block. The feature information of the tracking target object is compared, and based on the comparison result, it is determined whether or not the object existing at the position of the search block is the tracking target object of the search block. If the object is a tracking target object, the object ID of the tracking target object registered in the search block is assigned to the object. Thereby, the same object is tracked as the same object before and after disappearance.

  In the example of FIG. 23, the object A is found (detected) by the search block D1 at time t4. After the object to be tracked disappears, each search block moves and spreads with the random movement dominant, so if the object A appears at the position A ′ indicated by the broken line, It will be discovered by the search block D2 which has moved to the position. In this way, even if an intersection or a temporary hiding behind an obstacle occurs, the same object is tracked as the same object (with the same object ID) before and after that.

  In the present embodiment, as shown in FIG. 24, an image to be processed (image for one frame) is divided into a plurality of small regions (divided blocks Bk), and an object is captured in divided block units. Thereby, the amount of calculation related to tracking is reduced. Here, the image to be processed is divided by a grid-like boundary line so that each divided block Bk is rectangular.

  In the example of FIG. 24, the object A is divided into 33 rectangular divided blocks Bk that are shaded. Tracking / searching of the object A is performed for each divided block Bk. The object A is recognized as an aggregate of the 33 divided blocks Bk, and the feature information such as the shape, the center of gravity position, and the movement of the object A as a whole is grasped. Here, when the area of the object occupying the divided block Bk is equal to or greater than a predetermined occupancy ratio (for example, 50% or more), the divided block is regarded as a divided block where the object exists.

  FIG. 25 shows the flow of the object tracking / search process (FIG. 22; step S105). In object tracking (FIG. 25; step S121), an object ID is assigned to a newly detected object and an object data table 30 as shown in FIG. 3 is generated.

  For objects that have been detected at the past time and have already been assigned an object ID, tracking is performed by a method such as associating an object in the previous time image with an object in the current time image by a known method such as a motion vector. To do.

  Next, search block generation processing for generating a search block for the object that is newly tracked is performed (step S122). Details of the search block generation processing are shown in FIG. A state (status) that is present in the processing target image is ACTIVE (step S142; Yes), and there is no search block for which the object is a tracking target (step S143; Yes). A plurality of search blocks are generated around the center of gravity (steps S144 to S155).

  A search block is a unit area for tracking / searching an object to be tracked. Whether or not an object existing in the search block's existing area is the object to be tracked by the search block, with the search block as a unit. To be judged. A predetermined number (BLOCKFACTOR) of search blocks is generated for each divided block in which an object to be tracked exists. That is, the number of search blocks corresponding to the size of the tracking target object is generated. The number of BLOCKFACTOR is one or more and is a number that can be arbitrarily changed, and may be set to an appropriate number based on the ratio of the area size of the divided block and the area size of the search block. For example, the number of BLOCKFACTOR is set to 2 to 5. FIG. 27 shows the relationship between the origin (px0, py0) and the center position (px, py) of the search block. The search block is a pixel area of BLOCK_SIZE × BLOCK_SIZE (in the example of FIG. 27, an area of 5 × 5 pixels). Here, although the search block is rectangular, the shape of the search block is not limited to this, and may be another shape such as a circle.

  Further, a search block data table 70 as shown in FIG. 28 is generated for each generated search block. In the search block data table 70, information indicating the attribute of the corresponding search block is registered. The following values are registered in the search block data table 70 as initial values when they occur.

-Object ID: obj_id → Object ID to be tracked
Search block state (status): state ^obj_id _{n_block} → ON
Search block position (x coordinate): bx ^obj_id _{n_block} → x
^-Search block position (y coordinate): by ^obj_id _{n_block} → y
Search block motion vector (horizontal direction): dx ^obj_id _{n_block} → dx (gx ^obj_id , gy ^obj_id )
^-Search block motion vector (vertical direction): dy ^obj_id _{n_block} → dy (gx ^obj_id , gy ^obj_id )
・^Fitness of search block: cor ^obj_id _{n_block} → 1.0
Also, an HSV color space cumulative histogram as shown in FIG. 29 is generated from the HSV value (pixel value expressed in HSV color space) of each pixel in the search block existence region. This cumulative histogram is derived by scanning the search block area in units of pixels. For pixel value i, if j ≧ i,
hist _{H, S, V} (t, j) ^obj_id _{n_block} = hist _{H, S, V} (t, j) ^obj_id _{n_block} + 1
And That is, the possible values of the pixel value are divided into a predetermined number of classes, and if the pixel value of the pixel is i, “1” is added to each of the accumulated values of the classes to which i belongs. This process is performed for all the pixels in the search block.

  The HSV color space is a color space composed of three components: hue, saturation (saturation / chroma), and brightness (brightness / lightness / value), and the HSV color space cumulative histogram is the H (hue) component. , S (saturation) component and V (lightness) component. Each value related to the HSV color space cumulative histogram is stored in association with the search block as feature information of the object (image) at the position (existing region) of the search block.

Next, the fitness for all search blocks is updated (FIG. 25: step S123, details are shown in FIG. 30). In the process of updating the fitness level, the fitness level is updated (FIG. 30; steps S164 to S167) for all search blocks whose search block status (status) is ON (FIG. 30; step S163; Yes). Do. The goodness of fit is an index indicating the degree of coincidence between the object existing at the position (existing area) of the search block and the tracking target object of the search block. The degree of fitness is the feature information of the object existing in the existence area of the search block (feature information as an image such as color and the feature information related to the center of gravity position) and the feature information of the object stored in association with the search block (the relevant It is calculated based on a comparison with the feature information of the object existing at the previous time position of the search block. In this embodiment, a case where an HSV color space cumulative histogram is used will be described. Specifically, by the product of the time difference of the HSV color space cumulative histogram in the search block existence area and the distance coefficient based on the distance between the center of gravity of the object to be tracked by the search block and the center of the search block. To derive. The goodness of fit ranges from 1 to 0. Specifically, it is as the following formula.
The time difference total_diff of the HSV color space cumulative histogram is expressed by the following equation, for example.
For example, the distance coefficient is expressed by the following equation.
Therefore, the closer the distance between the center of the search block and the center of gravity of the object area is, the higher the fitness of the search block is. For the disappearing object (LOST object), the distance coefficient is obtained by using the center of gravity position at the frame time immediately before the LOST state as the center of gravity of the object.

  FIG. 31 illustrates the relationship between the distance between the center of the search block and the center of gravity of the object to be tracked, and the fitness. Since the search blocks D1 and D2 are not on the tracking target object, the distance from the object center of gravity is large (distance factor (dist_factor) → small), and the degree of coincidence of the object (image) feature information (HSV color space cumulative histogram) ) Is low (total_diff → small), so the fitness is low (approaching 0). Since the search blocks D3 and D4 are on the object to be tracked, the distance from the object center of gravity is small (distance factor (dist_factor) → large), and the degree of coincidence of object (image) feature information (HSV color space cumulative histogram) Since the degree of matching is also high (total_diff → large), the degree of matching is high (close to 1).

  Next, divided block region ID assignment processing is performed (FIG. 25: Step S124, details are shown in FIG. 32). In this process, the divided block Bk in which some object exists in the divided block area and one or more search blocks exist has the highest priority (having the highest fitness) among those search blocks. By assigning the tracking target object ID registered in the search block, the object ID is assigned to the divided block Bk, and the object of the object ID is set to the ACTIVE state. The object that was originally in the ACTIVE state continues to be in the ACTIVE state.

  When the object ID registered in the search block whose tracking target is the LOST object is assigned to any one of the divided blocks Bk by the above processing, the tracking target object in the LOST state is found (detection) Was). Note that an object that is a tracking target object but has not been assigned an object ID to any of the divided blocks Bk is determined to be a LOST object in the object state update process (S110) of FIG. 22, and the state is maintained or updated. .

  Next, search block erasure processing is performed (FIG. 25: Step S125, details are shown in FIG. 33). If the object state is UNDETECTED, or if the state is LOST and the object duration obj_time is longer than LOST_TIME (LOST state lasts longer than LOST_TIME), it belongs to the object (track the object) Delete all search blocks. Deletion is performed by clearing all search block data tables belonging to the object.

  Next, search block resampling processing is performed (FIG. 25: Step S128, details are shown in FIG. 34). The search block resampling process is performed for all search blocks belonging to the object in the ACTIVE state (FIG. 25; Step S127; Yes). In the search block resampling process, a search block with a low fitness is replaced with a search block with a high fitness between a plurality of search blocks with the same tracking target object.

  Here, a roulette is generated in which the ratio of the fitness of each search block to the sum of the fitness of all search blocks that track the same object (for example, object A) is the appearance probability of the search block, A random number lottery based on the roulette is performed for each search block whose object is to be tracked, and the search block is replaced with a search block corresponding to the result of the lottery result. That is, the randomness lottery is performed the same number of times as the number of all search blocks for which the object is tracked based on the roulette, and replaced by the search block corresponding to the winning point for each lottery, The search block having a low match is easily replaced with a search block having a high fitness.

More specifically, the following processing is performed for each object whose state (status) is ACTIVE among the objects existing in the processing target image. First, for the ACTIVE object obj_id of interest, a roulette is generated as follows based on the fitness cor ^obj_id _{n_block} of the search block n_block belonging to the object.

Search block n_block state (status) If state ^obj_id _{n_block} is ON,
roulette ^obj_id _{n_block} = roulette ^obj_id _{n_block-1} + cor ^obj_id _{n_block}
Search block n_block state (status) If state ^obj_id _{n_block} is OFF,
roulette ^obj_id _{n_block} = roulette ^obj_id _{n_block-1}
Next, after generating random number data for each search block belonging to the active object obj_id of interest, and determining the roulette output for the search block based on the random number data, Replace with the corresponding search block.

  FIG. 35 shows an example of a roulette generated for an arbitrary object. There are a total of M search blocks that use this object as a tracking target. On the roulette where the total fitness of all search blocks from search blocks 1 to M is 360 degrees, the output of each search block is arranged at an occupation angle corresponding to the ratio of the fitness. For example, the occupancy angle of the search block 1 is ((the suitability of search block 1) / (the sum of the suitability of all search blocks)) × 360 degrees.

  Since the search block with a high degree of matching has a large occupancy angle, the probability of winning in the random number lottery is high. On the other hand, a search block with a low degree of fitness has a small occupancy angle and a low probability of winning. Therefore, a search block with a low fitness is easily replaced with a search block with a high fitness.

  Here, to replace the search block is to update the contents of the search block data table 70 with the search block data table corresponding to the roulette for items other than the search block ID.

  As described above, the suitability of a search block is high when the search block exists on the tracking target object, and is low when the search block is located away from the tracking target object. Therefore, as shown in FIG. 36A, when search block replacement (search block resampling) is performed for the ACTIVE object, search blocks D5 to D7 on the tracking target object A and search blocks not on the object A There is a large difference in fitness between D1 and D4. Therefore, the roulette generated by the search block resampling is as shown in FIG. 36B (the occupied angle is small for D1 to D4 and the occupied angle is large for D5 to D7).

  When random number lottery is performed for the number of search blocks (seven times here) using this roulette, the result is as shown in FIG. 36 (c), for example. Probabilistically replaced with a high search block. As shown in FIG. 36D, many of the search blocks that were not on the tracking target object are replaced with the search blocks on the tracking target object, and the search blocks are easily aggregated on the tracking target object.

  Next, search block update processing is performed (FIG. 25: Step S129, details are shown in FIG. 37). In the search block update process, the movement of each search block and the update of the feature information regarding the tracking target object stored in association with each search block are performed. The position information and motion vector information are updated for all search blocks present in the image (status is ON) (steps S248 and S249). Further, a search block belonging to an object whose status (status) is ACTIVE (with the object as a tracking target), and an object assigned to the divided block of the tracking target object ID and the position where the search block exists For the search block whose ID matches, the HSV color space cumulative histogram is also updated (step S247).

The location information is updated as follows.
Here, rnd_x and rnd_y indicate random numbers in the x and y directions, respectively. By adding a random number, a random motion is added to the search block.

The update of the motion vector information is as follows, and is a linear load sum of the motion vector of the search block at the previous time and the motion vector in the divided block at the current search block location.
Here, k represents an update rate (inertia constant). The value of k can be changed. If tracking has succeeded, the motion vector of the object becomes dominant, and if tracking has failed (the tracking target object has disappeared (LOST state)), movement by random numbers becomes dominant. .

  By performing object tracking / searching as described above, even if the tracking target object is temporarily hidden behind the obstacle 61 as shown in FIG. That is, the same object ID can be assigned to the same object before and after being hidden by the obstacle 61 for tracking.

  Similarly, even when tracking target objects intersect, tracking can be performed by assigning the same object ID to the same object before and after the intersection (FIGS. 38A to 38D). In the intersection area, a search block having a high degree of matching has an advantage. In FIG. 38B, since the object A passes behind the object B, the search block that tracks the object B on the front side is superior. As shown in FIG. 5C, when the objects A and B are separated from each other from the intersection state, the search block belonging to the object A follows the object A, and the search block belonging to the object B follows the object B. (Fig. 4D).

  As described above, in the object tracking / search method according to the present embodiment, when a new object to be tracked is detected, a plurality of search blocks are generated near the center of gravity of the object, and each search block has its search block. Is stored in association with the feature information of the tracking target object in the existing region, and after that, the search block is moved so as to follow the tracking target object and the feature information of the tracking target object stored in the search block is stored. After updating and tracking has failed (object disappears), the search block continues to move so that random movement becomes dominant, and the disappearance is determined by determining the fitness based on the local area called the search block. By searching for the object to be tracked, the same object is detected before and after crossing or hiding. Tracking as-object.

  It should be noted that the feature amount data of the object such as shape and movement extracted from the tracked object in step S106 in FIG. 22 is stored in the storage area F in the FIFO (First in First Out) format as shown in FIG. It is stored as series data (FIG. 22, step S107). The stored feature amount time-series data is used when determining whether each tracking target object is a person or a vehicle in the person / vehicle determination process (step S111) of FIG.

  Next, the person / vehicle discrimination process (S111 in FIG. 22) will be described in detail. FIG. 39 shows the flow of the person / vehicle discrimination process. The person / vehicle discrimination process is performed for each object to be tracked. First, it is confirmed whether or not the state of the object to be tracked is ACTIVE (tracking is in progress). If the object is not ACTIVE (step S301; No), the person / vehicle discrimination process for the object to be tracked is terminated.

When the state of the tracking target object is ACTIVE (step S301; Yes), the time length (L time length) in which the feature amount data related to the tracking target object is stored in the storage area F in time series is set. Then, it is confirmed whether or not a predetermined time length ( _Tc time length) or longer (step S302). The _Tc time length can be set to an arbitrary time length equal to or less than the Lmax time length that can be stored in the storage unit 22.

If the L time length is less than the _Tc time length (step S302; No), the person / vehicle discrimination process for the tracking target object is terminated.

If the L time length is equal to or longer than the _Tc time length (step S302; Yes), the stored feature amount time series data for the L time length is input to the person / vehicle determination unit 24 (step S303). The vehicle discrimination unit 24 discriminates people / vehicles (step S304).

In this way, the object is tracked, and the feature amount data indicating the feature information of the object is sampled and accumulated and stored in a time series for a certain time length during the tracking, over a predetermined period ( _Tc time length or more). ) Since it is determined whether the object is a person or a vehicle based on the feature amount time-series data, the person / vehicle can be detected with higher reliability compared to the case of determining the person / vehicle immediately after detecting the object. Can be determined. That is, since a person has a high degree of freedom of movement and the “appearance” varies, an error is likely to occur in a determination based only on an image at a certain time, but the suspicious behavior detection device 10 of the present embodiment has a predetermined period. Since the determination is made based on the feature data for a plurality of times collected in time series during the object tracking, it is possible to determine the person / vehicle with high reliability.

  In addition, since the same object can be tracked as the same object even if there is a crossing or hiding, even if there is a crossing or hiding during tracking, the time series feature amount data of the same object can be collected.

Further, since the type (person / vehicle) of the tracking target object is not discriminated when the feature amount data is less than the predetermined time length ( _Tc time length), it may be erroneously discriminated by the feature amount data having a short sampling time length. It is prevented and the reliability of discrimination is ensured.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  For example, the basic operation is not limited to that exemplified in the embodiment. Any operation may be used as long as it is a basic action of a person's behavior in the monitoring target area, and may be appropriately selected according to the monitoring target and purpose.

  Suspicious behavior is behavior that deviates from daily behavior, or a predetermined type of behavior (for example, vandalizing a vehicle, stealing a vehicle, damaging a vehicle, etc. in a parking lot). Actually, the behavior pattern registered in the daily behavior pattern related reliability time series DB 15d1 is a daily behavior, and the behavior pattern registered in the daily behavior or the suspicious behavior pattern related reliability time series DB 15d2 is This is a suspicious behavior to be detected.

  The daily behavior pattern related reliability time series DB 15d1 and the suspicious behavior pattern related reliability time series DB 15d2 may have a configuration in which stored contents are sequentially updated or additional data is stored during the monitoring operation. The data may be stored. Further, the process of deriving the representative daily behavior pattern and the representative suspicious behavior pattern by a statistical method may be performed during the monitoring operation, but the daily behavior pattern related reliability time series DB 15d1 or the suspicious behavior pattern related reliability It may be configured to be derived and stored based on the stored contents of the series DB 15d2, and to use the stored representative daily action pattern or representative suspicious action pattern during the monitoring operation.

  The feature amount time-series data is not limited to the items exemplified in the embodiment such as the shape and size, and may be any data as long as it indicates the type of object to be determined and the feature amount of various human movements. Can be set.

  In the embodiment, as the object tracking method, a method of tracking / searching a lost object using a search block is exemplified. However, the present invention is not limited to this, and tracking may be performed by any method. A method capable of tracking the same object as the same object before and after crossing or hiding is preferable, but a tracking method without such a function may be used.

  In the tracking / search shown in the embodiment, the object is tracked / searched in divided block units, but tracking may be performed in object units. In the embodiment, the search block is moved by synthesizing the movement of the object to be tracked and the random movement (position change based on the random number, Equation 4), and the search block targeted for the search block belonging to the ACTIVE object Combined with resampling (probabilistic replacement of search block with low fitness with search block with high fitness), the search block moves to track the tracking target object while tracking is successful The search block is moved so that the random motion becomes dominant during the disappearance, but the method of moving the search block is not limited to the method exemplified in the embodiment. That is, the tracking target object may be tracked during tracking success, and it may be moved according to the movement rule that random motion becomes dominant during disappearance.For example, according to the tracking target motion vector without random elements during tracking success. It may be configured to move and move only with random movement while eliminating the inertial movement during disappearance.

  As the rule of movement during disappearance, it is preferable to add a random motion to the motion of the object immediately before the disappearance, and it is preferable that the random motion becomes more dominant as time passes.

  In the embodiment, the fitness of each search block is determined based on the time difference of the HSV color space cumulative histogram and the distance coefficient based on the distance between the object and the search block. May be determined. The degree of matching is preferably determined by using the degree of coincidence as the image feature amount and the positional relationship between the object and the search block as elements.

  In the embodiment, as a person / vehicle discrimination method, considering the response to the “appearance” variation of an object, after tracking an object for a predetermined time length (feature amount time-series data having a predetermined time length or more for an object to be tracked) After the accumulation, it is determined whether the object is a person or a vehicle based on the feature time series data. A highly reliable person / vehicle discrimination method is preferable, but such a person / vehicle discrimination method is not necessary.

  In the embodiment, the case where the person and the vehicle are discriminated is described as an example, but the type of the object to be discriminated is not limited to these.

DESCRIPTION OF SYMBOLS 10 ... Suspicious behavior detection apparatus 11 ... Camera part 12 ... Processing part 13 ... Input part 14 ... Output part 15 ... Database part 15a ... Human feature DB part 15b ... Vehicle feature DB part 15c ... Basic motion feature time series DB part 15d ... Action pattern reliability time series DB section 15d1 ... Daily action pattern related reliability time series DB
15d2 ... Suspicious behavior pattern related reliability time series DB
DESCRIPTION OF SYMBOLS 21 ... Tracking part 22 ... Memory | storage part 23 ... Accumulation control part 24 ... Person and vehicle discrimination | determination part 25 ... Basic operation | movement / behavior analysis part 25a ... Reliability deriving part 25b ... Suspiciousness deriving part 25b1 ... Normality deriving process part 25b2 ... Suspiciousness derivation processing unit 25c ... Suspicious judgment unit 26 ... Reliability deriving unit by basic operation 27 ... Discriminator 28 ... Adder 30 ... Object data table 40 ... Decision logic table 61 ... Obstacle 70 ... Search block data table F ... Memory Area P1 ... Object detection phase P2 ... Object tracking / search phase P3 ... Human / vehicle discrimination phase P4 ... Human object extraction phase

Claims (14)

The processing unit tracks a person in the image input from the camera and, during the tracking, a plurality of types of the basic degrees of reliability indicating the degree of basic motion likelihood determined in advance by the motion of the tracking person. Repetitively deriving for each of the actions, and determining whether or not the action of the person is a suspicious action based on the basic action reliability time-series data in which the reliability for each basic action is time-series. Suspicious behavior detection method. The basic operation reliability time series data has a database registered for a plurality of people,
Whether the action of the person is suspicious by a statistical method based on the basic action reliability time-series data related to the person to be determined and the basic action reliability time-series data registered in the database. The suspicious behavior detection method according to claim 1, further comprising: determining whether or not. The suspicious behavior detection method according to claim 2, wherein the database is obtained by registering the basic motion reliability time series data regarding normal behavior. The processing unit derives a degree of deviation of the basic motion reliability time-series data related to the determination target person from the basic motion reliability time-series data related to normal behavior registered in the database by a statistical method, The suspicious behavior detection method according to claim 3, wherein whether or not the behavior of the person is suspicious behavior is determined based on a degree of detachment. The suspicious behavior detection method according to claim 2, wherein the database registers the basic operation reliability time-series data regarding suspicious behavior. The database includes a first database in which the basic operation reliability time series data related to normal behavior is registered, and a second database in which the basic operation reliability time series data related to suspicious behavior is registered,
The processing unit indicates a degree to which the action of the person is a normal action based on the basic action reliability time series data regarding the person to be determined and the basic action reliability time series data registered in the first database. Degree of normality, and the degree to which the action of the person is suspicious based on the basic action reliability time series data related to the person to be determined and the basic action reliability time series data registered in the second database The suspicious behavior detection method according to claim 2, wherein a suspicious degree indicating suspicious behavior is derived, and whether or not the behavior of the person is suspicious behavior is determined based on the normality and the suspicious degree. If the normality is less than a first threshold value, or if at least one of the suspicious degree is greater than or equal to a second threshold value is true, it is determined that the action of the person is a suspicious action. The suspicious behavior detection method according to claim 6. A tracking unit for tracking a person in an image input from the camera;
A derivation unit that repeatedly derives the reliability indicating the degree of the basic behavior that the person's motion during the tracking is determined in advance for each of a plurality of types of the basic operations during the tracking by the tracking unit;
A storage unit for storing basic operation reliability time-series data in which the reliability of each basic operation derived by the derivation unit is time-series;
A suspicious determination unit that determines whether or not the action of the person is a suspicious action based on the basic operation reliability time-series data stored in the storage unit;
A suspicious behavior detection device characterized by comprising: The basic operation reliability time series data has a database registered for a plurality of people,
The suspicious determination unit is a statistical method based on basic operation reliability time-series data related to a determination target person stored in the storage unit and basic operation reliability time-series data registered in the database. The suspicious behavior detection device according to claim 8, wherein it is determined whether or not the human behavior is suspicious behavior. The suspicious behavior detection device according to claim 9, wherein the database registers the basic operation reliability time-series data regarding normal behavior. The suspicious determination unit derives a degree of deviation of the basic operation reliability time-series data related to the person to be determined from the basic operation reliability time-series data related to normal behavior registered in the database by a statistical method, The suspicious behavior detection apparatus according to claim 10, wherein whether or not the behavior of the person is suspicious behavior is determined based on the degree of detachment. The suspicious behavior detection device according to claim 9, wherein the database is registered with the basic operation reliability time series data regarding suspicious behavior. The database includes a first database in which the basic operation reliability time series data related to normal behavior is registered, and a second database in which the basic operation reliability time series data related to suspicious behavior is registered,
The suspicious determination unit is configured to store the person based on the basic operation reliability time-series data regarding the determination target person stored in the storage unit and the basic operation reliability time-series data registered in the first database. Normality indicating the degree of behavior being normal behavior, basic motion reliability time-series data related to the determination target person stored in the storage unit, and basic motion reliability time-series registered in the second database Deriving a suspicious degree indicating a degree of suspicious behavior of the person based on the data and determining whether the behavior of the person is suspicious based on the normality and the suspicious degree. 10. The suspicious behavior detection device according to claim 9. The suspicious determination unit determines that the action of the person is a suspicious action if at least one of the normality is less than a first threshold value or the suspicious degree is a second threshold value or more is true. The suspicious behavior detection device according to claim 13, wherein the suspicious behavior detection device is determined.