JP6113018B2 - Object detection device - Google Patents

Description

  The present invention relates to a target detection apparatus that detects a detection target from input data.

  2. Description of the Related Art Conventionally, techniques for detecting a detection target such as a human body from data such as images, sound, or sensor signals have been studied for various purposes such as crime prevention, photography or video shooting search or organization. In particular, in recent years, techniques for detecting a detection target from data using a classifier generated by machine learning have been widely studied. For example, a discriminator that determines whether or not a human body is shown in an image uses features extracted from each of a large number of learning images in which the human body is reflected and a large number of learning images in which the human body is not captured. It is generated by machine learning of an identification boundary that divides a space in which a human body feature amount is distributed from a space other than the space in which the human body features are distributed. When a feature amount extracted from an image is input, the discriminator determines whether or not a human body is reflected in the image depending on which side of the feature amount the feature amount is located on the identification boundary.

  However, there are a variety of images showing the human body and images not showing the human body, and it is generally difficult to find an identification boundary that can completely separate them. In particular, the feature amount extracted from an image showing something similar to the human body, such as a jacket on a hanger, is likely to be located on the space side where the human body feature amount is distributed with respect to the identification boundary. If the human body is reflected in the image, there is a risk of erroneous determination.

  Thus, for example, Patent Document 1 proposes an object detection device that learns a discriminator in accordance with the characteristics of the installation environment of the device. This object detection device learns a discriminator using an image from a surveillance camera when the surveillance camera is installed.

JP 2009-230284 A

  The object detection apparatus described in Patent Literature 1 is a detection target like a jacket hung on a hanger existing at an installation location by learning a discriminator using an image from the monitoring camera after the monitoring camera is installed. It is possible to reduce erroneous determination of an object similar to a human body as an example of a human body. However, when an object similar to the detection target is newly arranged after learning the classifier, the object detection device described in Patent Document 1 uses the image in which the object is reflected. Since the learning is not performed, the object may be erroneously determined as a detection target.

  The object detection apparatus can improve the detection accuracy of the detection target by performing machine learning using many images. However, there are a variety of images in which objects other than the detection target are shown, and it is impossible to predict what image will be input. Therefore, in the technology for determining whether or not the detection target is reflected in the image based on which side of the identification boundary in the feature amount space the feature amount extracted from the input image is located, erroneous determination is made for all images. It is difficult to train a classifier to completely prevent it.

  The problem that causes such misjudgment is a problem that is common not only when a detection target is detected using a discriminator, but also when a detection target is detected by another method such as a pattern matching method. This is a common problem not only when detecting a detection target from an image but also when detecting a detection target from various data such as sound and sensor signals.

  An object of the present invention is to provide an object detection apparatus that can improve the accuracy of detecting a detection object from input data.

  This invention for solving this subject provides the target detection apparatus which determines whether a detection target is contained in the input data acquired from the data input part. The target detection apparatus according to the present invention has a multi-level data generation unit that generates change data with different information levels representing the detection target from input data, and the degree of likelihood of detection for each of the change data. An evaluation value calculation unit that calculates an evaluation value to be represented, an evaluation value sequence generation unit that generates an evaluation value sequence by arranging the evaluation values in a preset order prescribed by the information level, and the evaluation value sequence includes a detection target An object determination unit that determines whether or not a detection target is included in the input data based on identification information for identifying whether or not the data is generated from the input data.

  In addition, the multi-level data generation unit generates first change data in which the detection target is expressed from the input data by the first change process, and inputs the second change process different from the first change process. It is preferable to generate second change data having different degrees of expressing the detection target from the data, and use the first change data and the second change data as the change data.

  The evaluation value series generation unit preferably generates an evaluation value series by smoothing an original series in which evaluation values are arranged in a preset order defined by an information level.

  The object detection device according to the present invention has an effect of improving the accuracy of detecting a detection object from input data.

It is a graph showing an example of the relationship between an information level and an evaluation value. It is a graph showing an example of the relationship between an information level and an evaluation value. It is a graph showing an example of the relationship between an information level and an evaluation value. It is a graph showing an example of the relationship between an information level and an evaluation value. 1 is a schematic configuration diagram of a monitoring system according to a first embodiment. It is a schematic block diagram of the control part of the monitoring apparatus by 1st Embodiment. It is a flowchart which shows operation | movement of the target detection process of the monitoring apparatus by 1st Embodiment. It is a schematic diagram for demonstrating a masking area | region. It is a graph showing an example of the evaluation value series which arranged the evaluation value calculated for every some change data in the predetermined order. It is a graph showing an example of the evaluation value series which arranged the evaluation value calculated for every some change data in the predetermined order. It is a flowchart which shows operation | movement of the target detection process of the monitoring apparatus by 2nd Embodiment. It is a graph for demonstrating the evaluation value series which smoothed the evaluation value. It is a graph for demonstrating the evaluation value series which smoothed the evaluation value.

Hereinafter, an object detection apparatus according to an embodiment of the present invention will be described with reference to the drawings.
The target detection device obtains a characteristic amount specific to the detection target from input data that is a target of detection processing, and inputs the calculated feature amount into a discriminator generated by machine learning to obtain an evaluation value that represents the degree of uniqueness of the detection target. It is calculated, and it is determined whether or not a detection target is included in the input data from the calculated evaluation value. The inventor has conducted intensive studies, generates a plurality of data with different information levels from one input data, calculates an evaluation value from each of the plurality of data, and if the detection target is included in the input data, the information level We obtained the knowledge that the aspect of the change in evaluation value due to the change in value has a common feature. Furthermore, the inventor has found that the input data including the detection target and the input data not including the detection target are greatly different in the aspect of the evaluation value change due to the change in the information level. The information level of the data is such that the data expresses the characteristics of the detection target, the level of detail that the data expresses the characteristics of the detection target (the level that can be expressed), or the data that represents the characteristics of the detection target It is a grade suitable for.

  FIG. 1A, FIG. 1B, FIG. 2A, and FIG. 2B show an example of the relationship between the information level of data and the evaluation value that represents the degree of likelihood of detection of data. In FIGS. 1A, 1B, 2A, and 2B, images 101, 111, 201, and 211 represent input data, and graphs 100, 110, 200, and 210 represent information levels and evaluation values for each input data. Represents a relationship. In the graphs 100, 110, 200, and 210, the horizontal axis is the information level, and the vertical axis is the evaluation value. The lower the information level, the lower the detail that the data expresses the feature of the detection target, and the higher the information level, the higher the detail that the data expresses the feature of the detection target. The data with the highest information level (33) is the original input data. When the evaluation value is 0, the feature quantity for the detection target obtained from the input data is located on the identification boundary in the feature quantity space. When the evaluation value is positive, the feature quantity is compared to the identification boundary in the feature quantity space. If the evaluation value is a negative value, the feature value is located on the non-detection side with respect to the identification boundary in the feature value space. The evaluation value indicates that the higher the value, the more likely it is to be detected, and the lower the value, the less likely the detection target.

  In FIGS. 1A, 1B, 2A, and 2B, the detection target is a human body. FIG. 1A shows an example in which a detection target is included in input data and the evaluation value for the original input data is a positive value. FIG. 1B shows an example in which a detection target is included in the input data and the evaluation value for the original input data is a negative value. FIG. 2A shows an example in which a detection target is not included in the input data, and the evaluation value for the original input data is a negative value. FIG. 2B shows an example in which a detection target is not included in the input data and the evaluation value for the original input data is a positive value.

  As shown in the graph 100 of FIG. 1A, when the detection target is included in the input data, the evaluation value does not change regardless of the information level change in the low information level region (1 to 18), and the information level is medium. In the area (18 to 25), the evaluation value tends to decrease as the information level increases, and in the area (25 to 32) where the information level is high, the evaluation value tends to increase rapidly as the information level increases. In the example shown in the graph 110 of FIG. 1B, the evaluation value does not change in the low information level region (1 to 15), and the evaluation value decreases in the medium information level region (15 to 23). In the high level region (23 to 32), the evaluation value tends to increase rapidly. As described above, when the detection target is included in the input data, the above-described tendency is seen regardless of the height of the evaluation value itself. That is, the above-mentioned tendency is similarly seen whether the feature amount for the detection target obtained from the input data is located on the detection target side or the non-detection side with respect to the identification boundary in the feature amount space. .

  On the other hand, as shown in the graph 200 of FIG. 2A and the graph 210 of FIG. 2B, when the detection target is not included in the input data, the evaluation value irregularly changes with the change in the information level, and the detection target is detected in the input data. There is no trend seen when included. As described above, the tendency seen when the detection target is included in the input data is not seen regardless of the height of the evaluation value itself when the detection target is not included in the input data. That is, the tendency seen when the detection target is included in the input data is that the feature amount of the detection target obtained from the input data is compared with the identification boundary in the feature amount space when the detection target is not included in the input data. Neither the case where it is located on the detection target side nor the case where it is located on the non-detection side is seen similarly.

  Therefore, the target detection apparatus according to an embodiment of the present invention generates a plurality of data having different information levels from one input data, and obtains a feature amount for the detection target for each generated data. The target detection device evaluates the obtained feature quantity by inputting it to a classifier that has been learned in advance using the feature quantity obtained from the input data including the detection target and the feature quantity obtained from the input data not including the detection target. Each value is calculated. Then, the target detection device generates an evaluation value series in which the calculated evaluation values are arranged in an order set in advance with respect to the information level (for example, ascending order or descending order). Furthermore, the target detection device has previously learned the generated evaluation value series using the evaluation value series generated for the input data including the detection target and the evaluation value series generated for the input data not included. It is input to the discriminator, and it is determined whether or not the detection target is included in the input data. That is, the target detection device is not based on which side the feature quantity for the detection target is located in the feature quantity space with respect to the identification boundary, but by a change in the position of the feature quantity with respect to the identification boundary when the information level is changed, It is determined whether or not a detection target is included in the input data. As a result, the target detection device improves the accuracy of detecting the detection target from the input data.

Hereinafter, a monitoring system in which an object detection apparatus according to a first embodiment of the present invention is mounted will be described with reference to the drawings.
The monitoring system according to the present embodiment detects an intruder who has entered the monitoring area and issues an alarm. This monitoring system has a monitoring device and a center device. This monitoring device determines whether or not a human body is reflected in an image obtained by photographing the monitoring area. If the monitoring device determines that a human body is captured, the monitoring device determines that a human body has entered the monitoring area and issues an alarm to the center device. That is, the monitoring system according to this embodiment uses input data as image data and a detection target as a human body.

  FIG. 3 is a diagram showing a schematic configuration of the monitoring system according to the present embodiment. As shown in FIG. 3, the monitoring system includes one or more monitoring devices 10 and a center device 50 connected to the monitoring devices 10 through a public communication line. When the monitoring device 10 detects an intruder in the monitoring area, the monitoring device 10 transmits an abnormal signal indicating that the intruder has been detected to the center device 50 connected via the public communication line. The monitoring device 10 includes an imaging unit 11, an interface unit 12, a communication unit 13, a storage unit 14, and a control unit 15. Hereinafter, each part of the monitoring apparatus 10 will be described in detail.

The imaging unit 11 is a camera that captures a monitoring region at a predetermined cycle (for example, 200 ms), and is, for example, a photoelectric conversion element (for example, a CCD sensor, a two-dimensional array that outputs an electrical signal corresponding to the amount of received light). And an imaging optical system for forming an image of the monitoring region on the photoelectric conversion element. The imaging unit 11 is connected to the interface unit 12 and sequentially transfers captured images to the interface unit 12.
The captured image can be a grayscale or color multi-tone image. In this embodiment, the captured image is a grayscale image having 320 pixels wide × 240 pixels vertically and having 8-bit luminance resolution. However, a photographed image having a resolution and gradation other than this embodiment may be used.

  The interface unit 12 includes an interface circuit connected to the imaging unit 11, for example, an interface circuit conforming to a serial bus such as a video interface or a universal serial bus. The interface unit 12 is connected to the control unit 15 via, for example, a bus, and sends the captured image received from the imaging unit 11 to the control unit 15.

  The communication unit 13 includes a communication interface that connects the monitoring device 10 to a public communication line and a control circuit thereof, and is connected to the control unit 15 via, for example, a bus. When notifying that an intruder has been detected in the monitoring area, the communication unit 13 performs a connection process between the monitoring device 10 and the center device 50 according to the control of the control unit 15. Then, the communication unit 13 transmits the abnormality signal received from the control unit 15 to the center device 50 after the connection is established between the monitoring device 10 and the center device 50. When the transmission of the abnormal signal ends, the communication unit 13 performs a process of releasing the connection between the monitoring device 10 and the center device 50.

  The storage unit 14 includes a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 14 stores a computer program and various data for controlling the monitoring device 10, and inputs and outputs these pieces of information to and from the control unit 15. The computer program may be installed in the storage unit 14 from a computer-readable storage medium such as a CD-ROM (Compact Disk Read Only Memory) or a DVD-ROM (Digital Versatile Disk Read Only Memory). The various data includes human body reference data and evaluation value series reference data.

  The control unit 15 is an example of a target detection apparatus, and includes at least one processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an MCU (Micro Control Unit), and its peripheral circuits. The control unit 15 stores the captured image received from the interface unit 12 in the storage unit 14. And the control part 15 reads the picked-up image memorize | stored in the memory | storage part 14, determines whether the human body is reflected in the picked-up image, and if it determines with the human body being reflected, if it determines that the human body is reflected, the center apparatus will be via the communication part 13. Alert 50.

FIG. 4 is a diagram illustrating a schematic configuration of the control unit 15. As illustrated in FIG. 4, the control unit 15 includes a data input unit 150, a cutout unit 151, a multilevel data generation unit 152, an evaluation value calculation unit 153, an evaluation value series, as functional modules implemented by software operating on the processor. A generation unit 154, a target determination unit 155, and a notification control unit 156 are included.
Note that these units included in the control unit 15 may be configured by independent integrated circuits, firmware, a microprocessor, and the like.
Hereinafter, each part of the control part 15 is demonstrated in detail.

  Each time a captured image is stored in the storage unit 14, the data input unit 150 reads out the captured image from the storage unit 14 and sends it to the cutout unit 151.

  Each time the cutout unit 151 receives a captured image from the data input unit 150, the cutout unit 151 sequentially cuts out an image of a predetermined size that is a target of detection processing from the captured image received from the data input unit 150, and the cutout image is a multilevel data generation unit. 152. The cutout unit 151 sets a plurality of cutout positions in the photographed image, and cuts out an image within the range of the human body size on the photographed image that is assumed from the installation state of the image pickup unit 11 and the individual difference of the human body entering the monitoring area. A plurality of sizes are set, and partial images corresponding to the set position and size combinations are sequentially cut out. For example, the size of the image to be cut out can be set to 64 pixels in the horizontal direction × 128 pixels in the vertical direction. The cutout position is a position shifted by half of the horizontal length of the image cut out in the horizontal direction from the upper left corner of the photographed image, and the vertical length of the image cut out further vertically from those positions. It can be set to a position shifted by half. For example, when the shooting area covers a wide range, the size of the image to be cut out may be changed according to the cutout position. In that case, the size of the clipped image is increased as the cutout position is located below the captured image in which the region close to the image capturing unit 11 is captured, and the image is clipped above the captured image in which the region distant from the image capturing unit 11 is captured. The size of the image to be cut out can be reduced as the position is located. Further, if there is a margin for processing time, it may be cut out by shifting one pixel each in the horizontal direction and the vertical direction, and the shift width can be determined as appropriate. Hereinafter, the image cut out by the cutout unit 151 is referred to as a partial image. In this embodiment, a partial image is an example of input data.

The multilevel data generation unit 152 generates a plurality of pieces of change data having different information levels from one input data, and sends the generated change data to the evaluation value calculation unit 153 in association with the information level. Hereinafter, a plurality of change data having different information levels are referred to as multi-level data. The multi-level data generation unit 152 according to the present embodiment generates a plurality of images (hereinafter referred to as individual level images) having different information levels from partial images cut out from a captured image that is input data. The partial image and the generated individual level image are generated as multilevel data.
In the present embodiment, an example will be described in which the information level of data is the level of detail (the level that can be expressed) that represents the characteristics of the detection target. The multilevel data generation unit 152 according to the present embodiment generates, as multilevel data, a plurality of images having higher sharpness as the information level is higher, lower sharpness as the information level is lower, and different sharpness from each other. The multi-level data generation unit 152 generates an individual level image with reduced sharpness by performing an averaging process on the partial image. For example, the multi-level data generation unit 152 applies a plurality of averaging filters having different sharpness levels to the partial image by applying averaging filters having smaller filter sizes as the information level is higher and larger filter sizes as the information level is lower. Generate individual level images. When the filter size of the averaging filter is (4 (n-1) +1) (for example, n is an integer of 2 to 33), each pixel of the individual level image is centered on the corresponding pixel in the partial image (4 An average value of the pixel values in the range of (n−1) +1) pixels × (4 (n−1) +1) pixels is set. In other words, the larger the filter size, the narrower the passband of the averaging filter becomes, and the individual level image becomes a blurred image and the sharpness decreases, so that the individual level image can express the characteristics of the human body. The degree is lower and the information level is lower. For example, the information level is defined in 33 levels from 1 to 33. When the information level is 1, the filter size is set to 129 (n = 33), and every time the information level increases by 1, the filter size is set to 4 smaller values (n is smaller by 1). When the information level is 33, which is the highest value, the original partial image to which the averaging filter is not applied is used.

The evaluation value calculation unit 153 calculates an evaluation value representing the degree of likelihood of detection for each change data included in the multilevel data, and sends each evaluation value to the evaluation value series generation unit 154 in association with the information level. . The evaluation value calculation unit 153 according to the present embodiment performs Real-Adaboost identification using HOG (Histograms of Oriented Gradients) feature quantities as human body feature quantities that are one or more feature quantities useful for identifying a human body. Has a vessel. The evaluation value calculation unit divides the input image into a plurality of blocks, and further divides each block into a plurality of cells. For example, each cell is a rectangular area of 5 pixels × 5 pixels, and each block is a rectangular area of 3 cells × 3 cells. Then, the evaluation value calculation unit calculates the gradient direction and gradient strength of the pixel value at each pixel in the input image. In addition, since it is not necessary to consider the direction of the gradient direction, the gradient direction is calculated in a range of 0 ° to 180 °, and is set to, for example, 8 directions divided every 22.5 °. For each cell, the evaluation value calculation unit obtains a histogram obtained by obtaining a histogram with the sum of the gradient intensities in each gradient direction as the frequency for each gradient direction, and sets the normalized value for each block as the HOG feature amount.
The real Adaboost classifier includes a plurality of weak classifiers and a strong classifier that is determined by integrating the determination results of the weak classifiers. For each weak classifier, a feature quantity is calculated from the image using a Haar-like feature quantity predetermined for each weak classifier. Each weak classifier outputs a higher value as the possibility that a human body is reflected in the corresponding partial image is higher, and a lower value as the possibility is lower. On the other hand, the strong classifier outputs the sum of the output values from the weak classifiers as an evaluation value. In addition, this evaluation value takes the value of at least 3 values or more, for example, is a continuous value.

なお、この結合度Zが小さいほど学習用人体画像の分布と学習用非人体画像の分布が分離していることを意味する。そのため、コンピュータは、評価値Zが最小となるブロックに関するＨＯＧ特徴量を一つの弱識別器として選択する。弱識別器の出力h(x)は、以下の式で表される。

ここで、xは弱識別器に入力するＨＯＧ特徴量の値であり、εは分母が0となることを防ぐための定数（例えば10^-10）である。式（２）に表されるように、弱識別器は、入力値（ＨＯＧ特徴量）に対応する、学習用人体画像の確率密度分布W₊ ^jが大きいほど大きい値を出力し、入力値に対応する、学習用非人体画像の確率密度分布W_- ^jが大きいほど小さい値を出力する。そして、弱識別器は、学習用人体画像の確率密度分布W₊ ^jが学習用非人体画像の確率密度分布W_- ^jより大きいとき正の値を出力し、小さいとき負の値を出力し、同一であるとき0を出力する。
（５）コンピュータは、選択したＨＯＧ特徴量を用いた弱識別器が識別に失敗した学習用画像の重みを大きくし、識別に成功した学習用画像の重みを小さくする。そして、全ての学習用画像の重みの和が１となるよう学習用画像の重みを正規化する。
（６）コンピュータは、（３）〜（５）の手順を所定回数繰り返す。
このようにして決定された各弱識別器として用いるＨＯＧ特徴量の情報とブロックを表す情報と、各弱識別器の出力関数を表す情報とは、人体の参照データとして記憶部１４に記憶される。 It should be noted that the HOG feature value for each adjacent rectangular region to be used as each weak classifier depends on a plurality of learning human body images in which a human body is photographed and a plurality of learning non-human body images in which a human body is not photographed (hereinafter referred to as learning The human body image and the learning non-human body image are collectively referred to as a learning image). The outline of the learning procedure is as follows.
(1) The computer that performs the pre-learning sets a plurality of blocks in the image area for each learning image, and calculates the HOG feature amount for each set block.
(2) The computer determines a weight for each learning image. The initial value of the weight is the same value for each learning image.
(3) For each set block, the computer calculates the probability density distribution W ₊ ^j of the learning human body image from the weights set for the learning human body image for each HOG feature amount related to the block. Note that j is a number corresponding to the value of the HOG feature value. Similarly, the computer, for each block set, for each value of the HOG features for that block, the probability density distribution W of the non-human body image for learning from weight set for the non-human body image for learning _- to calculate the ^j . The computer may quantize each HOG feature value by dividing the range of values that can be taken into a plurality of values, and calculate the probability density distribution W ₊ ^j and the probability density distribution W ₋ ^j .
(4) For each HOG feature amount, the computer calculates an evaluation value Z from the following equation using the probability density distribution W ₊ ^j of the learning human body image and the probability density distribution W ₋ ^j of the learning non-human body image.

It is to be noted that the smaller the coupling degree Z is, the more the learning human body image distribution and the learning non-human body image distribution are separated. Therefore, the computer selects the HOG feature amount related to the block having the smallest evaluation value Z as one weak classifier. The output h (x) of the weak classifier is expressed by the following equation.

Here, x is the value of the HOG feature value input to the weak classifier, and ε is a constant (for example, 10 ⁻¹⁰ ) for preventing the denominator from becoming zero. As expressed in Expression (2), the weak classifier outputs a larger value as the probability density distribution W ₊ ^{j of the} human body image for learning corresponding to the input value (HOG feature amount) is larger, and The smaller the probability density distribution W _- ^{j of the} corresponding non-human body image for learning is, the smaller the value is output. Then, the weak classifier, a probability density distribution W ₊ ^j of the learning human body image probability density distribution W of the non-human body image for learning _- outputs Toki ^j greater positive value, and outputs a negative value is smaller, Outputs 0 when they are the same.
(5) The computer increases the weight of the learning image in which the weak classifier using the selected HOG feature amount has failed to be identified, and decreases the weight of the learning image that has been successfully identified. Then, the weights of the learning images are normalized so that the sum of the weights of all the learning images is 1.
(6) The computer repeats the steps (3) to (5) a predetermined number of times.
Information on the HOG feature amount used as each weak classifier determined in this way, information representing a block, and information representing an output function of each weak classifier are stored in the storage unit 14 as reference data of the human body. .

  The evaluation value calculation unit 153 uses the sum of the output values of the weak classifiers selected in (4) as an evaluation value. This evaluation value is a positive value when the HOG feature amount is located in a region on the human body side with respect to the identification boundary in the feature amount space, and a negative value when the HOG feature amount is located in a region on the non-human body side. The absolute value increases as it is farther away, and the absolute value decreases as it is closer.

  Hereinafter, the evaluation value calculation process will be described with reference to FIGS. 1A, 1B, 2A, and 2B. The input data 101, 111, 201, and 211 shown in FIGS. 1A, 1B, 2A, and 2B are examples of partial images. Graphs 100, 110, 200, and 210 shown in FIGS. 1A, 1B, 2A, and 2B are obtained from individual level images generated by applying an information level and an averaging filter corresponding to the information level to a partial image. The example of the relationship of the calculated evaluation value is shown.

  The partial image 101 shown in FIG. 1A is a clear image that the human body is shown, and the partial image 201 shown in FIG. 2A is a clear image that the human body is not shown. In the graph 100 shown in FIG. 1A and the graph 200 shown in FIG. 2A, the evaluation value when the information level is the highest value 33, that is, the sign of the evaluation value for the original partial image to which the averaging filter is not applied is The correct sign is shown. Therefore, with respect to the partial image 101 and the partial image 201, it is possible to correctly determine whether or not a human body is captured even if the evaluation value for the original partial image is used.

  On the other hand, although the human body is shown in the partial image 111 shown in FIG. 1B, other people and complex patterns are shown in the background, and the partial image 111 is an image in which it is not clear that the human body is shown. In addition, although the human body is not shown in the partial image 211 shown in FIG. 2B, the human body and the building 212 that is likely to be mistaken are shown, and the partial image 211 is an image in which it is not clear that the human body is not shown. In the graph 110 shown in FIG. 1B and the graph 210 shown in FIG. 2B, the evaluation value when the information level is the highest value 33, that is, the sign of the evaluation value for the original partial image to which the averaging filter is not applied is The incorrect answer code is shown. Therefore, it is difficult for the partial image 111 and the partial image 211 to correctly determine whether or not a human body is captured from the evaluation value for the original partial image.

On the other hand, in the graph 100 of FIG. 1A, the evaluation value does not change regardless of the change in the information level in the low information level region (1-18), and the information level in the medium information region (18-25). The evaluation value decreases as the information level increases, and the evaluation value increases rapidly as the information level increases in the high information level region (25 to 32). Also, in the graph 110 of FIG. 1B, the evaluation value does not change in the region (1-15) where the information level is low, and the evaluation value decreases in the region (15-23) where the information level is medium, and the information level is high. The evaluation value rises rapidly in the region (23 to 32). On the other hand, such a tendency is not seen in the graph 200 of FIG. 2A and the graph 210 of FIG. 2B.
Therefore, if the control unit 15 determines whether or not the change in the evaluation value according to the information level indicates a tendency seen when the partial image includes a human body, each partial image 101, 111, 201, and 211 is determined. It is possible to correctly determine whether or not a human body is captured.

The evaluation value series generation unit 154 generates an evaluation value series in which evaluation values calculated for each change data included in the multi-level data are arranged in a preset order with respect to the information level, and the generated evaluation value series is targeted It passes to the determination part 155. This order is defined at the information level. For example, when the order is the ascending order of the information level and the information level is 33 stages of 1 to 33, the evaluation value series generation unit 154 uses the evaluation values V _{1 to} V ₃₃ calculated for the individual level images of each information level. A 33-element vector (V ₁ , V ₂ , V ₃ ,..., V ₃₃ ) arranged in ascending order is generated as an evaluation value series. Note that the order may be a descending order of information levels or another order, and the order at the time of prior learning and the order at the time of object detection need only be the same.

  When an evaluation value series is input, the object determination unit 155 outputs an identification information for identifying whether or not the evaluation value series is generated from input data including a detection target (hereinafter, a sequence identifier). Called). The object determination unit 155 determines whether or not a detection target is included in the input data based on the identification information output when the evaluation value sequence received from the evaluation value sequence generation unit 154 is input to the sequence identifier. , Output the judgment result. In the present embodiment, an example will be described in which the sequence discriminator outputs an identification result as to whether or not an evaluation value sequence is generated as identification information for input data including a detection target.

The object determination unit 155 according to the present embodiment uses a support vector machine as a sequence identifier. A computer that performs pre-learning performs pre-learning using an evaluation value sequence generated from a plurality of learning human body images and an evaluation value sequence generated from a plurality of learning non-human body images. When an evaluation value series generated from a specific image is input, the computer calculates a hyperplane identification function for determining whether or not a human body is reflected in the image. The hyperplane discriminant function g (x) is expressed by the following equation.
g (x) = w ^t x + b (3)
Here, x is a vector of evaluation value series input to the sequence discriminator, w is a weight vector, and b is a bias term. The weight vector w is calculated so as to maximize the margin between the human body class and the non-human body class. The hyperplane discriminant function of the present embodiment has a positive value when the input evaluation value series is located in a region on the human body side with respect to the discrimination boundary in the feature amount space, and is located in a region on the non-human body side. The absolute value increases as the position moves away from the identification boundary, and the absolute value decreases as the position is closer to the identification boundary. That is, the output value of this hyperplane identification function represents the probability that the evaluation value series is generated from the input data including the detection target, and the change according to the information level of each evaluation value calculated from the input data is described above. The degree representing the tendency seen when the detection target is included in the input data.

The sequence discriminator compares the output value of the hyperplane discrimination function with a preset determination threshold value. If the output value of the hyperplane discriminant function is greater than the determination threshold, the sequence discriminator outputs an identification result (for example, 1) indicating that the input evaluation value sequence is generated from the input data including the detection target. On the other hand, if the output value of the hyperplane discriminant function is equal to or less than the determination threshold, the series discriminator indicates that the input evaluation value series is not generated from the input data including the detection target, that is, the input evaluation value series is An identification result (for example, 0) indicating that the data is generated from input data not including the detection target is output.
Note that the lower limit value of the output value of the hyperplane identification function when the detection target is included in the input data is set as the determination threshold value. For example, the average of the output values calculated for a plurality of test human body images in which a human body is captured by a prior experiment and the average of output values calculated for a plurality of test non-human body images in which a human body is not captured An average value with the value can be used as a determination threshold value. Alternatively, the maximum output value calculated for the test non-human body image or the minimum output value calculated for the test human body image may be used as the determination threshold. That is, a value larger than the determination threshold is a value indicating that the evaluation value series is generated from input data including the detection target, and a value equal to or lower than the determination threshold is generated from the input data including the detection target. It is a value indicating that there is no.
Information representing the weight vector w and bias term b of the hyperplane discrimination function determined by the prior learning and the determination threshold are stored in the storage unit 14 as reference data of the evaluation value series.

  The object determination unit 155 inputs the evaluation value sequence received from the evaluation value sequence generation unit 154 to a sequence identifier that has been learned in advance. The object determination unit 155 determines that the human body is reflected in the partial image when the identification result output from the sequence identifier indicates that the evaluation value series is generated from the input data including the detection target, and the evaluation value When it represents that the series is not generated from the input data including the detection target, it is determined that the human body is not shown in the partial image.

  The series discriminator may output the output value of the hyperplane discriminant function instead of outputting the discrimination result as to whether or not the evaluation value series is generated from the input data including the detection target as the discriminating information. Good. In that case, the object determination unit 155 determines that the human body is reflected in the partial image if the output value output from the sequence discriminator is larger than the determination threshold, and if the output value is equal to or smaller than the determination threshold, the human body is included in the partial image. Judge that it is not reflected.

  When the target determination unit 155 determines that a human body is reflected in any partial image, the notification control unit 156 transmits an abnormal signal to the center device 50 via the communication unit 13.

  Hereinafter, the operation of the object detection process by the monitoring apparatus 10 according to the present embodiment will be described with reference to the flowchart shown in FIG. An operation flow described below is controlled by the control unit 15 according to a program stored in the storage unit 14 and read by the control unit 15.

  First, the control unit 15 causes the imaging unit 11 to capture a monitoring area, acquires a captured image via the interface unit 12, and stores the captured image in the storage unit 14. Then, the data input unit 150 reads the captured image from the storage unit 14 and sends it to the cutout unit 151 (step S501). Next, the cutout unit 151 cuts out a partial image from the acquired captured image, and sends the cutout partial image to the multilevel data generation unit 152 (step S502). Note that the control unit 15 performs the processes of steps S502 to S512 by the number of partial images cut out by the cutout unit 151.

  Next, the control unit 15 sets an information level (step S503). The control unit 15 sets predetermined information levels in order from the lowest, and executes the processes of steps S503 to S507 for the number of information levels to be set.

  The multilevel data generation unit 152 generates an individual level image corresponding to the information level set in step S503, and sends the generated individual level image to the evaluation value calculation unit 153 in association with the information level (step S504). If the information level is not the maximum value, the multi-level data generation unit 152 generates an individual level image corresponding to the information level from the partial image and sends it to the evaluation value calculation unit 153. On the other hand, when the information level is the maximum value, the multi-level data generation unit 152 sends the partial image as it is to the evaluation value calculation unit 153.

  Next, the evaluation value calculation unit 153 extracts a human body feature amount from the individual level image received from the multilevel data generation unit 152 (step S505). Next, the evaluation value calculation unit 153 calculates an evaluation value from the extracted human body feature amount, and sends the calculated evaluation value to the evaluation value series generation unit 154 in association with the information level (step S506).

  Next, the control unit 15 determines whether or not the processing of steps S503 to S506 has been executed for all information levels (step S507). If the processing in steps S503 to S506 has not been executed for all information levels (NO in step S507), the control unit 15 returns the processing to step S503 and repeats the processing in steps S503 to S506. On the other hand, if the processing of steps S503 to S506 has been executed for all information levels (YES in step S507), evaluation value series generation unit 154 outputs evaluation values received from evaluation value calculation unit 153 so far in a predetermined order. The evaluation value series arranged in (1) is generated (step S508).

  Next, the target determination unit 155 inputs the evaluation value sequence generated by the evaluation value sequence generation unit 154 to a previously learned sequence discriminator, and acquires the identification result output from the sequence discriminator (step S509). . Next, the target determination unit 155 determines whether or not the identification result output from the sequence discriminator indicates that the evaluation value sequence is generated from input data including the detection target (step S510). When the identification result indicates that the evaluation value series is generated from the input data including the detection target (YES in step S510), the target determination unit 155 determines that the partial image includes a human body, and the notification control unit 156 Transmits an abnormal signal to the center apparatus 50 via the communication unit 13 (step S511). When the abnormal signal is output, the control unit 15 ends the series of steps.

  On the other hand, when the identification result indicates that the evaluation value series is not generated from the input data including the detection target (NO in step S510), the target determination unit 155 does not determine that the partial image includes a human body. In this case, the control unit 15 determines whether all partial images having all predetermined positions and sizes have been cut out (step S512). If all partial images have not been cut out (NO in step S512), the control unit 15 returns the process to step S502 and repeats the processes in steps S502 to S512. On the other hand, if all the partial images have been cut out (YES in step S512), the control unit 15 ends the series of steps assuming that no intruder has been detected.

  For example, when the imaging unit 11 is installed so that the human body just fits in the captured image captured by the imaging unit 11, the control unit 15 does not need to cut out a partial image from the captured image. Direct individual level images may be generated. In that case, the cutout unit 151 is omitted from the control unit 15, and the processes in steps S502 and S512 are omitted in the flowchart of FIG.

  As described above, the monitoring apparatus according to the present embodiment generates and generates a plurality of individual level images with different information levels from partial images or captured images cut out from captured images obtained by capturing the monitoring region. An evaluation value is calculated from each individual level image. Then, the monitoring device generates an evaluation value sequence in which the calculated evaluation values are arranged in a predetermined order according to the information level, and inputs the generated evaluation value sequence to a pre-learned sequence discriminator to detect input data It is determined whether the target is included. Thereby, the monitoring system can improve the precision which detects a human body from a picked-up image.

  In addition, the monitoring device according to the present embodiment does not determine whether a human body is shown in the partial image based on the evaluation value itself, but instead determines based on the degree of change in the evaluation value according to the change in the information level. It is determined whether or not a human body is shown in the image. That is, the monitoring apparatus determines whether the human body is reflected in the partial image based on whether the feature amount extracted from the partial image is located on the human body side or the non-human body side with respect to the identification boundary in the feature amount space. Instead, the human body is detected based on the change in the position of the feature amount with respect to the identification boundary according to the change in the information level. Therefore, since it is not necessary to learn the identification boundary itself with high accuracy, it is not necessary to collect a large amount of learning images, and the development efficiency of the apparatus can be improved.

In the first modification example of the first embodiment, the evaluation value calculation unit uses a Haar-Like feature quantity instead of the HOG feature quantity as the human body feature quantity. The Haar-like feature amount is a luminance difference between a plurality of adjacent rectangular areas arbitrarily set in the image area. Details of the Haar-like features are disclosed in, for example, Paul Viola and Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, IEEE CVPR, vol.1, pp.511-518, 2001.
When using a Haar-like feature quantity as a human body feature quantity, each weak classifier of the evaluation value calculation unit is input with the Haar-like feature quantity obtained for a predetermined adjacent rectangular area, and each weak classifier is input Based on the Haar-like feature value, the higher the possibility that a human body is reflected in the corresponding partial image, the lower the possibility. Note that which adjacent rectangular area the Haar-like feature value to be input to each weak classifier is determined by prior learning. Since the learning procedure is the same as that in the case of using the HOG feature value as the human body feature value, the description is omitted. Information representing a Haar-like feature value used as each weak classifier determined by prior learning, information representing an adjacent rectangular area, and information representing an output function of each weak classifier are stored in the storage unit as reference data of the human body Is done.

In the second modification example of the first embodiment, the multi-level data generation unit generates a plurality of pieces of change data having different sharpness levels by changing the number of times of the filtering process of the averaging filter. In that case, the multi-level data generation unit uses a fixed size (for example, 3 × 3) averaging filter, and performs an image that has been filtered once, an image that has been performed twice, an image that has been performed n times, respectively. Generate. Since the generated image becomes more blurred as the filtering process is repeated, the sharpness decreases as the number of times of filtering is increased, and the sharpness increases as the number of times of filtering is decreased.
Alternatively, the multi-level data generation unit may generate a plurality of images having different sharpness levels by changing the filter coefficient of the averaging filter. In that case, the multi-level data generation unit weights the fixed size (for example, 5 × 5) filter so that the weight becomes larger as it is closer to the center of the filter, and the sharpness is increased by using filters having different weights. A plurality of different images are generated. The sharper the image, the lower the weight change from the edge to the center of the applied filter, and the sharper the weight change from the edge to the center. Becomes higher.

The third modification example in the first embodiment is an example in which change data is generated by a change process different from the above example. In the third modified example, the multi-level data generation unit superimposes a smaller amount of noise on an input image as the information level is higher, and a higher amount of noise on the input image so that the sharpness differs from each other. Generate multiple change data. In that case, the multi-level data generation unit randomly selects a predetermined number of pixels in the input image, and superimposes noise by changing the value of the selected pixel to a random value. The multi-level data generation unit generates a plurality of images in which different amounts of noise are respectively superimposed by changing the number of pixels whose values are changed in a plurality of ways. The larger the amount of noise to be superimposed, the lower the SN (signal to noise) ratio of the generated image and the lower the sharpness. The smaller the amount of noise to be superimposed, the higher the SN ratio and the higher the sharpness.
Alternatively, the multi-level data generation unit divides each pixel in the image into a group (segment) of adjacent pixels whose pixel values are similar to each other, and the pixel value of the pixels constituting the segment is the same value for each segment. A plurality of images having different sharpness levels may be generated by replacing with. In this case, the multi-level data generation unit sets a group of adjacent pixels whose absolute value of pixel value difference is equal to or less than a threshold value as a segment, and replaces the pixel value of the pixels constituting each segment with the average pixel value in the segment. The multi-level data generation unit generates a plurality of images having different sharpness levels by changing the threshold value in a plurality of ways. The sharpness of the generated image is lower as the threshold for grouping adjacent pixels is higher, and the sharpness is higher as the threshold for grouping adjacent pixels is lower.

The fourth modified example in the first embodiment is another example of generating change data by a change process different from the above example, and the degree of detail expressing the feature of the detection target (the degree that can be expressed). This is an example in which a certain level of clarity is used as an information level. In the fourth modified example, the multi-level data generation unit is configured as a plurality of pieces of change data having different information levels. The higher the information level, the higher the clarity of the image, and the lower the information level, the lower the clarity of the image. A plurality of change data having different intelligibility are generated. In that case, for example, the multi-level data generation unit increases the number of gradations of the pixel values in the image as the information level is higher, and reduces the number of gradations of the pixel values in the image as the information level is lower. Is generated.
Alternatively, the multi-level data generation unit may generate a plurality of images with higher contrast as the information level is higher and lower contrast as the information level is lower. In that case, the multi-level data generation unit generates an image obtained by converting the luminance value of each pixel so that the standard deviation of the luminance values of all the pixels in the image becomes small with respect to the input image.

The fifth modification in the first embodiment is an example in which the information level is set to a level suitable for the feature expression of the detection target. In the fifth modification, the multi-level data generation unit masks a portion where the detection target is represented in the input data as the information level is higher as a plurality of pieces of change data having different information levels (hereinafter referred to as a masking degree). A plurality of pieces of change data having different masking levels, and a higher masking level as the information level is lower. When the input data is image data, the multi-level data generation unit generates a plurality of images having different masking degrees for a portion representing a human body to be detected in the input image. In that case, the multi-level data generation unit sets a masking area of a predetermined size for masking a part of the image in the image, and replaces a pixel value in the masking area in the image with a fixed value. The multi-level data generation unit generates a plurality of images having different masking area sizes as a plurality of images having different masking degrees.
FIG. 6 is a schematic diagram for explaining the masking region. In the example shown in FIG. 6, the masking areas 600, 601, and 602 having different sizes are set so that the center position of each masking area matches the center position 604 of the image 603.
The larger the size of the masking area, the higher the degree of masking of the generated image and the lower the accuracy of human features appearing in the image, so the information level is lower, and the smaller the size of the masking area, the lower the degree of masking. The information level becomes higher. As an image with the lowest masking degree, an image with a masking area size of 0, that is, an original image can be used.

  Alternatively, the multilevel data generation unit may generate a plurality of images having different masking region positions as a plurality of images having different masking degrees. For example, when the detection target is a standing person, there is a high possibility that more characteristic parts of the human body exist near the upper part of the captured image. Therefore, in this case, by changing the position of the masking region from the upper part to the lower part, it is possible to increase the degree that the image is suitable for human body feature expression.

The sixth modified example in the first embodiment is an example in which change data different in degree suitable for the feature expression of the detection target is generated by a change process different from the above example. In the sixth modified example, the multi-level data generation unit generates a feature amount for the detection target, and as a plurality of pieces of change data having different information levels, the level of information amount represented by the feature amount as the information level is high , Referred to as the analysis level), a plurality of data with different amounts of information with respect to the feature amount so that the information level becomes smaller (the analysis level is lower) as the information level is lower (the analysis level is higher). Use. In this case, the feature quantity input to each weak classifier of the evaluation value calculation unit determined by the prior learning is obtained by the multilevel data generation unit, not the evaluation value calculation unit, and the multilevel data generation unit is obtained Change the analysis level of feature quantities in several ways.
For example, when using an HOG feature amount as a human body feature amount, the multi-level data generation unit increases the number of quantizations representing the feature amount as the analysis level is higher as a plurality of change data having different analysis levels. As the analysis level is lower, it is possible to generate a plurality of pieces of change data in which the number of quantizations representing the feature amount is reduced. As described above, the HOG feature amount is obtained as a histogram of the sum of the gradient intensities in each gradient direction in each cell in the image. The multi-level data generation unit changes the analysis level by changing the quantization number of the histogram, that is, the number of gradient directions, in a predetermined range of 2, 3, 4,. As the number of quantizations decreases, the human features are lost, so the information level of the extracted feature quantity decreases. Conversely, as the quantization number increases, the information level increases.
Alternatively, the multi-level data generation unit may change the analysis level by weighting each feature amount. For example, when using a HOG feature value as a feature value, the multi-level data generation unit sets different weighting factors for cells close to the center position of the partial image and for cells at the edge portion away from the center position of the partial image, The analysis level is changed by multiplying the histogram of each cell by this weighting factor. In this case, the greater the difference in weighting coefficient between the cell at the center of the partial image and the cell at the edge, the more the characteristics of the human body are lost. The smaller the is, the higher the information level is.
In addition, for example, the multi-level data generation unit, as a plurality of change data with different analysis levels, each element constituting the feature amount is larger as the analysis level is higher and lower as the analysis level is lower than 1 positive value. A plurality of pieces of change data each multiplied by a coefficient α (0 <α ≦ 1.0) are generated. The multilevel data generation unit changes the coefficient α in a plurality of ways within a preset range, for example, 0.1, 0.2, 0.3,. For example, when a Haar-like feature value is used, change data is generated by multiplying each luminance difference between a plurality of adjacent rectangular areas calculated as the Haar-like feature value by α. As α is smaller, the characteristics of the human body are lost, so the information level of the required feature amount is lower. Conversely, as α is larger, the information level is higher.

  When generating a plurality of pieces of change data having different analysis levels, in step S504 in the flowchart of FIG. 5, the multilevel data generation unit inputs a feature amount from each partial image to each weak classifier of the evaluation value calculation unit. Is multiplied by a coefficient α corresponding to the information level set in step S503, and sent to the evaluation value calculation unit. Then, step S505 is omitted, and in step S506, the evaluation value calculation unit calculates an evaluation value from the feature amount received from the multilevel data generation unit.

The seventh modified example in the first embodiment is another example in which change data different in degree suitable for the feature expression of the detection target is generated by a change process different from the above example. In the seventh modification, the multilevel data generation unit performs information conversion by performing geometric transformation on the input image in a plurality of ways, or by performing transformation processing corresponding to geometric transformation on the feature amount. A plurality of change data having different levels are generated.
For example, the multilevel data generation unit performs geometric transformation by rotating the partial image. Since the detection target shown in the image is different from the inclination of the detection target in many learning images as the rotation angle is larger, the degree to which the rotated image is suitable for the feature expression of the detection target decreases, and the information level Becomes lower. In this case, the multi-level data generation unit generates a plurality of individual level images obtained by rotating the partial images respectively with a smaller rotation angle with a higher information level and with a larger rotation angle with a lower information level.

Further, for example, the multilevel data generation unit performs geometric conversion by performing processing corresponding to image rotation on the feature amount. In this case, the feature value input to each weak classifier of the evaluation value calculation unit is obtained by the multilevel data generation unit, not the evaluation value calculation unit. For example, when the feature amount is an HOG feature amount, the multi-level data generation unit divides the partial image into a plurality of cells, and calculates the gradient direction and gradient strength of the pixel value in each pixel in the cell for each cell. For each gradient direction, a histogram with the sum of the gradient intensities in each gradient direction as the frequency is obtained. Then, the multi-level data generation unit obtains, as the HOG feature amount, a value obtained by circulating the frequency, that is, the sum of the gradient intensities in each gradient direction by a predetermined level, for each gradient direction of the obtained histogram. The multi-level data generation unit obtains a plurality of HOG feature amounts having different numbers of stages to be circulated and uses them as a plurality of pieces of change data having different information levels.
When the feature quantity is a Haar-like feature quantity, the multi-level data generation unit rotates the adjacent rectangular area for each adjacent rectangular area corresponding to the Haar-like feature quantity input to each weak classifier determined by pre-learning. Thus, the luminance difference between the rotated adjacent rectangular areas is obtained as a Haar-like feature value input to the weak classifier. In this case, the multi-level data generation unit sets a rotation angle for rotating the adjacent rectangular area for each adjacent rectangular area corresponding to the Haar-like feature input to each weak classifier determined by the prior learning from 0 ° to 180 °. A plurality of Haar-like feature quantities are obtained while rotating in a stepwise manner, and used as a plurality of change data having different information levels.
These processes for rotating the feature amount can be performed with a load lower than that for rotating the image.

Alternatively, the multilevel data generation unit may perform geometric conversion by converting the input image into a trapezoid or a parallelogram. Since the detection target shown in the image becomes different from the original shape as the degree of deformation increases, the degree of suitability of the deformed image for the feature expression of the detection target decreases and the information level decreases.
For example, the multi-level data generation unit performs trapezoidal conversion that shortens one of the four sides of the input image. The higher the information level, the closer the ratio of the one side to its opposite side is to 1, and the lower the information level. A plurality of change data in which the ratio is separated from 1 is generated.
Alternatively, the multi-level data generation unit performs geometric transformation that changes the angles of the four interior angles in the input image, and the higher the information level, the closer the converted angle is to 90 °, and the lower the information level, the more converted A plurality of data in which each angle of is separated from 90 ° is generated.

  In the eighth modification example of the first embodiment, the evaluation value calculation unit includes an Adaboost discriminator instead of the real Adaboost discriminator. This classifier includes a plurality of weak classifiers and a strong classifier that is determined by integrating the determination results of the weak classifiers. Each weak discriminator is inputted with a feature amount determined in advance for each weak discriminator, and each weak discriminator is based on the inputted feature amount and the human body is reflected in the corresponding partial image. If it is determined, 1 is output, and if it is determined that the human body is not captured, -1 is output. On the other hand, the strong classifier weights the output value from each weak classifier, obtains the weighted sum, and outputs it as an evaluation value. It should be noted that which feature quantity is input to each weak classifier and the weight for each weak classifier is based on a plurality of learning human body images in which a human body is photographed and a plurality of learning non-human body images in which a human body is not photographed. It is determined by prior learning using the calculated feature amount.

  Alternatively, the evaluation value calculation unit may calculate the evaluation value using a support vector machine, a perceptron having three or more layers, a random forest, or the like. In this case, the computer that performs the pre-learning extracts one or more feature amounts from the plurality of learning human body images and the plurality of learning non-human body images in advance, and performs the pre-learning using the extracted feature amounts. This pre-learning is performed so as to determine whether or not a human body is shown in a specific image when a feature amount extracted from the specific image is input. In the case of using a support vector machine, the evaluation value calculation unit has a positive value when the feature amount is located in the region on the human body side with respect to the identification boundary obtained by the prior learning in the feature amount space, and the region on the non-human body side. A value that becomes a negative value when positioned at the position and has an absolute value corresponding to the distance between the position of the feature amount and the identification boundary is calculated as the evaluation value. When a perceptron having three or more layers is used, the evaluation value calculation unit uses the sum of inputs to neurons in the output layer as an evaluation value. When a random forest is used, the evaluation value calculation unit combines the outputs of the decision trees generated by the prior learning so that the higher the probability that the human body is reflected in the image, the higher the evaluation value.

  Alternatively, the evaluation value calculation unit may output the evaluation value using a linear discriminant analysis method. In this case, the computer that performs the pre-learning extracts one or more feature amounts from the plurality of learning human body images and the plurality of learning non-human body images in advance, and creates a linear discriminant function using the extracted feature amounts. . When a feature amount extracted from a specific image is input, the computer creates a linear discriminant function so that a higher value is output as the probability that the human body is reflected in the image is higher. Then, the evaluation value calculation unit uses the output value of the linear discriminant function as the evaluation value.

  Alternatively, the evaluation value calculation unit may output the evaluation value using a mixed normal distribution. In this case, a computer that performs pre-learning extracts one or more feature amounts from a plurality of learning human body images in advance, and creates a mixed normal distribution using the extracted feature amounts. The evaluation value calculation unit uses a probability obtained when a feature amount extracted from a specific image is input to the created mixed normal distribution as an evaluation value. When the mixed normal distribution is used, pre-learning is performed using only the learning data to be detected, so that it is not necessary to collect learning data other than the detection target, that is, learning non-human body images in which no human body is shown.

  For example, the evaluation value calculation unit may calculate the evaluation value using a plurality of classifiers that have performed machine learning using different learning data. In that case, the evaluation value calculation unit connects each discriminator in series, executes discrimination processing in order from the first discriminator, and determines that any human discriminator is not shown in the image. Repeat the identification process until. The evaluation value calculation unit determines that the discriminator determines that the human body is not shown in the image when the output value from each discriminator is equal to or less than the threshold value. This threshold is based on output values calculated for a plurality of test human body images in which a human body is captured in a prior experiment and output values calculated for a plurality of test non-human body images in which a human body is not captured. Can be set to an identifiable value. Then, the evaluation value calculation unit uses the number of discriminators determined to have a human body in the image as the evaluation value.

  In the ninth modification example of the first embodiment, the evaluation value calculation unit calculates the evaluation value by pattern matching instead of calculating the evaluation value by the discriminator that performed machine learning. In this case, the control unit generates in advance a pattern of data obtained by performing an averaging process or the like on a plurality of learning data that is known to be detection target data, and stores the data pattern in the storage unit as reference data. . The evaluation value calculation unit calculates the degree of similarity between each of the images received from the multilevel data generation unit and the data pattern stored as the reference data as an evaluation value. The degree of similarity can be, for example, the inner product of each image and reference data.

  In the tenth modification of the first embodiment, the evaluation value calculation unit replaces the input data by calculating the evaluation value by a discriminator that has performed machine learning, or calculating the evaluation value by pattern matching. The degree of extraction from which data specific to the detection target is extracted is taken as the evaluation value. For example, when the detection target is a face, the data to be extracted can be pixels representing skin color (hereinafter referred to as skin color pixels). In this case, the control unit sets a range of pixel values of the skin color pixels to be extracted and a standard ratio of the skin color pixels that can be regarded as a face, and stores them in the storage unit in advance. The evaluation value calculation unit extracts skin color pixels from the image received from the multilevel data generation unit. The evaluation value calculation unit obtains the absolute value of the difference between the ratio of the number of skin color pixels to the total number of pixels in each image and the standard ratio stored in advance in the storage unit, and calculates the reciprocal of the obtained absolute value. The evaluation value.

  In the eleventh modification of the first embodiment, the evaluation value calculation unit outputs a degree representing the likelihood of being not a detection target, instead of outputting the degree representing the likelihood of detection as an evaluation value. In this case, the target sequence determination unit determines whether or not the detection target is included in the input data by the sequence identifier learned using the evaluation value sequence generated from the evaluation value that represents the probability that it is not the detection target. judge.

  In the twelfth modification of the first embodiment, the target sequence determination unit has a sequence identifier based on Adaboost instead of a sequence identifier based on a support vector machine. Each weak classifier is input with a predetermined number of consecutive elements among the elements of the evaluation value series determined in advance for each weak classifier. Each weak classifier outputs 1 when it is determined that the human body is reflected in the corresponding partial image based on the elements of the input evaluation value series, and when it is determined that the human body is not captured, -1 Is output. On the other hand, the strong classifier weights the output value from each weak classifier, and obtains the weighted sum. The strong discriminator outputs an identification result indicating that the input evaluation value series is generated from the input data including the detection target if the calculated weighted sum is larger than the determination threshold, An identification result indicating that the input evaluation value series is not generated from the input data including the detection target is output. Which element of the evaluation value series is input to each weak classifier and the weight for each weak classifier is a plurality of learning human images in which a human body is shown and a plurality of learnings in which a human body is not shown. It is determined by prior learning using the evaluation value series calculated from the non-human body image.

  Alternatively, the target sequence determination unit may include a sequence discriminator based on a mixed normal distribution instead of a sequence discriminator based on a support vector machine. In that case, a computer that performs pre-learning creates a mixed normal distribution using evaluation value sequences generated in advance from a plurality of learning human body images. This series discriminator inputs an evaluation value series generated from a specific image to the created mixed normal distribution, and acquires the probability of output. The sequence discriminator outputs an identification result indicating that the input evaluation value sequence is generated from the input data including the detection target if the acquired probability is larger than the determination threshold, and if it is equal to or less than the determination threshold, the input An identification result indicating that the evaluated evaluation value series is not generated from the input data including the detection target is output. When the mixed normal distribution is used, pre-learning is performed using only the learning data to be detected, so that it is not necessary to collect learning data other than the detection target, that is, learning non-human body images in which no human body is shown.

Next, a monitoring system in which the object detection device according to the second embodiment of the present invention is mounted will be described with reference to the drawings.
The monitoring apparatus according to the present embodiment generates a plurality of change data having different degrees of expressing the detection target from the input data by a plurality of change processes, and summarizes the evaluation values calculated for the generated plurality of change data. To generate an evaluation value series. The monitoring device further improves the accuracy of detecting the detection target from the input data by determining whether the detection target is included in the input data using the evaluation value series based on the change data generated by the different change processing Plan.

  The schematic configuration of the monitoring system according to the second embodiment is the same as the schematic configuration of the monitoring system according to the first embodiment shown in FIG. In the second embodiment, each monitoring device will be described as the monitoring device 20. The monitoring device 20 includes an imaging unit, an interface unit, a communication unit, a storage unit, and a control unit, similarly to the monitoring device 10 according to the first embodiment illustrated in FIG. Since the imaging unit, interface unit, communication unit, and storage unit of the monitoring device 20 are the same as the imaging unit 11, interface unit 12, communication unit 13, and storage unit 14 of the monitoring device 10, the imaging unit 11 and interface unit 12 respectively. The communication unit 13 and the storage unit 14 will be described. On the other hand, the control unit of the monitoring device 20 is described as the control unit 25 because the control unit 15 of the monitoring device 10 is partially different in function.

The control unit 25 is an example of a target detection device, and, similar to the control unit 15 according to the first embodiment illustrated in FIG. 4, as a functional module implemented by software operating on a processor, a data input unit, clipping Unit, a multi-level data generation unit, an evaluation value calculation unit, an evaluation value series generation unit, a target determination unit, and a notification control unit. In the second embodiment, the data input unit 250, the cutout unit, the data input unit, the cutout unit, the multilevel data generation unit, the evaluation value calculation unit, the evaluation value series generation unit, the target determination unit, and the notification control unit of the control unit 25 are used. 251, multi-level data generation unit 252, evaluation value calculation unit 253, evaluation value series generation unit 254, object determination unit 255, and notification control unit 256. Since the data input unit 250, the cutout unit 251 and the notification control unit 256 of the control unit 25 are the same as the data input unit 150, the cutout unit 151 and the notification control unit 156 of the control unit 15, the description thereof will be omitted. The multilevel data generation unit 252, the evaluation value calculation unit 253, the evaluation value series generation unit 254, and the target determination unit 255 will be described in detail.
Note that these units included in the control unit 25 may be configured by independent integrated circuits, firmware, a microprocessor, and the like.

The multi-level data generation unit 252 generates a plurality of change data with different degrees of expressing the detection target from one input data by a plurality of change processes, and associates the generated change data with each information level To the evaluation value calculation unit 253.
Hereinafter, an example in which the multilevel data generation unit 252 generates change data by two change processes will be described. The multi-level data generation unit 252 generates a plurality of first change data having different degrees of expressing the detection target by the first change process from one input data, and a second different from the first change process. A plurality of second change data is generated by varying the degree of expressing the detection target by the change process. For example, the multi-level data generation unit 252 sets the first change process as a process for changing the degree of expressing the detection target by the averaging filter, and the second change process as a process for changing the degree of expressing the detection target by adding noise. To do.
Note that the first change process and the second change process may be combined in any other combination, and different change processes may be combined from the various change processes described above as modifications of the first embodiment. Can do. Also, the number of combinations can be three or more.
The multi-level data generation unit 252 applies 33 averaging filters having different information levels to the partial image by applying averaging filters having smaller filter sizes as the information level is higher and larger filter sizes as the information level is lower. 1 Change data is generated. Further, the multi-level data generation unit 252 superimposes a smaller amount of noise as the information level is higher and a larger amount of noise as the information level is lower on the partial image, so that the tenth different information levels are obtained. 2. Generate change data.
The multilevel data generation unit 252 sends the generated first change data and second change data to the evaluation value calculation unit 253 as change data of one group.

  The evaluation value calculation unit 253 calculates an evaluation value for each change data, and sends each evaluation value to the evaluation value series generation unit 254 in association with each information level. For example, when there are two types of change data, the evaluation value calculation unit 253 calculates an evaluation value for each of the plurality of first change data and the plurality of second change data, and sets each evaluation value to the first information level and the second information. Correlate with information level.

The evaluation value series generation unit 254 generates an evaluation value series in which the evaluation values calculated for each change data are arranged in a predetermined order specified by the information level, and the generated evaluation value series is the target determination unit 255. To pass. For example, when 33 pieces of first change data are generated and 10 pieces of second change data are generated, the evaluation value series generation unit 254 uses the 33 evaluation values calculated for the first change data as the first change data. A first sequence arranged in ascending order of information levels in one change process is generated. Furthermore, the evaluation value series generation unit 254 generates a second series in which 10 evaluation values calculated for the second change data are arranged in ascending order of information levels in the second change process. Then, the evaluation value series generation unit 254 generates an evaluation value series by arranging the second series after the first series.
Note that the order in the first series and the second series can be the descending order of the corresponding information levels or another order, and the order at the time of prior learning and the order at the time of object detection need only be the same.

7A and 7B show an example of an evaluation value series in which evaluation values calculated for each of a plurality of first change data and a plurality of second change data are arranged and connected in ascending order of information levels. A graph 700 in FIG. 7A represents an example of an evaluation value series generated for input data including a detection target, and a graph 710 in FIG. 7B is an example of an evaluation value series generated for input data not including a detection target. Represents. In the graphs 700 and 710, the horizontal axis represents the level, and the vertical axis represents the evaluation value. Levels 1 to 33 correspond to the first information levels 1 to 33. The evaluation values of levels 1 to 32 are averaged filters whose filter size is (4 (n-1) +1) (n is an integer of 2 to 33, corresponding to the first information levels 32 to 1, respectively) Represents the evaluation value calculated for the first change data to which is applied, and the evaluation value of level 33 represents the evaluation value calculated for the original input data. On the other hand, levels 34-43 correspond to second information levels 1-10. The evaluation values of levels 34 to 42 represent the evaluation values calculated for the second changed data with 45%, 40%,..., 5% noise added to the number of pixels of the entire image. Represents an evaluation value calculated for the original input data.
As shown in the graph 700 of FIG. 7A, when the detection target is included in the input data, the evaluation value increases rapidly as the level increases in two locations, the level 25 to 32 region and the 34 to 43 region. There is a trend. On the other hand, as shown in the graph 710 of FIG. 7B, such a tendency is not seen when the detection target is not included in the input data. As described above, when the evaluation value series is generated for a plurality of types of information levels, when the detection target is included in the input data, a specific tendency appears as many as the number of types of information levels, and the detection target is included in the input data. In this case, since the tendency does not appear, the monitoring device 20 can further improve the accuracy of detecting the detection target.

  The object determination unit 255 includes a sequence discriminator that has been learned in advance using evaluation value sequences for learning arranged in the same order as the evaluation value sequence generated by the evaluation value sequence generation unit 254 for the information level. The object determination unit 255 inputs the evaluation value series generated by the evaluation value series generation unit 254 to the series discriminator, and determines whether or not the detection target is included in the input data based on the output identification result. Output the result.

  Hereinafter, the operation of the object detection process by the monitoring device 20 will be described with reference to the flowchart shown in FIG. This flowchart can be executed in the monitoring apparatus 20 instead of the flowchart shown in FIG. The operation flow described below is controlled by the control unit 25 in accordance with a program stored in the storage unit 14 and read into the control unit 25. The processes in steps S801 to S807 and S811 to S813 in the flowchart shown in FIG. 8 are the same as the processes in steps S501 to S507 and S510 to S512 in the flowchart shown in FIG. Only the process of S810 will be described.

  The processes in steps S803 to SS807 are executed for each change process (first change process, second change process,...). In step S808, the control unit 25 determines whether or not the processes in steps S803 to S807 have been executed for all the change processes. If the processes in steps S803 to S807 are not executed for all the changing processes (NO in step S808), the control unit 25 returns the process to step S803 and repeats the processes in steps S803 to S807. On the other hand, if the processes in steps S803 to S807 are executed for all the change processes (YES in step S808), evaluation value series generation unit 254 uses the evaluation values received from evaluation value calculation unit 253 so far. An evaluation value series is generated (step S809).

  Next, the object determination unit 255 generates an evaluation value sequence in a sequence discriminator previously learned using the evaluation value sequence for learning arranged in the same order as the evaluation value sequence generated by the evaluation value sequence generation unit 254. The evaluation value series generated by the unit 254 is input, and the identification result output from the series discriminator is acquired (step S810).

  As described above, the monitoring apparatus according to the present embodiment generates a plurality of change data with different degrees of expressing the detection target from the input data by a plurality of change processes, and the generated plurality of change data An evaluation value series is generated by combining the calculated evaluation values. The monitoring device further improves the accuracy of detecting the detection target from the input data by determining whether the detection target is included in the input data using the evaluation value series based on the change data generated by different change processing can do.

Next, a monitoring system in which an object detection device according to a third embodiment of the present invention is mounted will be described with reference to the drawings.
The monitoring apparatus according to the present embodiment generates an evaluation value sequence by smoothing an original sequence in which evaluation values are arranged in a preset order specified by an information level, and inputs the smoothed evaluation value sequence. It is determined whether the detection target is included in the data. Thereby, the monitoring apparatus removes the influence of the fine change of the evaluation value according to the information level, and further improves the accuracy of detecting the detection target from the input data.

  The schematic configuration of the monitoring system according to the third embodiment is the same as the schematic configuration of the monitoring system according to the first embodiment shown in FIG. In the second embodiment, each monitoring device will be described as the monitoring device 30. The monitoring device 30 includes an imaging unit, an interface unit, a communication unit, a storage unit, and a control unit, similarly to the monitoring device 10 according to the first embodiment shown in FIG. Since the imaging unit, interface unit, communication unit, and storage unit of the monitoring device 20 are the same as the imaging unit 11, interface unit 12, communication unit 13, and storage unit 14 of the monitoring device 10, the imaging unit 11 and interface unit 12 respectively. The communication unit 13 and the storage unit 14 will be described. On the other hand, the control unit of the monitoring device 30 is described as the control unit 35 because a part of the functions of the control unit 15 of the monitoring device 10 is different.

The control unit 35 is an example of a target detection device, and as a functional module implemented by software operating on a processor, like the control unit 15 included in the monitoring device 10 according to the first embodiment illustrated in FIG. A data input unit, a cutout unit, a multi-level data generation unit, an evaluation value calculation unit, an evaluation value series generation unit, a target determination unit, and a notification control unit; In the third embodiment, the data input unit 350, the cutout unit, the data input unit, the cutout unit, the multilevel data generation unit, the evaluation value calculation unit, the evaluation value series generation unit, the target determination unit, and the notification control unit of the control unit 35 351, a multi-level data generation unit 352, an evaluation value calculation unit 353, an evaluation value series generation unit 354, a target determination unit 355, and a notification control unit 356. The data input unit 350, the cutout unit 351, the multilevel data generation unit 352, the evaluation value calculation unit 353, the target determination unit 355, and the notification control unit 356 of the control unit 35 are the data input unit 150, the cutout unit 151, Since it is the same as that of the multilevel data generation part 152, the evaluation value calculation part 153, the object determination part 155, and the notification control part 156, description is abbreviate | omitted and below, the evaluation value series generation part 354 is demonstrated in detail.
Note that these units included in the control unit 35 may be configured by independent integrated circuits, firmware, a microprocessor, and the like.

  The evaluation value series generation unit 354 generates an original series in which evaluation values calculated for each change data included in the multi-level data are arranged in ascending or descending order of information levels, and smoothes the original series to generate the evaluation value series. Generate and pass to the target determination unit 155. For example, the evaluation value series generation unit 354 has, in the original series in which the evaluation values are arranged in ascending order of information levels, for each evaluation value, the evaluation value and the information level closer to the information level on the side where the information level is lower than the evaluation value. Smoothing is performed by taking a weighted moving average using a predetermined number of evaluation values. Alternatively, the evaluation value series generation unit 354 takes, for each evaluation value, a simple moving average using the evaluation value and a predetermined number of evaluation values in order of closer information level on the side where the information level is lower than the evaluation value. May be smoothed.

9A and 9B show graphs for explaining the smoothed evaluation value series. A graph 900 in FIG. 9A represents an example of the original series before smoothing, and a graph 910 in FIG. 9B represents an example of an evaluation value series obtained by smoothing the original series in the graph 900. In the graphs 900 and 910, the horizontal axis represents the information level, and the vertical axis represents the evaluation value.
As shown in the graph 900 of FIG. 9A, in the original series, the evaluation value increases as the information level increases even though the information level is a medium area such as the area where the information level is 16 to 18 and 22 to 23. In some cases, the evaluation value decreases or the evaluation value decreases as the information level increases, even though the information level is high, such as the region where the information level is 29 to 30. On the other hand, as shown in the graph 910 of FIG. 9B, by smoothing the original sequence of the graph 900, the evaluation value stably decreases as the information level increases in the region where the information level is medium. In the high region, it rises stably as the information level rises. Therefore, the monitoring device 10 can further improve the accuracy of detecting the detection target.

  The object determination unit 355 includes a sequence discriminator that has been learned in advance using a learning evaluation value sequence obtained by smoothing the original sequence in the same manner as the evaluation value sequence generated by the evaluation value sequence generation unit 354. The object determining unit 355 inputs the evaluation value series generated by the evaluation value series generating unit 354 to the series discriminator, and determines whether or not the detection target is included in the input data based on the output identification result. Output the result.

  Hereinafter, the operation of the object detection process performed by the monitoring device 30 will be described. The flowchart of the object detection process by the monitoring apparatus 30 is the same as the flowchart shown in FIG. However, in step S508, the evaluation value sequence generation unit 354 generates an original sequence in which the evaluation values received from the evaluation value calculation unit 353 are arranged in a predetermined order, smoothes the generated original sequence, and evaluates the evaluation value sequence. Is generated. In step S509, the object determination unit 355 applies the evaluation value to the sequence classifier previously learned using the learning evaluation value sequence smoothed in the same manner as the evaluation value sequence generated by the evaluation value sequence generation unit 354. The evaluation value sequence generated by the sequence generation unit 354 is input, and the identification result output from the sequence discriminator is acquired.

  As described above, the monitoring apparatus according to the present embodiment generates an evaluation value sequence by smoothing an original sequence in which evaluation values are arranged in a preset order defined by an information level, and is smoothed. It is determined whether the detection target is included in the input data using the evaluation value series. Thereby, the monitoring apparatus can remove the influence of the fine change of the evaluation value according to the information level, and can detect the detection target stably and accurately from the input data.

  In the monitoring system in the first to third embodiments, the center device may include a control unit similar to the control unit of the monitoring device, and determine whether or not a captured image includes a human body. In that case, the monitoring device transmits an image obtained by capturing the monitoring region to the center device, the center device determines whether or not a human body is included in the received image, and determines whether or not a person has entered the monitoring region. judge. Also in this case, the monitoring system can improve the accuracy of detecting the human body from the captured image.

Moreover, in the target detection apparatus of the first to third embodiments, the detection target is not limited to the human body, and may be a human face, a mask, or the like, for example. In this case, the target detection device determines whether or not a person's face is included in the captured image, or whether or not a face wearing a mask is included.
Furthermore, the input data is not limited to image data, and may be, for example, an acoustic signal. In that case, the detection target can be, for example, a scream. The target detection device cuts out a signal in a sound section from an acoustic signal generated from sound in a monitoring space for monitoring the occurrence of scream, generates a plurality of signals having different information levels from the cut out signal, and generates the signal An evaluation value representing the degree of screaming is calculated from each signal. Then, the target detection device generates an evaluation value series in which the calculated evaluation values are arranged in a predetermined order. Further, the target detection apparatus uses the evaluation value series generated by using the evaluation value series generated for the acoustic signal including scream and the evaluation value series generated for the acoustic signal not included in advance. It inputs into a discriminator and determines whether a scream is included in an acoustic signal.
Further, the input data may be a Doppler signal. In this case, the detection target can be, for example, the movement of a human body. The target detection device extracts only the Doppler component included in the reflected wave obtained by transmitting the electromagnetic wave to the monitoring area and sets it as a Doppler signal. The target detection device generates a plurality of signals having different information levels from the signal cut out from the Doppler signal, and calculates an evaluation value representing the degree of human bodyness from the generated signals. Then, the target detection device generates an evaluation value series in which the calculated evaluation values are arranged in a predetermined order. Further, the target detection device uses the evaluation value series generated for the Doppler signal caused by the movement of the human body and the evaluation value series generated for the Doppler signal not caused by the movement of the human body. It is input to a sequence discriminator learned in advance, and it is determined whether or not the Doppler signal is caused by the movement of the human body.

  As described above, those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

10, 20, 30 Monitoring device 15, 25, 35 Control unit 150, 250, 350 Data input unit 151, 251, 351 Cutout unit 152, 252, 352 Multi-level data generation unit 153, 253, 353 Evaluation value calculation unit 154, 254, 354 Evaluation value series generation unit 155, 255, 355 Target determination unit 156, 256, 356 Notification control unit

Claims (3)

A target detection device that determines whether or not a detection target is included in input data acquired from a data input unit,
A multi-level data generation unit that generates change data with different information levels that represent the detection target from the input data;
For each of the change data, an evaluation value calculation unit that calculates an evaluation value representing the degree of likelihood of detection,
An evaluation value sequence generation unit that generates an evaluation value sequence by arranging the evaluation values in a preset order prescribed by the information level;
A target determination unit that determines whether or not the detection target is included in the input data based on identification information for identifying whether or not the evaluation value series is generated from input data including the detection target;
An object detection apparatus comprising:   The multi-level data generation unit generates first change data having different degrees of expressing the detection target from the input data by the first change process, and a second change process different from the first change process 2. The target according to claim 1, wherein second change data in which degrees of expressing the detection target are different from each other is generated from the input data, and the first change data and the second change data are used as the change data. Detection device.   3. The evaluation value series generation unit according to claim 1, wherein the evaluation value series generation unit generates the evaluation value series by smoothing an original series in which the evaluation values are arranged in a preset order defined by the information level. Object detection device.

Images