JP7336835B2 - Attribute determination device, attribute determination system, and attribute determination method - Google Patents

Description

The present invention relates to an attribute determination device, an attribute determination system, and an attribute determination method for determining attributes of a person based on each frame image of the person photographed from above.

Conventionally, an attribute recognition method using a face image is widely known as a technique for accurately recognizing a person's attributes such as gender and age from an image of the person photographed by a camera (see Patent Documents 1 and 2, for example). . However, in a store, for example, in order to recognize the attributes of a person who contacts a certain product group based on the face image, it is necessary to install a camera for acquiring the face image at each sales floor and each display shelf. . In this case, the number of cameras to be installed increases, which not only increases the cost but also increases the psychological burden on the user to whom the camera is directed.

Therefore, from the viewpoint of reducing costs and reducing the psychological burden on users, a camera is installed on the ceiling of a store, etc., a person is photographed from above in a wide range, an image is acquired, and the attribute of the person is recognized from the acquired image. I can think of a way. For example, in the system of Patent Document 3, the main camera installed on the ceiling or wall captures the entire store and analyzes the image part of the person, comprehensively considering the customer's face, hairstyle, clothes, height, accessories, shoes, etc. If it is difficult to identify the person's gender and age group, the facial image captured by the auxiliary camera installed near the display shelf in the sales floor is analyzed in detail to guess the gender and age group. .

Japanese Patent Application Laid-Open No. 2010-61465 (see claims 1 and 2, FIG. 1, FIG. 5, etc.) Japanese Patent Application Laid-Open No. 2008-176689 (claims 1 and 2, see FIG. 1, etc.) Japanese Patent Application Laid-Open No. 2007-74330 (claims 1 to 3, paragraphs [0021], [0023], see FIG. 1, etc.)

However, it is difficult to accurately determine a person's attribute from an image captured by a camera installed on the ceiling or the like. This can be easily understood from the fact that in Japanese Patent Application Laid-Open No. 2002-200012, when it is difficult to specify attributes, the attributes are estimated using the face image acquired by the auxiliary camera.

For example, in a store, people walk, stop, crouch, and stand up to pick up items from the lower shelves. In this way, when the position, action, posture, etc. of a person change in time series, the recognition result of the attribute of the person based on the captured image may differ between frames. For example, in a frame in which a person is walking, a recognition result of "man in his 20s" is obtained based on the captured image, and in a frame in which the same person is standing still, a recognition result of "man in his 40s" is obtained based on the captured image. A recognition result may be obtained. This is because when a person is walking, the image of the person blurs in the image, and the recognition accuracy of the attribute of the person based on the image is lowered.

Similarly, for example, in a frame in which a person crouches down, a recognition result of "man in his 20s" is obtained based on the captured image, and in a frame in which the same person stands up, a recognition result of "man in his 40s" is obtained based on the captured image. can be obtained. This is because when a person crouching down is photographed from above, an image with a part of the person's body hidden is obtained, and image data of the person's whole body cannot be obtained. due to

When judging (determining) the attributes of the same person between each frame based on the image of each frame photographed from above, events such as "walking" and "squatting" that reduce the accuracy of attribute recognition ( event affecting the recognition of the attribute) continues for several frames (for example, m frames where m is a natural number of 2 or more), even if the attribute can be recognized with high accuracy due to the disappearance of the event in the subsequent frames, the above Due to the influence of low-accuracy attribute recognition results in a few frames, an erroneous attribute determination (for example, a person who is actually "a man in his 40s") is determined for all frames (for example, M frames where M is a natural number of 3 or more larger than m) (This example will be described as a comparative example in the embodiments described later). In this case, it cannot be said that the attributes of a person can be determined with high accuracy.

The present invention has been made to solve the above problems, and its object is to determine the attributes of the same person between each frame based on the image of each frame photographing the person from above, To provide an attribute determination device, an attribute determination system, and an attribute determination method capable of accurately determining a person's attribute as a whole (total of each frame) even when an event affecting recognition of the attribute continues over several frames. That's what it is.

An attribute determination device according to one aspect of the present invention is an attribute determination device that determines attributes of a person based on images of frames photographing the person from above, and based on the images of each frame, the a person recognizing unit for recognizing, for each frame, person information indicating information about the image of the person in the image, attributes of the person, and events affecting recognition of the attributes; and the person information of each frame. a person identification unit for determining whether or not the images of the person are the same person between frames based on Regarding a person, for each frame, attribute information obtained by adding the recognition result of the event to the recognition result of the attribute is obtained for each class of the recognized attribute, and the attribute information is integrated in a plurality of frames for each class. an attribute determining unit that determines the attribute of the person based on the result of the determination.

An attribute determination system according to another aspect of the present invention includes the attribute determination device described above, and a management server connected to the attribute determination device via a communication line, wherein the management server transmits from the attribute determination device The information includes the attribute determined by the attribute determination unit of the attribute determination device.

An attribute determination method according to still another aspect of the present invention is an attribute determination method for determining an attribute of a person based on an image of each frame photographing the person from above, the method comprising: a person recognition step of recognizing, for each frame, person information indicating information of the image of the person in the image, an attribute of the person, and an event affecting recognition of the attribute; a person identification step of determining whether or not the image of the person is the same person between frames based on the person information; and determining whether the image of the person is the image of the same person between frames. attribute information obtained by adding the recognition result of the event to the recognition result of the attribute is obtained for each class of the recognized attribute for each frame of the person, and the attribute information is obtained for a plurality of frames for each class. and an attribute determination step of determining the attributes of the person based on the integrated result of.

Even if an event affecting attribute recognition continues over several frames, it is possible to reduce the adverse effects of the above few frames on the final attribute determination, and to accurately determine a person's attribute as a whole (total of each frame). can.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the structure of the outline of the attribute determination system of one embodiment of this invention. It is a block diagram which shows the detailed structure of the attribute determination apparatus which the said attribute determination system has. FIG. 4 is an explanatory diagram showing an example of a person's image and a person's rectangle in an image of an arbitrary frame; FIG. 4 is an explanatory diagram schematically showing various positions of a person's image within an image; FIG. 4 is an explanatory diagram schematically showing an n-th frame image and an (n+1)-th frame image; It is a block diagram which shows the detailed structure of the management server which the said attribute determination system has. It is a flowchart which shows the flow of the process in the said attribute determination system. FIG. 4 is an explanatory diagram showing an example of information obtained for the first frame in the attribute determination system; FIG. 4 is an explanatory diagram showing an example of information obtained for a second frame in the attribute determination system; FIG. 9 is an explanatory diagram showing an example of information obtained for the third frame in the attribute determination system; FIG. 10 is an explanatory diagram schematically showing images of some frames out of a plurality of temporally different frames obtained in an attribute determination system according to another embodiment of the present invention; It is a flowchart which shows the flow of the process in the said attribute determination system. FIG. 4 is an explanatory diagram showing an example of information obtained for the first frame in the attribute determination system; FIG. 4 is an explanatory diagram showing an example of information obtained for a second frame in the attribute determination system; FIG. 9 is an explanatory diagram showing an example of information obtained for the third frame in the attribute determination system; FIG. 10 is an explanatory diagram schematically showing images of a part of a plurality of temporally different frames obtained in an attribute determination system according to still another embodiment of the present invention; It is a flowchart which shows the flow of the process in the said attribute determination system. FIG. 4 is an explanatory diagram showing an example of information obtained for the first and second frames in the attribute determination system; FIG. 9 is an explanatory diagram showing an example of information obtained for the third frame in the attribute determination system; FIG. 11 is an explanatory diagram schematically showing an arbitrary frame image of two persons photographed from above in an attribute determination system according to still another embodiment of the present invention; It is explanatory drawing which each shows the person rectangle which prescribe|regulates the position of the image of two people in the said image. It is a flowchart which shows the flow of the process in the said attribute determination system. FIG. 4 is an explanatory diagram showing an example of information obtained for the first and second frames in the attribute determination system; FIG. 9 is an explanatory diagram showing an example of information obtained for the third frame in the attribute determination system;

Each embodiment of the present invention will be described below with reference to the drawings. In addition, the present invention is not limited to the following contents.

<Embodiment 1>
[Attribute determination system]
FIG. 1 is a block diagram showing a schematic configuration of an attribute determination system 1 of this embodiment. The attribute determination device 1 includes an imaging unit 2, an attribute determination device 3, and a management server 4. FIG. The imaging unit 2 and the attribute determination device 3 are communicably connected via a communication line N1, and the attribute determination device 3 and the management server 4 are communicably connected via a communication line N2. The communication lines N1 and N2 are appropriately selected from, for example, cables, optical fibers, wired LANs (Local Area Networks), wireless LANs, Internet lines, and the like. Details of the imaging unit 2, the attribute determination device 3, and the management server 4 will be described below.

(imaging unit)
The imaging unit 2 is installed, for example, on the ceiling or wall of the store, and is composed of a camera that captures images of people in the store from above and obtains images of frames that differ in time. The number of imaging units 2 installed in the store is not particularly limited, and may be one or two or more. Image data acquired by at least one imaging unit 2 is output to the attribute determination device 3 via the communication line N1.

(attribute determination device)
The attribute determination device 3 is a terminal device that determines the attributes of a person based on the image acquired by the imaging unit 2 and input to the attribute determination device 3, that is, the image of each frame obtained by photographing the person from above. , for example, a personal computer. The attribute determination device 3 may be installed in the same store as the store where the imaging unit 2 is installed, or may be installed outside the store so as to communicate with the imaging unit 2 .

FIG. 2 is a block diagram showing the detailed configuration of the attribute determination device 3. As shown in FIG. The attribute determination device 3 includes a recognition processing section 11 , a storage section 12 , an input section 13 , a display section 14 , a communication section 15 and a control section 16 .

The storage unit 12 is a memory that stores an operation program for operating each unit of the attribute determination device 3, data obtained by processing in the recognition processing unit 11 (for example, information about the attribute of the determined person), etc. For example, it consists of a hard disk. Note that the storage unit 12 may be configured by appropriately selecting from recording media such as RAM (Random Access Memory), ROM (Read Only Memory), optical disk, magneto-optical disk, and nonvolatile memory.

The input unit 13 is composed of, for example, a keyboard, a mouse, a touch pad, a touch panel, etc., and receives various instruction inputs from an operator (user) who operates the attribute determination device 3, for example. The display unit 14 is a device for displaying various kinds of information including the processing result (for example, determined attributes) of the recognition processing unit 11, and is configured by, for example, a liquid crystal display device. The communication unit 15 is an interface including an input/output port for communicating with the outside. The communication unit 15 includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like when performing wireless communication with the outside. The control section 16 is composed of a central processing unit (CPU) that controls the operation of each section of the attribute determination device 3 , and operates according to an operation program stored in the storage section 12 .

The recognition processing unit 11 is an arithmetic device that performs processing related to determination of attributes of a person, and includes, for example, a GPU (Graphics Processing Unit) that is an arithmetic device specialized for real-time image processing. Note that the recognition processing unit 11 may be configured with the same CPU as the control unit 16 or a separate CPU. Such a recognition processing section 11 has a person recognition section 11a, a person identification section 11b, and an attribute determination section 11c. That is, the GPU that constitutes the recognition processing unit 11 has functions as a person recognition unit 11a, a person identification unit 11b, and an attribute determination unit 11c.

Based on an image of each frame in which a person is photographed from above, the person recognition unit 11a recognizes person information indicating information about the image of the person in the image, attributes of the person, and events affecting the recognition of the attributes. Recognize each frame.

The person information includes, for example, the position of the person's image in the image, the type and ratio of colors included in the person's image, the size of the person's image, and the like. Here, the position of the image of the person in the image is defined, for example, by a person rectangle containing the image in the image. FIG. 3 shows an example of a person image P and a person rectangle R in an image 2a of an arbitrary frame. In FIG. 3, reference numeral 5 indicates a product shelf in the store, and reference numeral 6 indicates an aisle in the store. Recognition of the image P of the person in the image 2a (recognition of the presence or absence and position of the person image) is performed by, for example, processing based on known image processing software to specify the shape and position of the person's head, the positions of each joint, etc. It can be carried out. When the person recognition unit 11a recognizes the person image P in the image 2a, the person recognition unit 11a can set a person rectangle R surrounding the person image P in the image 2a. can define the position of the image P of .

Note that the person rectangle R is a rectangle (frame) surrounding the person image P so as to be in contact with at least a part of the person image P in terms of accurately defining the position of the person image P in the image 2a. However, it is not necessarily required to be in contact with the image P of the person. For example, there may be a predetermined (small) margin between the person rectangle R and the image P of the person. Also, a rectangle generally refers to a rectangle, but here, the concept includes a square, which is a special shape of a rectangle.

The attribute of the person described above is at least one of age and sex of the person. The person recognizing unit 11a may recognize the age of the person in terms of the full age of each year, but may also recognize the age of the person in general ages such as the 20s, 30s, and 40s. , old people, and so on. Gender indicates male or female.

In this embodiment, the person recognition unit 11a includes a neural network capable of machine learning such as deep learning. Using this neural network, the person's attributes can be recognized. can. More specifically, when the image data of each frame is input to a neural network that has been trained for attribute recognition in advance, the neural network outputs attribute recognition results and scores that indicate the likelihood of the recognition results. Calculated. As a result, it is possible to obtain, for example, the recognition result that the person is in his 40s and the gender is male, and a score (for example, 0.8) indicating the probability of the recognition. The score is a value (score) between 0 and 1, and the closer to 1, the more likely the recognition result is.

Events that affect the recognition of the above attributes include, for example, the position of a person's image within an image. FIG. 4 schematically shows various positions of a person's image P within the image 2a. At the position (1) in the image 2a, a part of the image P of the person is blocked by the hanging signboard 7 inside the store, and at the position (2), the person is positioned directly below the imaging unit 2. Therefore, it is difficult for the whole body of the person to appear in the image 2a. In these cases, the image data of the person for recognizing the attributes is missing, which affects the recognition of the attributes of the person based on the image. That is, the position of the person's image P shown in (1) and (2) above is an event that affects the recognition of the person's attribute. On the other hand, in the image 2a, at the position (3) where the image P of the person is close to the edge of the image, the whole body of the person is photographed. It is ideal for recognizing attributes based on In other words, the position (3) above does not become an event that affects the recognition of a person's attribute. However, if the person's image P is too close to the edge of the image from the position of (3), the person's image P will not fit in the image 2a, and there is a possibility that the person's image data will be lost. It can be an event that affects the recognition of the attributes of

Based on the person information (time-series information) of each frame recognized by the person recognition unit 11a, the person identification unit 11b determines whether the image of the person (within the person rectangle) is the image of the same person between the frames. to judge whether FIG. 5 schematically shows the n-th frame image 2a and the (n+1)-th frame image 2a (where n is a natural number). For example, the person identification unit 11b compares the position of the person rectangle R _n in the n-th frame image 2a with the position of the person rectangle R _n+1 in the (n+1)-th frame image 2a, and determines these positions. (the amount of movement of the person rectangle) is within a predetermined range (determined according to the frame _rate ) _. That is, it is determined whether the person image _Pn in the person rectangle _Rn and the person image _{Pn +1 in the person rectangle Rn+} 1 are the same person. be able to.

The person identification unit 11b also determines whether the difference between the vertical (horizontal) length of the person rectangle R _n and the vertical (horizontal) length of the person rectangle R _n+1 is within a predetermined range. By determining whether the difference (or ratio) between the area occupied by each color _{within n} and the area occupied by each color within person rectangle R _n+1 is within a predetermined range, and the person image P _n+1 in the person rectangle R _n+1 are the same _person image.

The attribute determining unit 11c determines, for each frame, the attribute recognition result obtained by adding the event recognition result to the attribute recognition result for a person for whom the person identifying unit 11b determines that the image of the person in each frame is the image of the same person. Information is sought for each class of recognized attributes. For example, when considering age and gender as attributes, age classes include, for example, 20s, 30s, 40s, etc., and gender classes include two classes, male and female. . Therefore, as classes for all attributes, there are as many as the number of age classes×the number of gender classes. Therefore, the attribute determination unit 11c obtains attribute information for each class (for example, for each class of men in their 20s, men in their 30s, . . . women in their 40s). A specific example of attribute information will be described later. In particular, the attribute determination unit 11c sets the reliability (adoption rate) of the score corresponding to the recognition result of the event (here, the position of the image of the person) that affects the recognition of the attribute, and the person recognition unit 11a calculates The attribute information is obtained for each class based on the calculated score and the set reliability.

Here, a numerical range of 0 to 1, for example, can be considered as the above reliability. For example, in the image 2a shown in FIG. 4, when the image P of a person is at position (3), as described above, the position has little effect on the recognition of the attributes of the person. Set the score confidence to 1.0. Further, when the image P of the person is at the position (2) or (3), as described above, the above position may adversely affect the recognition of the person's attributes. sets the confidence of the score to 0.7, and for the position of (1) sets the confidence of the score to 0.2. Note that the user may arbitrarily set an area within the image 2a and set the above reliability for each set area.

The attribute determination unit 11c obtains attribute information for each class by multiplying the score calculated by the person recognition unit 11a by the reliability, and based on the result of integrating the attribute information in a plurality of frames for each class, Determine the attributes of a person. The details of the attribute determination process will be described later in the explanation of the operation.

(management server)
The management server 4 shown in FIG. 1 is a terminal device that stores information about a person's attributes determined by the attribute determination device 3, and is configured by, for example, a personal computer. FIG. 6 is a block diagram showing the detailed configuration of the management server 4. As shown in FIG. The management server 4 has a storage section 21 , a communication section 22 and a control section 23 .

The storage unit 21 is a memory that stores an operation program for operating each unit of the management server 4 and information sent from the attribute determination device 3 (for example, attributes determined by the attribute determination unit 11c). It is configured. The storage unit 21 may be configured by appropriately selecting recording media such as RAM, ROM, optical disk, magneto-optical disk, and non-volatile memory.

The communication unit 22 is an interface including an input/output port for communicating with the outside. The communication unit 22 includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like when performing wireless communication with the outside. The control unit 23 is composed of a CPU that controls the operation of each unit of the management server 4 and operates according to the operation program stored in the storage unit 21 .

The management server 4 may also include an input unit such as a keyboard, a display unit such as a display, and an arithmetic processing unit that performs the same processing as the recognition processing unit 11 of the attribute determination device 3 .

[Attribute determination method]
Next, the operation (attribute determination method) in the attribute determination system 1 of this embodiment will be described. FIG. 7 is a flow chart showing the flow of processing in the attribute determination system 1 described above. In order to simplify the explanation below, here, the attributes of a person are classified into two classes, "male in his 40s" and "male in his 20s". , “male in his 40s” (assuming that “male in his 40s” is the correct attribute). An attribute A _n shown below is an attribute recognized by the person recognition unit 11a based on the n-th frame image, and indicates the attribute of the person appearing in the n-th frame image. It shall be distinguished from attribute B which is finally determined.

First, when the attribute determination device 3 acquires the image of the first frame from the imaging unit 2 with n=1 (S1) (S2), the person recognition unit 11a recognizes the person by the method described above based on the image. A rectangle R _n (=R ₁ ), a person attribute A _n (=A ₁ ), and an event affecting the recognition of the attribute A _n (here, a person image P _{n (} = The position of P _{1 )) is recognized, and a score C n (} =C ₁ ) indicating the certainty of the recognition result of the attribute An is calculated (S3; person recognition step). These recognition results and scores C _n are stored in the storage unit 12 .

_Next , the person identification unit 11b identifies a person within the person rectangle R _n (S4; person identification step). A person is identified by assigning an identification number to the image P _n of (for example, ID=0001). Information such as the person rectangle _Rn recognized in S3 is stored in the storage unit 12 in association with the identification number assigned in S4. The process of S4 may be performed in parallel with the recognition of the attribute A _n in S3, the recognition of the event, and the calculation of the score C _n after the person rectangle R _n is recognized.

Subsequently, the attribute determination unit 11c sets the reliability f(A _n ) (=f(A ₁ )) of the score C _n corresponding to the recognition result of the event affecting the recognition of the attribute A _n in S3. (S5 to S7). In other words, if the event (the position of the person's image P _n ) recognized in S3 is a position that affects attribute recognition (Yes in S5), the attribute determination unit 11c determines whether the event recognition result corresponds to the event recognition result. Then, the reliability f(A _n ) of the score C _n is set to less than 1 (S6). On the other hand, if the event (the position of the person's image P _n ) recognized in S3 is a position that does not affect attribute recognition (No in S5), the attribute determining unit _11c determines the reliability f (A _n ) is set to 1 (S7). The above reliability f(A _n ) is a value set corresponding to the recognition result in S5, that is, the recognition result of an event that affects the recognition of attribute A _n . It does not depend on the recognition result (class) of _n (the same applies to the following embodiments).

Next, based on the score C _n calculated in S3 and the reliability f(A _n ) set above, the attribute determination unit 11c assigns the attribute information Q _n (=Q ₁ ) to the attribute A _n class (S8). For example, a value of score C _n ×reliability f(A _n ) is obtained as the attribute information Q _n for each class of “male in his 20s” and “male in his 40s”. The obtained attribute information Q _n is stored in the storage unit 12 in association with the person's identification information.

Here, FIG. 8 shows an example of information obtained for the first frame. In this example, for the person with ID=0001, the score C ₁ (P _20M ) indicating the likelihood that the attribute A ₁ is "male in his 20s" (this attribute class is "P _20M ") is 0.7. , and the score C ₁ (P _40M ) indicating the likelihood that the attribute A ₁ is "male in his 40s" (the class of this attribute is "P _40M ") is 0.01. In the image, the position of the person's image _P1 is the position that affects the recognition of the attribute _A1 (same as the position of (1) in FIG. 4), and the person recognition unit 11a recognizes the person based on the image. As a result of not being able to accurately recognize attribute A ₁ , even though “male in his 40s” was the correct answer, the score C ₁ (P _40M ) for “male in his 40s” was lower than “male in his 20s”. ' score C ₁ (P _20M ) is higher.

Therefore, considering that the _position of the person's image P ₁ is a position that affects the recognition of the attribute _{A 1} , in the _example of FIG. )=f ₁ (A ₁ )=0.2. As a result, for each class, the attribute information Q ₁ obtained by adding the recognition result of the position to the recognition result of the attribute A ₁ , Q ₁ (P _20M )=C ₁ (P _{20M )} for the class of "male in his twenties" ₎ ×f ₁ (A ₁ ) _{= 0.7} × _0.2 =0.14. f ₁ (A ₁ )=0.01×0.2=0.002 is obtained.

Next, the attribute determination unit 11c determines whether or not to continue the processing, that is, whether or not to perform the same processing as above on the next (n+1)-th frame image (S9). Basically, the attribute determination unit 11c determines to continue the processing in S9, and shifts to S10. In S10, the attribute determination device 3 sets n=n+1, and then repeats the processes after S2. That is, the attribute determination device 3 acquires the image of the second frame from the imaging unit 2, and repeats the processing from S2 onward. In this case, in S4, the person identification unit 11b identifies each frame based on the person information of the first frame and the person information of the second frame (for example, the position (movement amount), size, etc. of the person rectangle in each frame). In between, it is determined whether or not the images of the person in the person rectangle are the images of the same person.

After that, similarly, the processing after S2 is repeated also for the images after the (n+2)th frame. Then, for example, the ID of the person recognized based on the (n+k)-th frame image (where k is a natural number of 3 or more) is recognized based on the (n+(k−1))-th frame image. If it is different from the person's ID, the attribute determination unit 11c cannot determine the attribute B for the same person between frames, so in S9 it is determined not to continue the process, and the process proceeds to S11.

FIG. 9 shows an example of information obtained for the second frame. In this example, for the person with ID=0001, the score C _{2 (} P _20M ) indicating the likelihood that the attribute A ₂ is “male in his 20s” is 0.7, as in the first frame. The score C ₂ (P _40M ) indicating the likelihood that the person is a “male in his 40s” is 0.01. In the image, the position of the person image _P2 is the position that affects the recognition of the attribute _A2 (same as the position (2) in FIG. 4), and the person recognition unit 11a recognizes the person based on the image. As a result of not being able to accurately recognize attribute A ₂ , even though "male in his 40s" is the correct answer, the score of "male in his 20s" is lower than the score C ₂ (P _40M ) ' score C ₂ (P _20M ) is higher.

Considering that _the position of the _person image _P2 affects the recognition of the attribute _A2 , _in the example of FIG. )=f ₂ (A ₂ )=0.7. As a result, for each class, the attribute information _Q2 obtained by adding the recognition result of the position to the recognition result of the attribute _A2 is calculated as follows: _Q2 ( _P20M )= _C2 ( _P20M ) )×f ₂ (A _{2 ) =} 0.7× _0.7 = _0.49 . f ₂ (A ₂ )=0.01×0.7=0.007 is obtained. Since the position (2) of the human image has a smaller effect on attribute recognition than the position (1), the reliability f(A ₂ ) of the score C ₂ is replaced by the reliability of the score C ₁ It is set higher than f(A ₁ ).

On the other hand, FIG. 10 shows an example of information obtained for the third frame. In this example, for the person with ID=0001, the score C ₃ (P _20M ) indicating the probability that the attribute A ₃ is “male in his 20s” is 0.05, and the attribute A ₃ is “male in his 40s”. The score C ₃ (P _40M ) indicating certain certainty is 0.9. In the image, the position of the image _P3 of the person is a position that hardly affects the recognition of the attribute _A3 (same as the position (3) in FIG. 4), and the person recognition unit 11a detects the image based on the image. As a result of being able to accurately recognize the person's attribute _A3 , the score _C3 ( _P40M ) of "male in his 40s" is higher than the score _C3 ( _P20M ) of "male in his 20s". ing. The magnitude relationship between these scores C ₃ (P _20M ) and C ₃ (P _40M ) can be said to be a relationship corresponding to the correct answer of “male in his 40s”.

Considering that the position of the image _P3 of the person is a position that hardly affects the recognition of the attribute _A3 , the reliability f( _A3 ) of the score _C3 is changed to f( _A3 )=f ₃ (A ₃ )=1.0. As a result, for each class, the attribute information _Q3 obtained by adding the recognition result of the position to the recognition result of the attribute _A3 is calculated as follows: _Q3 ( _P20M )= _C3 ( _P20M ) ₎ ×f ₃ (A ₃ ) _{= 0.05} × _1.0 =0.05. f ₃ (A ₃ ))=0.9×1.0=0.9 is obtained.

In S11, the attribute determination unit 11c integrates the attribute information Q _n obtained for each frame for each class of the attribute A _n in a plurality of frames, and determines the attribute B of the person based on the result. The steps S5 to S9 and S11 described above correspond to the attribute determination step.

Here, in the above example where the number of frames n is 3, if the attribute B is determined without considering the position (reliability f(A _n )) that affects attribute recognition (comparative example 1) , an evaluation value Z(P _20M )′ indicating the recognition result of “male in his twenties” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P20M )'= _C1 ( _P20M )+ _C2 ( _P20M )+ _C3 ( _P20M )
= 0.7 + 0.7 + 0.05
= 1.45
On the other hand, the evaluation value Z(P _40M )′ indicating the recognition result of “man in his 40s” in the total of 3 frames is calculated by the following formula using the score C _n .
Z( _P40M )'= _C1 ( _P40M )+ _C2 ( _P40M )+ _C3 ( _P40M )
= 0.01 + 0.01 + 0.9
= 0.92
From the above, Z(P _40M )′>Z(P _40M )′, so in this case attribute B is determined to be “male in twenties”. In other words, even though the correct answer is "male in his 40s", attribute B is erroneously determined to be "male in his 20s" in the three-frame total.

On the other hand, as in the present embodiment, when attribute B is determined in consideration of the position (reliability f(A _n )) that affects attribute recognition, the total of three frames of "male in his twenties" The evaluation value Z (P _20M ) indicating the recognition result is calculated by the following formula using the attribute information Q _n .
Z ( _P20M ) = _Q1 ( _P20M ) + _Q2 ( _P20M ) + _Q3 ( _P20M )
= _C1 ( _P20M )* _f1 ( _A1 )+ _C2 ( _P20M )* _f2 ( _A2 )
+ C ₃ (P _20M )・f ₃ (A ₃ )
= 0.14 + 0.49 + 0.05
= 0.68
On the other hand, the evaluation value Z (P _40M ) indicating the recognition result of “man in his 40s” in the total of three frames is calculated by the following formula using the attribute information Q _n .
Z ( _P40M ) = _Q1 ( _P40M ) + _Q2 ( _P40M ) + _Q3 ( _P40M )
= C ₁ (P _40M )·f ₁ (A ₁ )+C ₂ (P _40M )·f ₂ (A ₂ )
+ C ₃ (P _40M )・f ₃ (A ₃ )
= 0.002 + 0.007 + 0.9
= 0.909
From the above, since Z(P _20M )<Z(P _40M ), the attribute determining unit 11c determines that the attribute B of the person is "male in 40's" for the three frames in total. In this case, the determined attribute B matches the correct attribute.

The attribute B determined by the attribute determination unit 11c in S11 is stored in the storage unit 12 (S12; storage step). In S12, instead of storing the attribute B in the storage unit 12, the information of the attribute B is sent to the management server 4 via the communication unit 15 and stored in the storage unit 21 (see FIG. 6) of the management server 4. Alternatively, the information of the attribute B may be stored in both the storage unit 12 and the storage unit 21. FIG.

〔effect〕
As described above, the attribute determination unit 11c adds the recognition result of the attribute A _n by the person recognition unit 11a to the recognition result of the attribute A _n for each frame regarding the person determined to be the same person in each frame. Attribute information Q _n that takes into account the recognition result of the influencing event (here, the position of the person's image P _n in the image) is obtained for each class of the recognized attribute A _n (S5 to S8). As a result, when the attribute determining unit 11c determines the attribute B of the person based on the result of integrating the attribute information Q _n for each class in a plurality of frames (S11), the recognition of the attribute A _n is affected. For frames in which an event occurs, the degree of contribution of the recognition result of attribute A _n to final attribute determination is made relatively small, and for frames in which an event affecting the recognition of attribute A _n does not occur, attribute The degree of contribution of the recognition result of A _n to final attribute determination can be made relatively large. As a result, even if the event that affects the recognition of attribute A _n lasts for several frames (the first and second frames in the above example), the adverse effect of the above few frames on the final determination of attribute B (attribute A _n ) can be reduced, and the attribute B of the person can be accurately determined as a whole (total of a plurality of frames).

Further, the attribute determining unit 11c sets the reliability f(A _n ) of the score C _n corresponding to the recognition result of the event that affects the recognition of the attribute A _n , and the score C calculated by the person recognizing unit 11a. Attribute information Q _n is obtained for each class based on _n and the above reliability f(A _n ). Thus, by setting the reliability f(A _n ) corresponding to the recognition result of the event and obtaining the attribute information Q _n for each class, the attribute B of the recognition result obtained for each frame can be determined. can be appropriately adjusted for each class in accordance with the event, and the attribute B of the person can be determined reliably and accurately.

Further, the reliability f(A _n ) described above is set based on the position of the person's image P _n as an event that affects the recognition of the attribute A _n . Thereby, the attribute determination unit 11c can acquire appropriate attribute information Q _n considering the position of the person image P _n for each frame using the reliability f(A _n ).

Further, the reliability f(A _n ) is set based on whether or not the position of the person's image P _n in the image is the position where the whole body is photographed. As a result, as in the present embodiment, the reliability f(A _n ) differs depending on whether the position of the person's image P _n is the position where the whole body is photographed or not. Appropriate attribute information Q _n corresponding to the position of image P _n can be obtained.

In particular, in the present embodiment, the reliability f(A _n ) when the position of the image P _n of the person in the image is the position where the whole body is photographed (for example, the position of (3) in FIG. 4) is is higher than the reliability f(A _n ) when the position of the person's image P _n is the position where only a part of the whole body is photographed (for example, the position of (1) or (2) in FIG. 4) is set. As a result, the reliability f(A _n ) of the recognition result of a person's attribute A _n that is recognized with high accuracy based on the image of the whole body of the person is increased, and the final attribute B is determined. It is possible to obtain the attribute information Q _n in which the degree of contribution of the recognition result to is increased. On the other hand, for a person's attribute A _n that is recognized with low accuracy based on an image in which only a part of the whole body of the person is captured, the reliability f(A _n ) of the recognition result is lowered, and the final attribute Attribute information Q _n can be obtained in which the degree of contribution of the recognition result to the determination of B is reduced.

Also, the person's attributes (A _n and B) are the person's age and sex. As a result, the age and sex of a person can be determined with high accuracy in total for a plurality of frames. Note that the attribute may be only one of the person's age and gender. Even in this case, by adopting the attribute determination method of the present embodiment described above, it is possible to accurately determine the age or sex of a person in total for a plurality of frames.

The attribute determination device 3 of this embodiment also includes a storage unit 12 that stores the attribute B determined by the attribute determination unit 11c. As a result, for example, an administrator (responsible person) of a store or system can analyze people who visit the store based on the information of the attribute B stored in the storage unit 12 (what age groups often visit the store). , development and sales of products according to the attribute B of the person, and analysis of marketing can be performed.

In addition, the attribute determination system 1 of this embodiment includes the attribute determination device 3 described above and the management server 4. The management server 4 includes a storage unit 21 for storing information sent from the attribute determination device 3, The information includes the attribute B determined by the attribute determination unit 11c of the attribute determination device 3. FIG. As a result, the system administrator (responsible person) can analyze the people visiting the store based on the attribute B information stored in the storage unit 21 of the management server 4 . Further, when there are multiple stores and each store is provided with the attribute determination device 3, the information (attribute B) sent from each attribute determination device 3 is collectively managed (centralized) in the storage unit 21 of the management server 4. management), and based on the stored information, it becomes easy to compare analysis results between multiple stores.

<Embodiment 2>
This embodiment is the same as the first embodiment except that the event that affects the recognition of the attribute A _n includes a person's behavior, and the attribute B is determined in consideration of the above behavior. As will be described later, the behavior of a person can be grasped from the image (image data) of the person in the image. The parts different from the first embodiment will be described below.

FIG. 11 schematically shows images 2a ₁ to 2a ₄ of some frames out of a plurality of temporally different frames. Images 2a ₁ to 2a ₄ in FIG. 11 are images obtained by photographing a person obliquely from above. As shown in the figure, patterns of behavior of a person in a store include walking and stopping, and other patterns such as running and turning. When a person's action involves movement, such as walking, running, or turning, the image of the person tends to blur in the image. In this case, the recognition accuracy of the person's attribute A _n based on the image is likely to deteriorate. On the other hand, when a person's action is a dwelling action (stop walking) such as standing still, the image of the _person hardly blurs in the image. rarely occur.

Therefore _, in the present embodiment, the attribute determination unit 11c determines the reliability f(A _n is set to less than 1 (for example, 0.2), and for actions that do not affect the recognition of the attribute A _n (staying behavior such as stopping), the confidence f(A _n ) is set to 1, and each frame The attribute information Q _n is obtained for each class for each class, and the attribute B of the person is determined based on the result of integrating the obtained attribute information Q _n in a plurality of frames.

FIG. 12 is a flow chart showing the flow of processing in the attribute determination system 1 of this embodiment. In the flowchart of FIG. 12, S3 and S5 in the flowchart of FIG. 7 are replaced with S3-1 and S5-1, respectively. Here, as in Embodiment 1, it is assumed that the person whose attribute is to be determined is "male in his 40s" (assuming that "male in his 40s" is the correct attribute).

In S3-1 (human recognition step), the human recognition unit 11a affects the recognition of the human rectangle R _n , the human attribute A _n , and the human attribute A _n based on the n-th frame image of the human being photographed from above. (human behavior in this case) and calculates a score C _n indicating the likelihood of the recognition result of the attribute A _n (person recognition step).

Here, the recognition of the person's actions and the calculation of the score C _n can be performed by using a pre-learned neural network. In other words, the person recognition unit 11a inputs the data of each of the images 2a ₁ to 2a ₄ to a neural network that has been trained in advance for action recognition, thereby indicating the recognition result of the person's action and its likelihood from the neural network. A score C _n can be output. Therefore, based on the output from the neural network, the person recognition unit 11a determines whether the person's action is an action (behavior accompanied by movement) that affects attribute recognition or a staying action that does not affect attribute recognition. can be recognized. The recognition result and score C _n obtained in S3-1 are stored in the storage unit 12. FIG.

In S5-1, the attribute determination unit 11c sets the reliability f(A _n ) of the score C _n corresponding to the recognition result of the event that affects the recognition of the attribute A _n in S3-1 (S5-1). 1 to S7). In other words, if the event (person's behavior) recognized in S3-1 is an action (behavior accompanied by movement) that affects attribute recognition (Yes in S5-1), the attribute determination unit 11c Corresponding to the result, the reliability f(A _n ) of the score C _n is set to less than 1 (for example, 0.2) (S6). On the other hand, if the event (human behavior) recognized in S3-1 is a staying behavior that does not affect attribute recognition (No in S5-1), the attribute determining unit 11c determines the reliability of the score C _n f(A _n ) is set to 1 (S7).

Next, the attribute determination unit 11c converts the attribute information Q _n (=Q ₁ ) to the attribute A _{n based on the score C n} calculated in S3-1 and the reliability f(A _n ) set above _. (S8). The obtained attribute information Q _n is stored in the storage unit 12 in association with the person's identification information.

The processing after S2 is repeated for the (n+1)-th frame and subsequent images (S9, S10). , it determines not to continue the process, and shifts to S11. In S11, the attribute determination unit 11c integrates the attribute information Q _n obtained for each frame for each class of the attribute A _n in a plurality of frames, and determines the attribute B of the person based on the result. The steps S5-1 to S9 and S11 described above correspond to the attribute determination step.

FIG. 13 shows an example of information obtained for the first frame. In this example, for the person with ID=0001, the score C ₁ (P _20M ) indicating the probability that the attribute A ₁ is "male in his 20s" is 0.7, and the attribute A ₁ is "male in his 40s". The score C ₁ (P _40M ) indicating certain certainty is 0.01. If the action of the person is an action accompanied by movement (walking), the image of the person is blurred in the image, and the person recognition unit 11a cannot accurately recognize the attribute _A1 of the person based on the image. As a result, although "male in his 40s" is the correct answer, the score C ₁ (P _20M ) of "male in his 20s" is higher than the score C ₁ (P _40M ) of "male in his 40s". It's becoming

Considering that the behavior of a _person affects the recognition of attribute A ₁ , in the example of FIG _. 13, the reliability f(A ₁ ) of score C ₁ is set to (A ₁ )=0.2 is set. As a result, for each class, the attribute information Q ₁ obtained by adding the recognition result of the behavior to the recognition result of the attribute A ₁ , Q ₁ (P _20M )=C ₁ (P _{20M )} ×f ₁ (A ₁ ) _{= 0.7} × _0.2 =0.14. f ₁ (A ₁ )=0.01×0.2=0.002 is obtained.

FIG. 14 shows an example of information obtained for the second frame. In this example as well, for the person with ID=0001, the behavior of the person involves movement (walking). The score C ₂ (P _20M ) of “male in his 20s” is higher than the score C ₂ (P _40M ) of “male in his 40s”. Therefore, as in the first frame, considering that the behavior of the person affects the recognition of the attribute A ₂ , the reliability f(A ₂ ) of the score C ₂ is given by f(A ₂ )=f ₂ (A ₂ )=0.2 is set. As a result, for each class, the attribute information _Q2 obtained by adding the recognition result of the behavior to the recognition result of the attribute _A2 is calculated as follows: _Q2 ( _P20M )= _C2 ( _P20M ) )×f ₂ (A ₂ )=0.7×0.2=0.14, and for the class of “men in their 40s”, Q ₂ (P _40M )=C ₂ (P _40M )× f ₂ (A ₂ )=0.01×0.2=0.002 is obtained.

FIG. 15 shows an example of information obtained for the third frame. In this example, for the person with ID=0001, the score C ₃ (P _20M ) indicating the probability that the attribute A ₃ is “male in his 20s” is 0.05, and the attribute A ₃ is “male in his 40s”. The score C ₃ (P _40M ) indicating certain certainty is 0.9. The behavior of the person is a staying behavior (pause), the image of the person does not blur in the image, and the person recognition unit 11a is able to accurately recognize the attribute _A3 of the person based on the image. , the score C ₃ (P _40M ) of the “male in his 40s” is higher than the score C ₃ (P _20M ) of the “male in his 20s”. The magnitude relationship between these scores C ₃ (P _20M ) and C ₃ (P _40M ) can be said to be a relationship corresponding to the correct answer of “male in his 40s”.

Considering that the behavior of a person hardly affects the recognition of attribute _A3 , in the example _of FIG _{. 15, the reliability f(A3 )} of score _C3 is set to (A ₃ )=1.0 is set. As a result, for each class, the attribute information _Q3 obtained by adding the recognition result of the position to the recognition result of the attribute _A3 is calculated as follows: _Q3 ( _P20M )= _C3 ( _P20M ) ₎ ×f ₃ (A _{3 ) =} 0.05× _1.0 =0.05. f ₃ (A ₃ )=0.9×1.0=0.9 is obtained.

In the above example where the number of frames n is 3, if the attribute B is determined without considering the behavior of the person (reliability f(A _n )) that affects attribute recognition (comparative example 2), The evaluation value Z(P _20M )′ of the recognition result of “male in his twenties” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P20M )'= _C1 ( _P20M )+ _C2 ( _P20M )+ _C3 ( _P20M )
= 0.7 + 0.7 + 0.05
= 1.45
On the other hand, the evaluation value Z(P _40M )′ of the recognition result of “male in his 40s” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P40M )'= _C1 ( _P40M )+ _C2 ( _P40M )+ _C3 ( _P40M )
= 0.01 + 0.01 + 0.9
= 0.92
From the above, Z(P _40M )′>Z(P _40M )′, so in this case attribute B is determined to be “male in twenties”. In other words, even though the correct answer is "male in his 40s", attribute B is erroneously determined to be "male in his 20s" in the three-frame total.

On the other hand, as in the present embodiment, when attribute B is determined in consideration of a person's behavior (reliability f(A _n )) that affects attribute recognition, "a male in his twenties ” _is calculated by the following formula using the attribute information Q _n .
Z ( _P20M ) = _Q1 ( _P20M ) + _Q2 ( _P20M ) + _Q3 ( _P20M )
= _C1 ( _P20M )* _f1 ( _A1 )+ _C2 ( _P20M )* _f2 ( _A2 )
+ C ₃ (P _20M )・f ₃ (A ₃ )
= 0.14 + 0.14 + 0.05
= 0.33
On the other hand, the evaluation value Z (P _40M ) of the recognition result of "male in his 40s" in the total of three frames is calculated by the following formula using the attribute information _Qn .
Z ( _P40M ) = _Q1 ( _P40M ) + _Q2 ( _P40M ) + _Q3 ( _P40M )
= C ₁ (P _40M )·f ₁ (A ₁ )+C ₂ (P _40M )·f ₂ (A ₂ )
+ C ₃ (P _40M )・f ₃ (A ₃ )
= 0.002 + 0.002 + 0.9
= 0.904
From the above, since Z(P _20M )<Z(P _40M ), the attribute determining unit 11c determines that the attribute B of the person is "male in 40's" for the three frames in total. In this case, the determined attribute B matches the correct attribute.

As described above, also in the present embodiment, the attribute determination unit _11c adds an event (in this case, a person _'s Attribute information Q _n that takes into account the recognition result of behavior) and recognized attributes A _n are obtained for each class (S5-1 to S8). As a result, when the attribute determining unit 11c determines the attribute B of the person based on the result of integrating the attribute information Q _n for each class in a plurality of frames (S11), the recognition of the attribute A _n is affected. For frames in which an action (behavior accompanied by movement) occurs, the degree of contribution of the recognition result of attribute A _n to final attribute determination is made relatively small, and actions that affect the recognition of attribute A _n occur. For frames without attribute A _n , the degree of contribution of the recognition result of attribute A n to final attribute determination can be made relatively large. As a result, even if the behavior that affects the recognition of attribute A _n lasts for several frames (the first and second frames in the above example), the adverse effect of the above few frames on the final determination of attribute B (attribute A _n ) can be reduced, and the attribute B of the person can be accurately determined as a whole (total of a plurality of frames).

The reliability f(A _n ) described above is set based on an event that affects the recognition of the attribute A _n , that is, the action of the person ascertained from the image of the person in the image. Accordingly, the attribute determination unit 11c can obtain appropriate attribute information Q _n in consideration of the behavior of the person using the reliability f(A _n ).

Also, the reliability f(A _n ) described above is set based on whether or not the action of the person is an action involving movement. As a result, it is possible to acquire the attribute information Q _n according to the behavior of the person by giving a difference in the reliability f(A _n ) depending on whether the behavior of the person involves movement or not. can.

In particular, in the present embodiment, the reliability f(A _n ) when a person's action is an action accompanied by movement is set lower than the reliability f(A _n ) when a person's action is a staying action. ing. As a result, for a person's attribute A _n that is recognized with low accuracy based on an image in which a person's behavior accompanied by movement is captured, the reliability f(A _n ) of the recognition result is lowered, and the final attribute Attribute information Q _n can be obtained in which the degree of contribution of the recognition result to the determination of B is reduced. On the other hand, for a person's attribute A _n that is recognized with high accuracy based on an image of a person's staying behavior, the reliability f(A _n ) of the recognition result is increased, and the final attribute B is determined. It is possible to obtain the attribute information Q _n in which the degree of contribution of the recognition result to is increased.

<Embodiment 3>
This embodiment is the same as the second embodiment except that the posture of the person is included in the events that affect the recognition of the attribute A _n , and the attribute B is determined with the posture further taken into consideration. As will be described later, the posture of the person can be grasped from the image (image data) of the person in the image. Portions different from the second embodiment will be described below. In addition, "posture" refers to the posture of the body, so it is distinguished from "action" that focuses on the presence or absence of movement, but some postures such as standing posture overlap with action (stopping). sometimes.

FIG. 16 schematically shows images 2a ₁₁ to 2a ₁₅ of some frames out of a plurality of temporally different frames. Images 2a ₁₁ to 2a ₁₅ in FIG. 16 are images obtained by photographing a person obliquely from above. As shown in the figure, the posture taken by a person in a store includes, for example, "squatting down" when observing the products at the bottom of the product shelf in the store. An action from standing still to squatting (for example, starting to squat) and an action from squatting to stopping (for example, standing up) can be recognized as actions.

When the posture of the person is "crouching", an image in which the lower half of the body is hidden by the upper half of the body is obtained in an image of the person photographed from above. In addition, when viewed from above, an image in which only a part of the whole body is photographed is obtained in a posture in which a part of the whole body is hidden by the product shelf. In these cases, since the image data of the person is missing (because the image data of the whole body cannot be obtained), the accuracy of recognizing the attribute A _n of the person based on the image tends to decrease. On the other hand, when the posture of the person is such that the whole body is photographed, such as standing still, there is no lack of image data of the person in the photographed image of the _person . almost no decline.

Therefore, in the present embodiment, when the posture of the person affects the recognition of the attribute A _n (for example, a posture in which only a part of the whole body is photographed), as shown in FIG. When the reliability f(A _n ) is set to less than 1, and the posture of the person is a posture that does not affect the recognition of the attribute A _n (for example, a posture in which the whole body is photographed), the reliability f(A _n ) is set to 1, the attribute information Q _n is obtained for each class for each frame, and the attribute B of the person is determined based on the result of integrating the obtained attribute information Q _n in a plurality of frames. As for human behavior, as in the second embodiment, the reliability f(A _n ) is set to less than 1 for behavior involving movement, and the reliability f(A _n ) is set to 1, and attribute information Q _n is obtained for each frame and each class.

FIG. 17 is a flow chart showing the flow of processing in the attribute determination system 1 of this embodiment. In the flowchart of FIG. 17, S3-1 and S5-1 in the flowchart of FIG. 12 are replaced with S3-2 and S5-2, respectively. Here, as in Embodiment 2, it is assumed that the person whose attribute is to be determined is "male in his 40s" (assuming that "male in his 40s" is the correct attribute).

In S3-2 (human recognition step), the human recognition unit 11a affects the recognition of the human rectangle R _n , the human attribute A _n , and the human attribute A _n based on the n-th frame image of the human being photographed from above. (person's behavior and posture in this case), and a score C _n indicating the likelihood of the recognition result of the attribute _An is calculated (person recognition step).

Here, the recognition of the posture of the person and the calculation of the score C _n can be performed by using a pre-learned neural network. In other words, the person recognition unit 11a inputs the data of each of the images 2a ₁ to 2a ₄ to a neural network that has been trained for posture recognition in advance, thereby indicating the recognition result of the person's posture and its likelihood from the neural network. A score C _n can be output. Therefore, based on the output from the neural network, the person recognition unit 11a determines whether the posture of the person is a posture that affects attribute recognition (posture in which only a part of the whole body is photographed) or not. It is possible to recognize whether it is a non-existent posture (a posture in which the whole body is photographed). The recognition result and score C _n obtained in S3-1 are stored in the storage unit 12. FIG.

In S5-2, the attribute determining unit 11c sets the reliability f(A _n ) of the score C _n corresponding to the recognition result of the event affecting the recognition of the attribute A _n in S3-2 (S5- 2 to S7). In other words, if the event (person's behavior, posture) recognized in S3-2 is an event that affects attribute recognition (Yes in S5-2), the attribute determination unit 11c Then, the reliability f(A _n ) of the score C _n is set to less than 1 (S6). On the other hand, if the event (person's behavior, posture) recognized in S3-2 is an event that does not affect attribute recognition (No in S5-2), the attribute determination unit _11c The degree f(A _n ) is set to 1 (S7).

Next, the attribute determining unit 11c converts the attribute information Q _n (=Q ₁ ) to the attribute A _{n based on the score C n} calculated in S3-2 and the reliability f(A _n ) set above _. (S8). The obtained attribute information Q _n is stored in the storage unit 12 in association with the person's identification information.

The processing after S2 is repeated for the (n+1)-th frame and subsequent images (S9, S10). , it determines not to continue the process, and shifts to S11. In S11, the attribute determination unit 11c integrates the attribute information Q _n obtained for each frame in a plurality of frames, and determines the attribute B of the person based on the result. The steps S5-2 to S9 and S11 described above correspond to the attribute determination step.

FIG. 18 shows an example of information obtained for the first and second frames. In this example, for the person with ID=0001, C ₁ (P _20M )=C ₂ (P _20M )=0.8 and C ₁ (P _40M )=C ₂ (P _40M )=0.01. ing. Since the posture of the person in the image is "squatting down" and only a part of the whole body is photographed, the person recognition unit 11a can accurately recognize the attributes _A1 and _A2 of the person based on the image. As a result of not being able to do so, despite the fact that "male in his 40s" is the correct answer, the score of "male in his 20s" was higher than the score C ₁ (P _40M ) and C ₂ (P _40M ) of "male in his 40s". 's scores C ₁ (P _20M ) and C ₂ (P _20M ) are higher.

Therefore, considering that the posture of a person affects the recognition of attributes A ₁ and _{A 2} , in _the example of FIG. (A ₁ )=f ₁ (A ₁ )=0.2, and the reliability f(A ₂ ) of the score C ₂ in the second frame is set to f(A ₂ )=f ₂ (A ₁ )=0 .2. As a result, attribute information Q ₁ obtained by adding the posture recognition result to the attribute A ₁ recognition result of the first frame is Q ₁ (P _20M )=C ₁ (P _20M )×f ₁ (A ₁ )=0. .8 x 0.2 = 0.16 is obtained, and Q ₁ (P _40M ) = C ₁ (P _40M ) x f ₁ (A ₁ ) = 0.01 x 0.2 = 0.002. It is Also, as the attribute information _Q2 obtained by adding the recognition result of the posture to the recognition result of the attribute _A2 in the second frame, _Q2 ( _P20M )= _C2 ( _P20M )× _f2 ( _A2 )=0. 8*0.2=0.16 is obtained, and _Q2 ( _P40M )= _C2 ( _P40M )* _f2 ( _A3 )=0.01*0.2=0.002 is obtained. ing.

FIG. 19 shows an example of information obtained for the third frame. In this example, the person with ID=0001 has a score C ₃ (P _20M ) of 0.05 and a score C ₃ (P _40M ) of 0.9. Since the posture of the person in the image is "stopping" and the whole body is photographed, the person recognition unit 11a was able to accurately recognize the attribute _A3 of the person based on the above image. The score C ₃ (P _40M ) of the “male in his 40s” is higher than the score C ₃ (P _20M ) of the “male in his 20s”. The magnitude relationship between these scores C ₃ (P _20M ) and C ₃ (P _40M ) can be said to be a relationship corresponding to the correct answer of “male in his 40s”.

Considering _{that the} posture of a person has little effect on the recognition of attribute _A3 , in _the example of FIG _. (A ₃ )=1.0 is set. As a result, for each class, the attribute information _Q3 obtained by adding the recognition result of the position to the recognition result of the attribute _A3 is calculated as follows: _Q3 ( _P20M )= _C3 ( _P20M ) ₎ ×f ₃ (A ₃ ) _{= 0.05} × _1.0 =0.05. f ₃ (A ₃ )=0.9×1.0=0.9 is obtained.

In the above example where the number of frames n is 3, if the attribute B is determined without considering the posture of the person (reliability f(A _n )) that affects attribute recognition (comparative example 3), The evaluation value Z(P _20M )′ of the recognition result of “male in his twenties” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P20M )'= _C1 ( _P20M )+ _C2 ( _P20M )+ _C3 ( _P20M )
= 0.8 + 0.8 + 0.05
= 1.65
On the other hand, the evaluation value Z(P _40M )′ of the recognition result of “male in his 40s” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P40M )'= _C1 ( _P40M )+ _C2 ( _P40M )+ _C3 ( _P40M )
= 0.01 + 0.01 + 0.9
= 0.92
From the above, Z(P _40M )′>Z(P _40M )′, so in this case attribute B is determined to be “male in twenties”. In other words, even though the correct answer is "male in his 40s", attribute B is erroneously determined to be "male in his 20s" in the three-frame total.

On the other hand, as in the present embodiment, when attribute B is determined in consideration of a person's posture (reliability f(A _n )) that affects attribute recognition, "a male in his twenties" in three frames in total ” _is calculated by the following formula using the attribute information Q _n .
Z ( _P20M ) = _Q1 ( _P20M ) + _Q2 ( _P20M ) + _Q3 ( _P20M )
= _C1 ( _P20M )* _f1 ( _A1 )+ _C2 ( _P20M )* _f2 ( _A2 )
+ C ₃ (P _20M )・f ₃ (A ₃ )
= 0.16 + 0.16 + 0.05
= 0.37
On the other hand, the evaluation value Z (P _40M ) of the recognition result of "male in his 40s" in the total of three frames is calculated by the following formula using the attribute information _Qn .
Z ( _P40M ) = _Q1 ( _P40M ) + _Q2 ( _P40M ) + _Q3 ( _P40M )
= C ₁ (P _40M )·f ₁ (A ₁ )+C ₂ (P _40M )·f ₂ (A ₂ )
+ C ₃ (P _40M )・f ₃ (A ₃ )
= 0.002 + 0.002 + 0.9
= 0.904
From the above, since Z(P _20M )<Z(P _40M ), the attribute determining unit 11c determines that the attribute B of the person is "male in 40's" for the three frames in total. In this case, the determined attribute B matches the correct attribute.

As described above, also in the present embodiment, the attribute determination unit _11c adds an event (in this case, a person _'s Attribute information Q _n that takes into account the recognition result of posture) is obtained for each class (S5-2 to S8). As a result, when the attribute determining unit 11c determines the attribute B of the person based on the result of integrating the attribute information Q _n for each class in a plurality of frames (S11), the recognition of the attribute A _n is affected. For frames in which _a pose occurs, the degree of contribution of the recognition result of attribute A _n to final attribute determination is made relatively small. The degree of contribution of the recognition result of A _n to final attribute determination can be made relatively large. As a result, even if the posture that affects the recognition of attribute A _n lasts for several frames (the first and second frames in the above example), the adverse effect of the above few frames on the final determination of attribute B (attribute A _n ) can be reduced, and the attribute B of the person can be accurately determined in total for a plurality of frames.

The reliability f(A _n ) described above is set based on an event that affects the recognition of the attribute A _n , that is, the posture of the person ascertained from the image of the person in the image. Accordingly, the attribute determination unit 11c can obtain appropriate attribute information Q _n in consideration of the posture of the person using the reliability f(A _n ).

Further, the reliability f(A _n ) is set based on whether or not the posture of the person in the image is a posture in which only a part of the whole body is photographed. As a result, the attribute information Q _n corresponding to the posture of the person is acquired by giving a difference in reliability f(A _n ) depending on whether the posture of the person is a posture in which the whole body is photographed or not. be able to.

In particular, in the present embodiment, the reliability f(A _n ) when the posture of a person in an image is a posture in which only a part of the whole body is photographed is calculated as follows: It is set to be lower than the reliability f(A _n ) in the case of the posture that is in the normal position. As a result, for a person's attribute A _n that is recognized with low accuracy based on an image in which only a part of the whole body is captured, the reliability f(A _n ) of the recognition result is lowered, and the final attribute B It is possible to obtain the attribute information Q _n in which the degree of contribution of the recognition result to the determination of is reduced. On the other hand, for a person's attribute A _n that is recognized with high accuracy based on an image of the whole body, the reliability f(A _n ) of the recognition result is increased, and the above recognition for the final determination of attribute B is performed. It is possible to obtain attribute information Q _n with a higher degree of contribution of the result.

<Embodiment 4>
This embodiment is similar to the first embodiment in that the position of a person is considered as an event that affects the recognition of the attribute A _n . That is, the attribute B is determined in consideration of the positional relationship of each person rectangle, which is different from the first embodiment. The parts different from the first embodiment will be described below.

FIG. 20 schematically shows an arbitrary frame image 2a of two persons photographed from above. For example, in a store (real space), if two people are physically in close contact, or if one person is not in close contact with the other and is covering the other, the two people In the image 2a obtained by photographing from above, images Pa and Pb of two persons overlap each other as shown in the figure. As a result, as shown in FIG. 21, the person rectangles Ra and Rb that define the positions of the images Pa and Pb of the two persons also overlap each other in the image 2a. In this case, of the information within the person rectangle Ra, the information of the portion overlapping the person rectangle Rb affects the recognition of the attribute of the person corresponding to the image Pb within the person rectangle Rb. Similarly, of the information within the person rectangle Rb, the information of the portion overlapping the person rectangle Ra affects the recognition of the attribute of the person corresponding to the image Pa within the person rectangle Ra. As a result, there is a possibility that the recognition accuracy of the attributes of both persons will decrease. On the other hand, when the person rectangles Ra and Rb are separated from each other in the image 2a, the information in the person rectangles Ra and Rb do not mutually affect the attribute recognition of each person. Accuracy is improved.

Therefore, in the present embodiment, the attribute determination unit 11c determines that one person rectangle Ra that defines the position of the person image Pa in the image 2a is replaced by another person rectangle Rb that defines the position of another person image Pb. If one person rectangle Ra _is separated from another person rectangle Rb, the reliability f(A _n ) is set to 1 , the attribute information Q _n is obtained for each class for each frame, and the attribute B of the person is determined based on the result of integrating the obtained attribute information Q _n in a plurality of frames.

FIG. 22 is a flow chart showing the flow of processing in the attribute determination system 1 of this embodiment. In the flowchart of FIG. 22, S3 and S5 in the flowchart of FIG. 7 are replaced with S3-3 and S5-3, respectively. Here, as in Embodiment 1, it is assumed that the person (person with ID=0001) whose attribute is to be determined is "male in his 40s" ("male in his 40s" is the correct attribute). assuming there is).

In S3-3 (person recognition step), the person recognition unit 11a uses the same method as in the first embodiment, based on the n-th frame image of the two persons photographed from above, to identify the images of the two persons. , the person rectangle R _na of one person, the attribute A _n of the person corresponding to the image within the person rectangle R _na , and an event that affects the recognition of the attribute A _n (here, the person R _nb ) are recognized, and a score C _n indicating the certainty of the recognition result of the attribute A _n is calculated (person recognition step). The obtained recognition results and scores C _n are stored in the storage unit 12 . In S4, the identification number of ID=0001 is given to the person of one person rectangle _Rna , and the identification number of ID=0002 is given to the person of another person rectangle _Rnb .

In S5-3, the attribute determination unit 11c determines the recognition result of the event that affects the recognition of the attribute A _n in S3-3 (whether or not another person rectangle R _nb overlaps one person rectangle R _na ). The reliability f(A _n ) of the score C _n is set corresponding to (S5-3 to S7). That is, in the image, one person rectangle R _na and another person rectangle R _nb overlap each other, and the other person rectangle R _nb corresponds to the image within the one person rectangle R _na (ID=0001). (Yes in S5-3), the attribute determining unit 11c sets the reliability f(A _n ) of the score C _n to less than 1 (S6) . On the other hand, one person rectangle _Rna and another person rectangle _Rnb are separated from each other in the image, and the other person rectangle _Rnb affects the attribute recognition of the person corresponding to the image in one person rectangle _Rna . is not given (No in S5-2), the attribute determining unit 11c sets the reliability f(A _n ) of the score C _n to 1 (S7).

Next, the attribute determination unit 11c converts the attribute information Q _n (=Q ₁ ) to the attribute A _{n based on the score C n} calculated in S3-3 and the reliability f(A _n ) set above _. (S8). The obtained attribute information Q _n is stored in the storage unit 12 in association with the person's identification information (ID=0001).

The processing after S2 is repeated for the (n+1)-th frame and subsequent images (S9, S10), and when the continuation of the processing becomes unnecessary, the process proceeds to S11. In S11, the attribute determination unit 11c integrates the attribute information Q _n obtained for each frame in a plurality of frames, and determines the attribute B of the person based on the result. The steps S5-3 to S9 and S11 described above correspond to the attribute determination step.

FIG. 23 shows an example of information obtained for the first and second frames. In this example, for the person with ID=0001, C ₁ (P _20M )=C ₂ (P _20M )=0.7 and C ₁ (P _40M )=C ₂ (P _40M )=0.01. ing. The person rectangles _Rna and _Rnb overlap in the image, and the person recognition unit 11a accurately recognizes the attributes _A1 and _A2 of the person (ID=0001) corresponding to the image within the person rectangle _Rna . As a result, despite the fact that "male in his 40s" was the correct answer, the score of "male in his 20s" was higher than the score C ₁ (P _40M ) and C ₂ (P _40M ) of "male in his 40s". Scores C ₁ (P _20M ) and C ₂ (P _20M ) are higher.

Considering that the person rectangle R _nb is located at a position that affects the recognition of the attributes A ₁ and A ₂ of the person (ID=0001), in the example of _FIG . The degree f(A ₁ ) is set to f(A ₁ )=f ₁ (A ₁ ) 0.1, and the reliability f(A ₂ ) of the score C ₂ in the second frame is set to f(A ₂ )= It is set to f ₂ (A ₂ )=0.1. As a result, as attribute information _Q1 obtained by adding the recognition result (position) of the person rectangle _Rnb to the recognition result of the attribute _A1 of the first frame, _Q1 ( _P20M )= _C1 ( _P20M )× _f1 (A ₁ )=0.7×0.1=0.07 is obtained and Q ₁ (P _40M )=C ₁ (P _40M )×f ₁ (A ₁ )=0.01×0.1 = 0.001 is obtained. Also, as attribute information _Q2 obtained by adding the recognition result (position) of the person rectangle _Rnb to the recognition result of the attribute _A2 in the second frame, _Q2 ( _P20M )= _C2 ( _P20M )× _f2 ( A ₂ ) = 0.7 x 0.1 = 0.07 and Q ₂ (P _40M ) = C ₂ (P _40M ) x f ₂ (A ₂ ) = 0.01 x 0.1 = 0.001 is obtained.

FIG. 24 shows an example of information obtained for the third frame. In this example, for the person with ID=0001, the score C ₃ (P _20M ) is 0.01 and the score C ₃ (P _40M ) is 0.9. In the image, the person rectangles _Rna and _Rnb are separated from each other, and the person recognition unit 11a recognizes the person's attribute _A3 based on the image (the image of the person (ID=0001) in the person rectangle _Rna ). As a result of being able to do this with good accuracy, the score C ₃ (P _40M ) for the "male in his 40s" is higher than the score C ₃ (P _20M ) for the "male in his 20s". The magnitude relationship between these scores C ₃ (P _20M ) and C ₃ (P _40M ) can be said to be a relationship corresponding to the correct answer of “male in his 40s”.

Considering that the person rectangle _Rnb is _a position that hardly affects the recognition of the attribute _A3 of the person (ID=0001), in the example of FIG _. , f(A ₃ )=f ₃ (A ₃ )=1.0. As a result, for each class, the attribute information _Q3 obtained by adding the recognition result of the person rectangle _Rnb to the attribute _{A3 of} the person (ID=0001) is obtained. (P _20M ) = C ₃ (P _20M ) x f ₃ (A ₃ ) = 0.01 x 1.0 = _0.01 . _40M )= _C3 ( _P40M )* _f3 ( _A3 )=0.9*1.0=0.9 is obtained.

In the above example where the number of frames n is 3, if the attribute B is determined without considering the position (reliability f(A _n )) of the person rectangle R _nb that affects attribute recognition (comparative example 4 and ), and the evaluation value Z(P _20M )′ of the recognition result of “male in his twenties” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P20M )'= _C1 ( _P20M )+ _C2 ( _P20M )+ _C3 ( _P20M )
= 0.7 + 0.7 + 0.01
= 1.41
On the other hand, the evaluation value Z(P _40M )′ of the recognition result of “male in his 40s” in the total of three frames is calculated by the following formula using the score C _n .
Z( _P40M )'= _C1 ( _P40M )+ _C2 ( _P40M )+ _C3 ( _P40M )
= 0.01 + 0.01 + 0.9
= 0.92
From the above, Z(P _40M )′>Z(P _40M )′, so in this case attribute B is determined to be “male in twenties”. In other words, even though the correct answer is "male in his 40s", attribute B is erroneously determined to be "male in his 20s" in the three-frame total.

On the other hand, as in the present embodiment, when the attribute B is determined in consideration of the position (reliability f(A _n )) of the person rectangle R _nb that affects attribute recognition, " The evaluation value Z (P _20M ) of the recognition result of "male in his twenties" is calculated by the following formula using the attribute information Q _n .
Z ( _P20M ) = _Q1 ( _P20M ) + _Q2 ( _P20M ) + _Q3 ( _P20M )
= _C1 ( _P20M )* _f1 ( _A1 )+ _C2 ( _P20M )* _f2 ( _A2 )
+ C ₃ (P _20M )・f ₃ (A ₃ )
= 0.07 + 0.07 + 0.01
= 0.15
On the other hand, the evaluation value Z (P _40M ) of the recognition result of "male in his 40s" in the total of three frames is calculated by the following formula using the attribute information _Qn .
Z ( _P40M ) = _Q1 ( _P40M ) + _Q2 ( _P40M ) + _Q3 ( _P40M )
= C ₁ (P _40M )·f ₁ (A ₁ )+C ₂ (P _40M )·f ₂ (A ₂ )
+ C ₃ (P _40M )・f ₃ (A ₃ )
= 0.001 + 0.001 + 0.9
= 0.902
From the above, since Z(P _20M )<Z(P _40M ), the attribute determining unit 11c determines that the attribute B of the person is "male in 40's" for the three frames in total. In this case, the determined attribute B matches the correct attribute.

As described above, also in _the present embodiment, the attribute determining unit _11c adds an event (person's position ( In particular, the attribute information Q _n that takes into account the recognition results of the person rectangles R _na and R _nb overlapping)) is obtained for each class (S5-3 to S8). As a result, when the attribute determining unit 11c determines the attribute B of the person based on the result of integrating the attribute information Q _n for each class in a plurality of frames (S11), the recognition of the attribute A _n is affected. For the frame in which the event occurred (the frame in which the person rectangles _Rna and _Rnb overlap in the image), the contribution of the attribute A _n to the final attribute determination of the recognition result is made relatively small, and the attribute A For frames in which an event affecting the recognition of _n does not occur (frames in which the person rectangles _Rna and _Rnb are separated in the image), the degree of contribution of the recognition result of attribute A _n to final attribute determination is It can be relatively large. As a result, even if the event that affects the recognition of the attribute A _n (overlapping of the person rectangles R _na and R _nb ) continues over several frames (the first and second frames in the above example), the final attribute B It is possible to reduce the adverse effect of the above several frames on the determination of (influence of deterioration in the recognition accuracy of attribute A _n ), and accurately determine the attribute B of the person in the total of a plurality of frames.

Further, the reliability f(A _n ) is set based on whether or not one person rectangle R _na overlaps another person rectangle R _nb in the image. As a result, the reliability f(A _n ) differs depending on whether or not one person rectangle R _na overlaps another person rectangle R _nb in the image, and the person rectangle R _na · Attribute information Q _n corresponding to the position of R _nb can be obtained.

In particular, in the present embodiment, the reliability f(A _n ) when one person rectangle R _na overlaps another person rectangle R _nb in an image is _calculated as follows. It is set lower than the reliability f(A _n ) when far from R _nb . As a result, based on the image in which the person rectangles R _na and R _nb are superimposed, the reliability f(A _n ) of the recognition result for the attribute A _n of the person recognized with low accuracy is lowered, and the final Attribute information Q _{n in} which the degree of contribution of the recognition result to determination of attribute B is reduced can be obtained. On the other hand, for a person attribute A _n that is recognized with high accuracy based on an image in which the person rectangles R _na and R _nb are distant, the reliability f(A _n ) of the recognition result is increased, and the final attribute It is possible to obtain attribute information Q _n in which the degree of contribution of the recognition result to the determination of B is increased.

<Program and recording medium>
The attribute determination device 3 described in each of the above embodiments can be configured by, for example, a computer (PC) in which a predetermined program (application software) is installed. A computer (for example, the control unit 16 as a CPU) reads and executes the above program, so that each unit of the attribute determination device 3 can be operated to execute each process (each process) described above. Such a program is acquired by, for example, downloading from the outside via a network and stored in the storage unit 12 . The program is recorded in a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory), and the computer reads the program from the recording medium and stores the program in the storage unit 12. may

<Supplement>
By combining the above embodiments, it is possible to determine the attributes of a person in total for a plurality of frames. For example, as an event that affects the recognition of attributes, a reliability level is set by appropriately combining a person's position, action, and posture, and attribute information is obtained for each frame based on the above reliability level and score, Attributes may be determined by integrating attribute information.

The attribute determination device, attribute determination system, and attribute determination method of this embodiment described above may be expressed as follows. Further, the contents described in this embodiment include the following programs and recording media.

1. An attribute determining device for determining an attribute of a person based on an image of each frame photographing the person from above,
A person who recognizes, for each frame, based on the image of each frame, person information indicating information of the image of the person in the image, attributes of the person, and events affecting recognition of the attributes. a recognition unit;
a person identification unit that determines whether or not the image of the person is the image of the same person between frames based on the person information of each frame;
With respect to the person whose images are determined to be the same person in each frame, the recognized attribute information obtained by adding the recognition result of the event to the recognition result of the attribute is added for each frame. an attribute determination unit that determines the attributes of the person based on results obtained for each class of attributes and integrating the attribute information in a plurality of frames for each of the classes. .

2. The person recognition unit calculates a score indicating the likelihood of the recognition result of the attribute based on the image of each frame,
The attribute determination unit sets the reliability of the score corresponding to the recognition result of the event, and based on the score calculated by the person recognition unit and the reliability, the 2. The attribute determination device according to 1 above, which obtains attribute information.

3. the event includes the position of the image of the person within the image;
3. The attribute determination device according to 2, wherein the reliability is set based on the position of the image of the person.

4. 4. The attribute determination device according to 3 above, wherein the reliability is set based on whether or not the position of the image of the person in the image is a position where the whole body is photographed.

5. The reliability when the position of the image of the person in the image is the position where the whole body is photographed is the position where the image of the person in the image is the position where only a part of the whole body is photographed. 5. The attribute determination device according to 4 above, wherein the reliability is set higher than the reliability in a certain case.

6. The reliability is set based on whether or not one person rectangle defining the position of the image of the person overlaps another person rectangle defining the position of the image of another person in the image. 4. The attribute determination device according to 3 above, characterized in that:

7. In the image, the confidence when the one person rectangle overlaps with the other person rectangle is lower than the confidence when the one person rectangle is separated from the other person rectangle. 7. The attribute determination device according to 6 above, characterized in that it is set.

8. The event includes the behavior of the person grasped from the image of the person in the image,
8. The attribute determination device according to any one of 2 to 7, wherein the reliability is set based on the behavior of the person.

9. 9. The attribute determination device according to 8, wherein the reliability is set based on whether or not the action of the person involves movement.

10. 10. The attribute according to 9 above, wherein the reliability when the action of the person is an action involving movement is set lower than the reliability when the action of the person is a staying action. decision device.

11. The event includes the posture of the person grasped from the image of the person in the image,
11. The attribute determination device according to any one of 2 to 10, wherein the reliability is set based on the posture of the person.

12. 12. The attribute determination apparatus according to 11 above, wherein the reliability is set based on whether or not the posture of the person in the image is a posture in which only a part of the whole body is photographed. .

13. The reliability when the posture of the person in the image is a posture in which only a part of the whole body is photographed is the reliability when the posture of the person in the image is a posture in which the whole body is photographed. 13. The attribute determination device according to 12 above, wherein the attribute is set lower than the reliability.

14. 14. The attribute determination device according to any one of 1 to 13, wherein the attribute of the person is at least one of age and sex of the person.

15. 15. The attribute determination device according to any one of 1 to 14, further comprising a storage unit that stores the attribute determined by the attribute determination unit.

16. 16. The attribute determination device according to any one of 1 to 15;
including a management server connected to the attribute determination device via a communication line,
The management server comprises a storage unit for storing information sent from the attribute determination device,
The attribute determination system, wherein the information includes the attribute determined by the attribute determination unit of the attribute determination device.

17. An attribute determination method for determining an attribute of a person based on an image of each frame photographing the person from above,
A person who recognizes, for each frame, based on the image of each frame, person information indicating information of the image of the person in the image, attributes of the person, and events affecting recognition of the attributes. a recognition process;
a person identification step of determining whether or not the images of the person between the frames are images of the same person based on the person information of each frame;
With respect to the person whose images are determined to be the same person in each frame, the recognized attribute information obtained by adding the recognition result of the event to the recognition result of the attribute is added for each frame. and an attribute determination step of determining the attributes of the person based on results obtained for each attribute class and integrating the attribute information in a plurality of frames for each class.

18. In the person recognition step, based on the image of each frame, a score indicating the likelihood of the recognition result of the attribute is calculated;
In the attribute determination step, the reliability of the score is set corresponding to the recognition result of the event, and based on the score calculated by the person recognition unit and the reliability, the 18. The attribute determination method according to 17 above, wherein attribute information is obtained.

19. the event includes the position of the image of the person within the image;
19. The attribute determination method according to 18 above, wherein the reliability is set based on the position of the image of the person.

20. 20. The attribute determination method according to 19, wherein the reliability is set based on whether or not the position of the image of the person in the image is a position where the whole body is photographed.

21. The reliability when the position of the image of the person in the image is the position where the whole body is photographed is the position where the image of the person in the image is the position where only a part of the whole body is photographed. 21. The attribute determination method as described in 20 above, wherein the reliability is set higher than the reliability in a certain case.

22. The reliability is set based on whether or not one person rectangle defining the position of the image of the person overlaps another person rectangle defining the position of the image of another person in the image. 20. The attribute determination method according to 19 above, characterized in that

23. In the image, the confidence when the one person rectangle overlaps with the other person rectangle is lower than the confidence when the one person rectangle is separated from the other person rectangle. 23. The attribute determination method according to the above 22, wherein the attribute is set.

24. The event includes the behavior of the person grasped from the image of the person in the image,
24. The attribute determination method according to any one of 18 to 23, wherein the reliability is set based on behavior of the person.

25. 25. The attribute determination method according to 24 above, wherein the reliability is set based on whether or not the action of the person involves movement.

26. 26. The attribute according to 25 above, wherein the reliability when the action of the person is an action involving movement is set lower than the reliability when the action of the person is a staying action. How to decide.

27. The event includes the posture of the person grasped from the image of the person in the image,
27. The attribute determination method according to any one of 18 to 26, wherein the reliability is set based on the posture of the person.

28. 28. The attribute determination method according to 27 above, wherein the reliability is set based on whether or not the posture of the person in the image is a posture in which only a part of the whole body is photographed. .

29. The reliability when the posture of the person in the image is a posture in which only a part of the whole body is photographed is the reliability when the posture of the person in the image is a posture in which the whole body is photographed. 29. The attribute determination method according to 28 above, wherein the attribute is set lower than the reliability.

30. 30. The attribute determination method according to any one of 17 to 29, wherein the attribute of the person is at least one of age and sex of the person.

31. 31. The attribute determination method according to any one of claims 17 to 30, further comprising a storage step of storing said attributes determined by said attribute determination step.

32. 32. An attribute determination program for causing a computer to execute the attribute determination method according to any one of 17 to 31 above.

33. 33. A computer-readable recording medium recording the attribute determination program according to 32 above.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention is applicable to apparatuses, systems, and methods for determining attributes of a person based on each frame image of the person photographed from above.

1 attribute determination system 3 attribute determination device 4 management server 11a person recognition unit 11b person identification unit 11c attribute determination unit 12 storage unit 21 storage unit

Claims (19)

An attribute determining device for determining an attribute of a person based on an image of each frame photographing the person from above,
A person who recognizes, for each frame, based on the image of each frame, person information indicating information of the image of the person in the image, attributes of the person, and events affecting recognition of the attributes. a recognition unit;
a person identification unit that determines whether or not the image of the person is the image of the same person between frames based on the person information of each frame;
With respect to the person whose images are determined to be the same person between frames, the recognized attribute information obtained by adding the recognition result of the event to the recognition result of the attribute is obtained for each frame. an attribute determining unit that determines the attributes of the person based on results obtained for each class of attributes and integrating the attribute information in a plurality of frames for each class;
The person recognition unit calculates a score indicating the likelihood of the recognition result of the attribute based on the image of each frame,
The attribute determination unit sets the reliability of the score corresponding to the recognition result of the event, and multiplies the score calculated by the person recognition unit by the reliability to obtain the 1. An attribute determination device, wherein attribute information is obtained, and the attribute information obtained for each class is integrated in a plurality of frames , and the attribute of the person is determined based on the magnitude relation of the evaluation values for each class . the event includes the position of the image of the person within the image;
2. The attribute determination device according to claim 1, wherein the reliability is set based on the position of the image of the person. 3. The attribute determination device according to claim 2, wherein the reliability is set based on whether the position of the image of the person in the image is a position where the whole body is photographed. The reliability when the position of the image of the person in the image is the position where the whole body is photographed is the position where the image of the person in the image is the position where only a part of the whole body is photographed. 4. The attribute determination device according to claim 3, wherein the reliability is set higher than the reliability in a certain case. The reliability is set based on whether or not one person rectangle defining the position of the image of the person overlaps another person rectangle defining the position of the image of another person in the image. 3. The attribute determination device according to claim 2, wherein: In the image, the confidence when the one person rectangle overlaps with the other person rectangle is lower than the confidence when the one person rectangle is separated from the other person rectangle. 6. The attribute determination device according to claim 5, wherein the attribute is set. The event includes the behavior of the person grasped from the image of the person in the image,
7. The attribute determination device according to any one of claims 1 to 6, wherein the reliability is set based on behavior of the person. 8. The attribute determination device according to claim 7, wherein the reliability is set based on whether or not the action of the person involves movement. 9. The method according to claim 8, wherein the reliability when the action of the person is an action involving movement is set lower than the reliability when the action of the person is a staying action. Attribute determination device. The event includes the posture of the person grasped from the image of the person in the image,
10. The attribute determination device according to any one of claims 1 to 9, wherein the reliability is set based on the posture of the person. 11. The attribute determination according to claim 10, wherein the reliability is set based on whether the posture of the person in the image is a posture in which only a part of the whole body is photographed. Device. The reliability when the posture of the person in the image is a posture in which only a part of the whole body is photographed is the reliability when the posture of the person in the image is a posture in which the whole body is photographed. 12. The attribute determination device according to claim 11, wherein the attribute is set lower than the reliability. 13. The attribute determination device according to any one of claims 1 to 12, wherein the attribute of the person is at least one of age and sex of the person. 14. The attribute determination device according to any one of claims 1 to 13, further comprising a storage unit that stores the attribute determined by the attribute determination unit. an attribute determination device according to any one of claims 1 to 14;
including a management server connected to the attribute determination device via a communication line,
The management server comprises a storage unit for storing information sent from the attribute determination device,
The attribute determination system, wherein the information includes the attribute determined by the attribute determination unit of the attribute determination device. An attribute determination method for determining an attribute of a person based on an image of each frame of the person photographed from above,
A person recognition unit recognizes, based on the image of each frame, person information indicating information of the image of the person in the image, attributes of the person, and an event affecting recognition of the attributes for each frame. a person recognition process for recognizing each
a person identification step in which the person identification unit determines whether or not the images of the person between the frames are images of the same person based on the person information of each frame;
attribute information obtained by adding the recognition result of the event to the recognition result of the attribute for each frame with respect to the person whose image is determined to be the same person in each frame by the attribute determination unit; is obtained for each class of the recognized attribute, and for each class, the attribute of the person is determined based on the result of integrating the attribute information in a plurality of frames,
In the person recognition step, the person recognition unit calculates a score indicating the likelihood of the recognition result of the attribute based on the image of each frame,
In the attribute determination step, the attribute determination unit sets the reliability of the score corresponding to the recognition result of the event, and multiplies the score calculated by the person recognition step by the reliability. , the attribute information is obtained for each class, and the attribute of the person is determined based on the magnitude relationship of the evaluation values for each class, which is obtained by integrating the attribute information obtained for each class in a plurality of frames. attribute determination method. 17. The attribute determination method according to claim 16, wherein said attribute of said person is at least one of age and sex of said person. 18. The attribute determination method according to claim 16, further comprising a storage step of storing the attribute determined by the attribute determination step. An attribute determination program for causing a computer to execute the attribute determination method according to any one of claims 16 to 18.

Images