JP4061405B2 - Face image classification registration device - Google Patents

Description

ここで、各出力ニューロンは、ｎ次元の重みベクトルｍ＝｛ｍ_１，ｍ_２，…ｍ_ｎ｝として定義されている。学習ベクトルとして、Ｘがサンプルとして入力されたとき、全ての重みベクトルｍに対してユークリッド距離の計算を行い、入力ベクトルに対して最も距離の近い出力ニューロンを勝者ニューロンｍ_ｃとして決定する。
【００４１】
自己組織化マップの学習は、周囲の出力ニューロン間の位置関係を考慮しながら行われていくという特徴がある。つまり、出力ニューロンへの学習は、勝者ニューロンｍ_ｃだけに対して行われるのではなく、周囲の出力ニューロンに対しても同時に行われる。ある重みベクトルｍ_ｉに対する自己組織化マップの学習則は次の式（数式２）により表される。
【数２】

このような式の自己組織化マップによる学習の様子は、模式的には図７に示されるようなものとなる。概略を説明すると、この式において、α（ｔ）は、ある時間ｔにおける学習強度を示しており、ｈ_ci（ｔ）は、ある時間ｔにおける勝者ニューロンｍ_ｃが近傍に対して影響を及ぼす範囲を示している。学習は、入力ベクトルを使って繰り返し行われ、学習の第一段階では、大きなα（ｔ）、ｈ_ci（ｔ）を用いて大局的な学習が行われる。これらの値は学習を進めるにつれて減少していき、徐々に学習範囲を狭めながら局所的な微調整が行われる。
【００４２】
このように、出力ニューロンの位置関係を維持しながら学習を行うことによって、入力データ空間における多次元特徴量は、その関係を保ったまま２次元平面に射影される。また、自己組織化マップ上において、隣接ニューロン間の類似度は、ニューロン間の距離によって表されている。したがって、この距離を用いてマップにおけるクラス分けを行うことにより、類似した特徴ごとに分類することができる。
【００４３】
識別しようとする個人の全特徴を用いた自己組織化マップは、顔向きを２次元平面に配置した結果である。この結果を、個人顔マップデータベース３６に保存する。
【００４４】
このように、自己組織化マップを用いて行う分類処理を具体例で説明すると、図８は、２次元平面に分布された１人の配置結果を示しており、多くの点が集まっているＢ点には、特徴を取る際にゼロとされた点が集中している。さらに学習を続けると、図９に示すようになり、分類されていく様子がうかがえる。図９においては、マップ上には特徴の類似度がグレー階調によって表されている。色の黒い部分は隣接するニューロン間の距離が遠く、特徴が類似していないことを表している。
【００４５】
このようにして得られたマップを構成するニューロン間の距離を用いてクラス分けを行い、顔画像データを自動分類するが、ここではｋ−ｍｅａｎｓクラスタリングによって得られたマップを、正面側、横向き、それ以外（後ろ向きや下向きなどで顔領域が少なく特徴量が少ない画像）を想定して、３つに自動分類を行う。後ろ向きなどの画像は顔領域が少ないので、それが集まる領域の反対側が、正面画像になり、中間が横向きの顔画像となる。この場合の分類結果は、図１０に示すようになり、明確に３つの領域に分類されている様子が現れている。
【００４６】
ここで多くの点が集まっているＢ点には、特徴を取る際にゼロとされた点が集中している。これを手がかりとし、Ｂ点から距離の最も遠い集合を選ぶことによって正面顔の集合を決定できる。その場合、エッジ特徴を基にしているので、３つに自動分類された黒色の部分には、後ろ向きの顔画像が、その後ろ向きから連続して変化している横向き顔画像が中央のグレーの部分に、反対側の白色の部分には正面向き付近の顔画像が集まっているものとなっている。
【００４７】
このようにして自動分類を行い、正面付近の顔画像と判断された顔領域の顔画像データを各個人の顔データベースとして保存する。これにより、正面向き付近のみの顔画像を集めた顔データベースを作ることが可能になる。このとき横向きの領域のみを集めれば、横向きの画像群データを作ることが可能である。
【００４８】
次に、図５を参照して、全体の顔から顔分布マップを作成する場合について、各個人に依存しない顔向きマップを作成する処理を説明する。この処理においては、各個人のデータを登録する際に保存しておいた個人の顔向きの特徴ベクトルを全て読み込む（ステップ５１）。次に前述の場合と同様にして、自己組織化による顔マップの自動作成処理を行い、顔向きマップを作成し（ステップ５２）、３つ分類に顔マップの自動クラスタリングを行い、その結果を全顔マップデータベース３５に登録する（ステップ５３，ステップ５４）。このとき、マップ内の特徴点を均等に数十個程度抽出して、比較のためのベクトル群としておく。これら全体のデータベースはリアルタイムでの認識時に利用される。
【００４９】
図６は、実際に個人を識別する場合の処理を説明するフローチャートである。図６に示すフローチャートを参照して、リアルタイムで顔認識する動作を説明する。顔認識時には、オンラインリアルタイムで、画像処理装置１１から画像処理を行った画像が１枚単位で与えられる（ステップ６１）。与えられた画像は、色情報などにより顔領域のみを含む部分を取り出し、それ以外の部分は黒色（Ｒ、Ｇ、Ｂの画素値が０）などの情報量をゼロにする処理を行う（ステップ６２）。次に、この顔領域のみが表示された画像から、顔領域のみの特徴抽出およびエッジ特徴を抽出する（ステップ６３）。このエッジ特徴の画像から、高次局所自己相関特徴によるベクトル化を行い、高次局所自己相関特徴を使った３５次元のベクトルを取得する（ステップ６４）。
【００５０】
次に、全顔マップデータベース３５を参照して、識別を行う全個人から作られた全顔マップデータベース３５の分布マップのどこに（正面向き、横向き、それ以外）近いかを判定する（ステップ６５）。この結果、対象とする正面顔でない場合は、識別処理を行わず次のフレームの画像の処理に戻る。
【００５１】
正面向きと判断された場合は、その顔領域画像から、顔識別のための特徴を抽出して（ステップ６６）、その特徴を個人正面付近顔データベースの特徴と比較して顔識別を行い、その結果を出力する（ステップ６７）。識別の精度を上げるために複数枚の顔を識別する場合には、処理の継続をするかどうかの判断を行って（ステップ６８）、処理を継続する場合には、この処理を繰り返す（ステップ６１〜ステップ６８）。
【００５２】
【発明の効果】
以上、説明したように、本発明の顔画像分類登録装置によれば、顔画像データの登録時には、入力された画像群から顔向き別に分類を行うことができ、あらゆる顔向きの画像の中から正面顔などの顔認識に相応しい顔画像データのみを辞書に登録することができる。また、同時に、顔向きのマップを作成しておくことによって、顔認識時にはそのマップから正面向きであるかを自動に判断し、認識可能な顔画像のみを識別することで認識精度を上げることができる。
【図面の簡単な説明】
【図１】本発明を実施する場合のシステム構成の第１の例を示す図である。
【図２】本発明を実施する場合のシステム構成の第２の例を示す図である。
【図３】顔自動分類装置の詳細な構成を説明するブロック図である。
【図４】個人の顔を学習する場合の処理を説明するフローチャートである。
【図５】全体の顔から顔分布マップを作成する場合の処理を説明するフローチャートである。
【図６】実際に個人を識別する場合の処理を説明するフローチャートである。
【図７】自己組織化マップによる学習の様子を模式的に示す図である。
【図８】２次元平面に分布された１人の配置結果を示す図である。
【図９】２次元平面に分布された特徴点が分類されていく様子を説明する図である。
【図１０】２次元平面に分布された特徴点が分類された分類結果を説明する図である。
【符号の説明】
１０ａ，１０ｂ，１０ｃ，…，１０ｎ カメラ
１１ 画像処理装置
１２ 顔自動分類装置
１３ 顔識別装置
１４ 室内空間（識別対象空間）
１５ ユーザ
１６ 操作対象（ディスプレイ、インターホン、パソコンなど）
３１ 顔領域抽出部
３２ 特徴抽出部
３３ 顔自動分類部
３４ 全体顔向き特徴データベース
３５ 全顔マップデータベース
３６ 個人顔マップデータベース
３７ 個人正面付近顔画像データベース[0001]
BACKGROUND OF THE INVENTION
The present invention Numerous image data taken in a wide space such as indoor space where the user is not restricted at all. Face image classification registration that can extract and register face image data suitable for registration as a face image data dictionary for face image recognition apparatus It is about.
[0002]
[Prior art]
Since facial image recognition technology is a fundamental element technology for a wide range of applications such as human interfaces or security systems, various methods have been developed.
[0003]
The face image recognition processing is performed by processing elements such as face region extraction, creation of a face data dictionary in which feature amounts are extracted from the face region, and identification processing using the face data dictionary. Face image identification methods can be broadly classified from features into methods that use features of the face structure, methods that directly identify face patterns (grayscale images), and methods that use both.
[0004]
The method of using the features of the facial structure is to generate a feature vector by parameterizing the position, shape, and size of feature points such as eyes, nose and mouth, and the similarity to the feature vector of the target person registered in advance This is a method for recognizing a face image by calculating.
[0005]
A method for identifying a grayscale image as it is as a pattern for face recognition is to extract the face image portion from the grayscale image of the face image, then normalize the face position and size, and register the face image registered in advance. This is a method of recognizing a face image based on the pattern similarity to data.
[0006]
The method using both features is a method of using an image of a partial portion of the face as a pattern while utilizing the relationship between the structural features of the face. In order to enable identification of more human faces, a method using both of these features is used.
[0007]
Regardless of which face recognition method is used, in the actual application of face recognition, in addition to the face recognition processing described above, the user registration, that is, the face for recognition of the face image of the user to be registered. Data dictionary generation is indispensable. Face recognition has a large variation in shape and brightness, such as illumination conditions, face orientation, and facial expression changes, and particularly requires a large amount of learning sample images compared to character recognition and the like. Also, the face changes with time. For this reason, in order to maintain a high recognition rate, the face data dictionary needs to be updated at regular intervals.
[0008]
For registration of face images for performing face recognition at a high recognition rate, for example, as shown in Patent Document 1, a face area is extracted from an input image and face data is automatically registered in a dictionary. An image registration apparatus has been proposed. As will be described later, with regard to the pattern recognition technology related to face recognition processing and the technology related to automatic classification of feature data, for example, the principle of “recognition using higher-order local autocorrelation features” is described in Non-Patent Document 1. You can refer to it. Non-Patent Document 2 can be referred to for the principle of creating a self-organizing map using SOM.
[Patent Document 1]
JP-A-10-232934
[Non-Patent Document 1]
Noriyuki Otsu, Takio Kurita, Satoshi Sekida “Pattern Recognition Theory and Application” Asakura Shoten First edition published on July 10, 1996
[Non-Patent Document 2]
T.A. Kohonen's “Self-Organizing Map” Springer Fairlark Tokyo June 15, 1996 First edition issued
[0009]
[Problems to be solved by the invention]
By the way, many of the conventional techniques related to face recognition are, for example, face recognition of a user using a personal computer, face recognition of a person using an automatic teller machine (ATM), It is the face recognition of a person standing in front, or the face recognition of a person passing through a certain narrow space such as the entrance, etc., all sitting in a very narrow place of a specific place, standing, It was face recognition on the premise of passing. In this case, it is assumed that the person to be recognized is basically facing the front, and is not assumed to be looking away or looking at the direction tomorrow.
[0010]
Therefore, the face image registration method there is a method of registering a stationary front face or a method of removing a face image unsuitable for recognition from the front face. In addition, even in the method of creating a dictionary for each angle of face orientation, for example, a face is manually cut out from an image taken with the face of the registrant facing in the direction of the front, 15 ° left and right, and 15 ° up and down. Had made a choice.
[0011]
Furthermore, a method for determining whether or not these can be automatically registered has also been proposed. However, what is assumed is a narrow space as described above, or a face image with a deviation of about 15 degrees to the left or right. Therefore, it is not assumed to select a face image suitable for face recognition from face images taken in a wider space such as the entire room.
[0012]
The present invention has been made to solve such problems, and the object of the present invention is to capture a large number of images taken in a wide space such as an indoor space in which the user is not restricted at all. Painting Face image classification registration that can select and register face images close to the front from image data apparatus Is to provide. Another object of the present invention is to register face image classification that can register face image data for face recognition without any restriction on the user when used for face recognition. apparatus Is to provide.
[0013]
[Means for Solving the Problems]
To achieve the above object, face image classification registration according to the present invention apparatus As the first form, A face image classification and registration device for extracting and registering face image data of a face image close to the front from a large number of image data photographed in a wide space in which the user is not restrained at all. Image capturing means for capturing a large number of image data by photographing a person in various directions, and image processing means for performing basic image processing on the image data captured by the image capturing means to form digital image data, A face area is extracted from the digital image data, an edge feature is extracted from the image data of the face area, a high-order local autocorrelation feature is obtained from the image data of the edge feature, vectorized, and used as a face pattern feature. Using feature extraction means to extract and vectorized data of higher-order local autocorrelation features of the features of the face pattern, secondary by face orientation features Create a map, comprising a face automatic classification means for classifying the features of the face direction of the face pattern, a facial map database for registering the 2-dimensional map by created face orientation wherein It is characterized by this.
[0014]
As a second aspect, The face image classification registration device according to the present invention further comprises a face image database for registering face image data used as a face image data dictionary of the face identification device, wherein the automatic face classification means is a two-dimensional map based on face orientation features. Is created by creating a self-organizing map, and based on the created self-organizing map, the face pattern is classified according to the face orientation characteristics, the face image facing front, the face image facing side, and the other face orientations Only face image data of face patterns classified as front-facing face images are registered in the face image database. It is characterized by this.
[0015]
In these forms, face image classification registration In the equipment If an individual person in the room is identified, Registers face image data of face patterns classified as front-facing face images in the face image database as face image data in the personal face image data dictionary You may make it do.
[0016]
Face image classification registration of the present invention apparatus According to the above, the face area is extracted from the face image data obtained from various directions of the person in the indoor space, and the feature of the face pattern of the face area is extracted. As vector data of higher-order local autocorrelation features The face image data is classified from the extracted features, and for example, a face image group close to the front capable of face recognition is extracted. Here, a self-organizing map is created in order to automatically classify face image data. The face image data is classified based on the created self-organizing map. Since the classification result in this case is a self-organizing map created based on the facial pattern features of the facial area, it is classified according to the facial orientation from the facial area features. It is possible to construct a front face image data dictionary from a dictionary for each face direction, for example, a face image group close to the front face. By using this face image data dictionary, the recognition performance of face recognition can be enhanced.
[0017]
Create face orientation map from face image classification results When, At the time of face recognition, the target face image is recognized from the face orientation map. In After determining whether or not the face image is close to a suitable front, the discrimination performance can be improved by identifying the face image using the image data of the identifiable face image.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment for carrying out the present invention will be specifically described with reference to the drawings. FIG. 1 is a diagram showing a first example of a system configuration when implementing the present invention, and FIG. 2 is a diagram showing a second example of a system configuration when implementing the present invention. FIG. 3 is a block diagram illustrating a detailed configuration of the automatic face classification apparatus.
[0019]
In FIG. 1, 10a, 10b, 10c,..., 10n are cameras, 11 is an image processing device, 12 is an automatic face classification device, 13 is a face identification device, 14 is an indoor space, and 15 is a user. In the large indoor space 14 surrounded by the plurality of cameras 10a, 10b, 10c,..., 10n, the user 15 freely moves in the indoor space 14. At that time, the face image of the user 15 is taken from various directions by a plurality of cameras 10a, 10b, 10c,. A large number of face image data obtained by photographing with a plurality of cameras 10 a, 10 b, 10 c,..., 10 n at regular intervals is sent to the image processing device 11.
[0020]
In the system configuration of FIG. 1, a plurality of cameras 10 a, 10 b, 10 c,..., 10 n take a face image of the user 15 from various directions and send the face image data to the image processing device 11. However, this camera may be a single unit or a single unit, and when it enters the camera's field of view, it captures an image including the user's face at a resolution higher than the required level for facial image recognition. What is necessary is just to be able to send the face image data obtained in this way to the image processing apparatus 11.
[0021]
For this reason, the cameras 10a, 10b, 10c,..., 10n may be general single cameras or stereo cameras. The existing stereo camera itself can be used. For example, DigiCrops from Point Gray or Acadia from Sarnoff Laboratories can be used.
[0022]
The image processing apparatus 11 receives face image data (video) from each camera 10a, 10b, 10c,..., 10n, performs image processing of about 10 to 30 images per second, and includes the face images obtained therefrom. The face image data is sent to the automatic face classification device 12.
[0023]
Further, the system configuration shown in FIG. 1 shows a system configuration in which a plurality of cameras 10a, 10b, 10c,..., 10n are provided so as to cover the entire indoor space 14. For example, the system shown in FIG. As in the configuration example, one camera 17 installed in front of the operation target 16 such as a display, an intercom, or a personal computer may be used for the user 15 who is a target for acquiring a face image.
[0024]
In this case, only one camera 17 may be installed in the direction in which the face position of the user 15 is photographed, and face image data may be obtained from the one camera 17. Since the user 15 is not required to fix the face position, the user 15 who is freely operating the operation target 16 The face The image is taken from various directions by the camera 17 and is acquired as face image data. Face image data obtained from the camera 17 installed in the vicinity of the operation target 16 is sent to the image processing apparatus 11.
[0025]
This is suitable as an embodiment in the case where only one user such as an automatic teller machine (ATM), an interphone, or a personal computer user is targeted. Also in this case, it is assumed that the user 15 does not necessarily look at the camera 17, and the face image of the user 15 is sent to the image processing apparatus 11 as face image data taken from various directions. . The basic mechanism is the same as that of the system configuration example in FIG. The image processing device 11 performs basic image processing such as shading correction on the obtained face image data, and sends the result to the automatic face classification device 12 as digital image data.
[0026]
Face As shown in FIG. 3, the motion classification device 12 includes a face area extraction unit 31, a feature extraction unit 32, an automatic face classification unit 33, an entire face orientation feature database 34, an entire face map database 35, an individual face map database 36, an individual It is composed of a front face database 37.
[0027]
The face area extraction unit 31 is, for example, a processing module that extracts a face area (only the face part of the user) from face image data including a face image. Here, a skin color region using the hue of the face is used to extract only the peripheral portion of the face. Regarding the method of extracting the face area, for example, not only the skin color area but also a number of techniques such as a method using eyes, nose, and mouth have been proposed, and these may be used.
[0028]
The feature extraction unit 32 is a processing module that extracts features suitable for determining the face orientation from the obtained image of only the face portion (image data of the face area). Here, for example, after extracting an edge feature, a higher-order local autocorrelation feature is acquired from the edge image and vectorized to extract the feature of the face pattern. This vectorized feature has a feature that can determine the face orientation. The extracted face pattern of the feature is stored in the whole face orientation feature database 34.
[0029]
The face automatic classification unit 33 is a processing module that creates and automatically classifies a two-dimensional map based on face orientation features using the higher-order local autocorrelation feature vectors extracted by the feature extraction unit 32. The two-dimensional map created here is stored in the whole face map database 35 and the personal face map database 36, and an automatic classification process described later is performed. As a result of the final classification, for each individual, only face data close to the front image that is easy to recognize from the personal face map database 36 is registered in the individual front vicinity face image database 37. Then, at the time of identification for performing face recognition, it is determined from the entire face map database 35 whether the input image is a recognizable face image, and the facial features are sent to the face identification device 13 only when identification is possible, Identify for face recognition. At this time, face data close to the front face image registered in the individual front face image database 37 becomes data of the face image data dictionary. This face data is extracted and used when a face is recognized as necessary.
[0030]
The face identification device 13 here identifies only the face image data that can be identified by the determination in the automatic face classification unit 33. Since the face identification device 13 is not a main part of the present invention, its description is omitted, but a known configuration can be used as it is. Various methods have already been proposed, and these are used. Since the face image data that has passed through the automatic face classification device 12 from the image processing device 11 is already output only to the face pattern close to the front face image and is input to the face identification device 13, it is necessary from the face pattern of the face image data. The feature to be extracted may be extracted and used.
[0031]
4 to 6 are flowcharts for explaining examples of face image classification registration processing according to the present invention, corresponding to applications. FIG. 4 is a flowchart for explaining processing when learning an individual's face. FIG. 5 is a flowchart for explaining processing when a face distribution map is created from the entire face. FIG. 6 is a flowchart for explaining the process for actually identifying an individual.
[0032]
The processing according to these flowcharts is mainly performed by the processing module of the automatic face classification device 12 described with reference to FIG. 3, and is a partially detailed version of FIG.
[0033]
The system configuration in FIG. 1 and the system configuration in FIG. 2 are indoor examples, but the present invention is not limited to this example. For example, the indoor space is replaced with a factory or a public space. Thus, the present invention can be applied to any scene using an image including a head region. Even in this case, all persons who enter and leave the indoor space are photographed by a plurality of cameras, and this imaging output is given to the image processing apparatus 11. The image processing device 11 gives the obtained image including the head to the automatic face classification device 12.
[0034]
A case where an individual's face is learned will be described with reference to FIG. In this case, a certain image group is given as the face image data photographed by the camera. When shooting a face image to be registered as face image data, until now it was essential to stand still with respect to the camera and look at the direction of the camera, but when using the face image classification registration method according to the present invention In this case, it is sufficient that the head is included in the imaging range. Instead, a certain number of image groups (for example, 100 or more) are photographed. This image group is given from the image processing apparatus 11 (step 41).
[0035]
A given image group is a process in which a part including only a face area is extracted based on color information and the like, and the other part is black (R, G, B pixel values are 0). Is performed (step 42). In this processing, an image with a clearly small face area (such as backwards) is entirely black and the information amount is completely zero.
[0036]
Next, the edge feature of the part is extracted from the image displaying only the face area (step 43). From this edge feature image, a 35-dimensional vector is obtained using higher-order local autocorrelation features, and processing for storing this vector in the whole face orientation feature database 34 is performed (step 44). The vectorization process using the higher-order local autocorrelation feature here uses a known processing method. The principle of “recognition using higher-order local autocorrelation features” is described in detail in Non-Patent Document 1.
[0037]
In the present invention, the use of high-order local autocorrelation features that are position-free eliminates the need for strict normalization of the face region, thereby reducing the amount of data processing. However, if the face region is normalized based on the face structure (eyes, nose, mouth, etc.), it can be vectorized separately using the features.
[0038]
Further, in the present invention, since the subject is not restrained at the time of shooting, various face-oriented images are obtained, and these face images are processed, but the face image itself here is a profile from the front face. The features obtained from the face are also changing continuously. Therefore, in order to automatically (unsupervised) arrange the continuously changing image group on the two-dimensional plane, a face map is created using a self-organizing map (step 45).
[0039]
Here, a self-organizing map (SOM) is used to perform automatic face classification. The details of the principle of creating a self-organizing map using SOM are described in detail in Non-Patent Document 2, so detailed description will be omitted, but an outline related to the main part of the present invention will be described. A self-organizing map is a kind of unsupervised learning using Kohonen's learning rules. When a non-linear multi-dimensional feature is given, the self-organizing map shows the distribution of features on a two-dimensional map. Can be expressed continuously. Therefore, in the present invention, the classification of the face image data is automatically performed using the map. On the map, similar features are expressed so as to be gathered nearby, so this property is used to classify face image data for each face direction.
[0040]
A self-organizing map is composed of a set of output neurons. It is represented by the following formula (Formula 1).
[Expression 1]

Here, each output neuron has an n-dimensional weight vector m = {m ₁ , M ₂ , ... m _n }. When X is input as a sample as a learning vector, the Euclidean distance is calculated for all weight vectors m, and the output neuron closest to the input vector is the winner neuron m. _c Determine as.
[0041]
The learning of the self-organizing map is characterized in that it is performed while considering the positional relationship between surrounding output neurons. In other words, learning to the output neuron is the winner neuron m. _c It is not only performed for the output neurons, but also for the surrounding output neurons. A weight vector m _i The learning rule of the self-organizing map for is expressed by the following equation (Equation 2).
[Expression 2]

The state of learning by such a self-organizing map is schematically as shown in FIG. In summary, in this equation, α (t) indicates the learning intensity at a certain time t, and h _ci (T) is the winner neuron m at a certain time t _c Indicates the range that affects the neighborhood. Learning is repeated using the input vector, and in the first stage of learning, a large α (t), h _ci Global learning is performed using (t). These values decrease as learning progresses, and local fine adjustment is performed while gradually narrowing the learning range.
[0042]
In this way, by performing learning while maintaining the positional relationship of the output neurons, the multidimensional feature value in the input data space is projected onto the two-dimensional plane while maintaining the relationship. On the self-organizing map, the similarity between adjacent neurons is represented by the distance between neurons. Therefore, by classifying in the map using this distance, it is possible to classify each similar feature.
[0043]
The self-organizing map using all the features of the individual to be identified is the result of placing the face orientation on a two-dimensional plane. This result is stored in the personal face map database 36.
[0044]
As described above, the classification process performed using the self-organizing map will be described as a specific example. FIG. 8 shows the result of arrangement of one person distributed on a two-dimensional plane, and B is a collection of many points. In the points, the points that were set to zero when taking the feature are concentrated. If the learning is further continued, as shown in FIG. 9, it can be seen that classification is performed. In FIG. 9, the similarity of features is represented by gray gradation on the map. The black part of the color indicates that the distance between adjacent neurons is long and the features are not similar.
[0045]
Classification is performed using distances between neurons constituting the map thus obtained, and face image data is automatically classified. Here, the map obtained by k-means clustering is converted to a front side, sideways, Assuming other cases (images that face backward and face down and have a small face area and a small amount of features), automatic classification is performed. Since an image such as backwards has a small face area, the opposite side of the area where the images gather is a front image, and the middle is a sideways face image. The classification result in this case is as shown in FIG. 10, and it is clearly shown that it is classified into three regions.
[0046]
Here, the points that are set to zero when collecting features are concentrated at the point B where many points are gathered. Using this as a clue, the set of front faces can be determined by selecting the set having the longest distance from point B. In that case, since it is based on the edge feature, the black part automatically classified into three has a rear face image, and a side face image continuously changing from the rear part is a gray part in the center. In addition, face images near the front face are gathered in the white portion on the opposite side.
[0047]
In this way, automatic classification is performed, and face image data of a face area determined to be a face image near the front is stored as a personal face database. This makes it possible to create a face database that collects face images only near the front. At this time, if only the horizontal regions are collected, it is possible to create the horizontal image group data.
[0048]
Next, with reference to FIG. 5, a process of creating a face orientation map that does not depend on each individual when creating a face distribution map from the entire face will be described. In this process, all the feature vectors of the individual face orientation stored when registering the data of each individual are read (step 51). Next, in the same manner as described above, the face map is automatically created by self-organization, a face orientation map is created (step 52), and the face map is automatically clustered into three classifications. It is registered in the face map database 35 (step 53, step 54). At this time, about several tens of feature points in the map are extracted equally and set as a vector group for comparison. These entire databases are used for real-time recognition.
[0049]
FIG. 6 is a flowchart for explaining processing in the case of actually identifying an individual. The operation for recognizing a face in real time will be described with reference to the flowchart shown in FIG. At the time of face recognition, an image processed image is given from the image processing apparatus 11 in units of one sheet in real time online (step 61). For a given image, a part including only the face area is extracted from the color information and the like, and the other part is processed to make the information amount zero, such as black (R, G, B pixel values are 0) (step) 62). Next, the feature extraction and edge feature only of the face area are extracted from the image displaying only the face area (step 63). From this edge feature image, vectorization based on higher-order local autocorrelation features is performed to obtain a 35-dimensional vector using higher-order local autocorrelation features (step 64).
[0050]
Next, with reference to the full face map database 35, it is determined where the distribution map of the full face map database 35 created from all individuals to be identified (front-facing, lateral, or otherwise) is close (step 65). . As a result, if it is not the target front face, the identification process is not performed and the process returns to the next frame image process.
[0051]
If it is determined to be front-facing, a feature for face identification is extracted from the face area image (step 66), and the feature is compared with the features of the personal front-facing face database to identify the face. The result is output (step 67). In order to identify a plurality of faces in order to increase the accuracy of identification, it is determined whether or not to continue the process (step 68). If the process is continued, this process is repeated (step 61). To Step 68).
[0052]
【The invention's effect】
As described above, face image classification registration of the present invention apparatus According to the face image data Can be classified by face direction from the input image group, and face images suitable for face recognition such as front face from images of any face orientation data Can only be registered in the dictionary. At the same time, by creating a face-oriented map, it is possible to automatically determine whether the face is facing from the map during face recognition, and to improve recognition accuracy by identifying only recognizable face images. it can.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a first example of a system configuration when implementing the present invention.
FIG. 2 is a diagram showing a second example of a system configuration when implementing the present invention.
FIG. 3 is a block diagram illustrating a detailed configuration of the automatic face classification device.
FIG. 4 is a flowchart for explaining processing in the case of learning an individual's face.
FIG. 5 is a flowchart illustrating a process for creating a face distribution map from the entire face.
FIG. 6 is a flowchart illustrating a process for actually identifying an individual.
FIG. 7 is a diagram schematically showing a state of learning by a self-organizing map.
FIG. 8 is a diagram showing a placement result of one person distributed on a two-dimensional plane.
FIG. 9 is a diagram for explaining how feature points distributed on a two-dimensional plane are classified.
FIG. 10 is a diagram for explaining a classification result in which feature points distributed on a two-dimensional plane are classified.
[Explanation of symbols]
10a, 10b, 10c, ..., 10n camera
11 Image processing device
12 Automatic face classification device
13 Face recognition device
14 Indoor space (identification target space)
15 users
16 Operation target (display, intercom, PC, etc.)
31 Face area extraction unit
32 Feature extraction unit
33 Face automatic classification part
34 Whole face orientation feature database
35 Full face map database
36 Personal face map database
37 Face image database near individual front

Claims (3)

A face image classification and registration device for extracting and registering face image data of a face image close to the front from a large number of image data taken in a wide space where the user is not restricted in position,
Imaging means for capturing a large number of image data by photographing a person in an indoor space from various directions;
Image processing means for performing basic image processing on the image data captured by the imaging means to form digital image data;
A face area is extracted from the digital image data, an edge feature is extracted from the image data of the face area, a high-order local autocorrelation feature is obtained from the image data of the edge feature, vectorized, and used as a face pattern feature. Feature extracting means for extracting;
Automatic face classification means for creating a two-dimensional map by face orientation features using vectorized data of higher-order local autocorrelation features of the features of the face patterns, and classifying the face patterns according to face orientation features;
A face map database for registering a two-dimensional map with the created face orientation features;
A face image classification registration device comprising: The face image classification registration device according to claim 1, further comprising:
A face image database for registering face image data used as a face image data dictionary of the face identification device;
The automatic face classification means creates a two-dimensional map based on face orientation features by creating a self-organizing map, and classifies the face patterns based on the face orientation features based on the created self-organizing map. The face image data is classified into the face image data, the face image facing sideways, and the face image facing the other face, and only the face image data of the face pattern classified as the face image facing the front is registered in the face image database. Face image classification registration device. In the face image classification registration device according to claim 2,
When a person of a person in the indoor space is specified, the face image data of the face pattern classified as a face image facing the front is registered in the face image database as face image data of a face image data dictionary for each individual. A face image classification registration device characterized by the above.

Images