JP2003317100A - Information terminal device, authentication system, and registering and authenticating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent an authentication result from being different even if photographic conditions change when biological information is extracted by using a camera for individual authentication using a mobile terminal device. <P>SOLUTION: There are provided an input means 1 of inputting biological information of a user, a biological information confirming means 5 of displaying the inputted biological information, and an authenticating means 4 of authenticating a previously registered user according to the inputted biological information. The biological information confirming means 5 is equipped with a display means 2 and an index display means 3 of displaying an index specifying the size and position of the inputted biological information. This index display means 3 has a function of confirming the state of the biological information that the user inputs and a function of specifying the size and position of the biological information inputted when the biological information is inputted. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information terminal device having a function of performing personal authentication using biometric information of a user,
Authentication system and registration / authentication method

[0002]

2. Description of the Related Art Conventional information terminal devices having a function of performing personal authentication and user authentication means in an authentication system are classified into an access token type and a stored data type. The access token type includes a smart card, a credit card, a key, and the like, and the stored data type includes a password, a user name, a personal authentication number, and the like.

The problem with the access token type is that it is easily lost or stolen. On the other hand, the problem of the stored data type is to set easy data because of forgetting or fear of forgetting. Although the safety can be improved by using a combination of both methods, the same problem remains.

The biometrics technology uses the characteristics of the body (biological information) as a means for personal authentication, and can solve the problems of loss and memory as described above. Fingerprints, handprints, faces, irises, retinas, voiceprints, etc. are known as specific biometric information.

For user authentication using a face image,
A portable information processing device, such as a videophone device, equipped with an image pickup unit (TV camera) necessary for realizing the original function of the device, and a mobile phone that realizes a security function by using image data captured from the image pickup unit. A type information processing device (Japanese Patent Laid-Open No. 2000-137809) and the like are known.

In addition, the user authentication technology of the above-mentioned conventional example can also be used by adding an image input / output function and an image transmission function to a mobile phone or a portable personal computer which has been rapidly spread in recent years. It is possible.

[0007]

However, in the above-mentioned conventional example, the face image data of the user (or the characteristic parameter extracted from the face image data) registered in advance and the face image data of the user input at the time of authentication (or Since the user authentication is performed by collating with the feature parameter extracted from the face image data), there are the following problems.

(1) Problem of recognition accuracy For example, when biometric information such as a face is extracted using a camera attached to a portable terminal, lighting conditions change, the background changes, or the direction and distance of the camera that captures one's face. , It becomes difficult to obtain the recognition result that the person is the same as the registered image. In other words, there is a problem that the chance that the person is denied authentication increases compared to the conventional access token type and stored data type.

(2) Security Issues Related to Recognition of Biometric Information For example, when a face image for face recognition is input, another person may impersonate himself / herself by using the photograph of the person himself / herself.

An object of the present invention is to provide an information terminal device having a biometric information input interface for a face image, an authentication system, and a registration / authentication method for solving the above two problems.

[0011]

An information terminal device according to the present invention for solving these conventional problems includes an input means for inputting biometric information of a user and a display means for displaying the input biometric information. A display means for displaying an index for designating the size and position of the input biometric information, the authentication means for personally authenticating a user registered in advance based on the input biometric information. Is characterized by.

By displaying the index for designating the size and position of the biometric information input as described above and displaying the result of the biometric information input, the user can determine how to illuminate and the size of the input face. By providing an input interface that allows you to check deviations in orientation, direction, etc., you can easily adjust lighting conditions, camera direction, distance, face, etc., and obtain biometric information under conditions suitable for user authentication. It becomes possible to import.

Further, the information terminal device and the biometric information input from the information terminal via the communication network are registered in a database, and the biometric information of each individual is registered from the biometric information and each biometric information from the already registered database. By providing the learning means for learning the identification function and the registration server having the biometric information and the system management means for managing the identification function and ID of each individual, a highly accurate authentication system can be realized.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION The information terminal device of the present invention comprises a display means for displaying the input biometric information of the user, and an authentication means for personally authenticating a user registered in advance based on the biometric information. The display means displays an index for designating the size and position of the biometric information.
Thereby, the biometric information can be accurately input.

Further, in the information terminal device of the present invention, the biometric information is either the face image of the user, or the face image of the user or the voice. This allows non-contact input using a camera or a microphone, and does not require a special input device.

In the information terminal device of the present invention, the index defines the positions of both eyes. As a result, there is an effect that a face image having a size and a face orientation suitable for authentication can be input.

Further, the information terminal device of the present invention further comprises an evaluation means for evaluating biological information. This makes it possible to determine the suitability of the extracted biometric information as an authentication target, and to determine the content to be instructed to the user.

The evaluation means of the information terminal device of the present invention has at least one of face position detection, face size detection, face orientation detection, light source direction detection, and blink detection. This makes it possible to determine the suitability of the extracted biometric information as an authentication target, and to determine the content to be instructed to the user.

Further, the information terminal device of the present invention comprises an instruction means for giving an instruction to the user when the biometric information is input,
Further equipped. This allows the user to take appropriate measures to improve the extraction accuracy.

In the information terminal device of the present invention, the instruction means is one of an instruction to blink, an instruction to change the body direction, an instruction to move the face up and down or left and right, and an instruction to move the position. give. With these, it is possible to prevent people from impersonating themselves by using photos, improve the authentication accuracy by changing the condition of the lighting that hits the face, and improve the authentication accuracy due to the vertical or horizontal orientation of the face. You can prevent the decline.

Further, in the instruction means of the information terminal device of the present invention, the instruction given to the user is an instruction for feeding back the detection result in the evaluation means. As a result, the instruction determined by the evaluation means based on the evaluation result can be transmitted to the user.

Further, the information terminal device of the present invention converts the face image into a mirror image and displays it. This makes it easier to align the position of one's face captured from the camera with the center.

The information terminal device of the present invention is any one of a portable information terminal having a communication means, a portable personal computer and a portable telephone. Thereby, the biometric information can be accurately input anywhere by the portable terminal.

The index of the information terminal device of the present invention is 2
It is one or more points, circles, crosses or polygons. This makes it possible to accurately specify the size and position of the biometric information.

The authentication system of the present invention registers the information terminal device of the present invention and the biometric information input from the information terminal device via the communication network in a database, and the biometric information and each of the registered biometric information from the already registered database. The registration server has a learning means for learning the identification function of each individual from the biometric information, and a registration server having a system management means for managing the biometric information and the identification function and ID. This allows it to function as a personal authentication means for accessing services on the network, such as electronic commerce and electronic banking.

Further, in the authentication system of the present invention, the biometric information of an individual is updated at regular intervals. As a result, the biometric information of the individual is updated at regular intervals, and safety can be ensured.

Further, in the authentication system of the present invention, the registration server prompts each information terminal device to update the biometric information of an individual at regular intervals. This enables more secure authentication.

Further, the registration / authentication method of the present invention includes a step of displaying an index for designating the size and position of biometric information, a step of displaying biometric information of the user input based on this index, A step of evaluating whether or not the biometric information is allowed with respect to the index, and if the evaluation result is not within the allowable range, give a user an instruction for improvement, and if it is within the allowable range, register the biometric information or
Instructing at least one of the step of performing authentication processing based on pre-registered information, the instruction to change the face direction after the registration processing or the authentication processing, and the instruction to blink, and recognize whether the change has been made as instructed There is a step of detecting impersonation to the person who performs the authentication and a step of stopping the registration processing or the authentication processing when the impersonation is detected. Thereby, the extraction accuracy of the face image is improved, and the accuracy of registration and authentication can be increased. Moreover, it becomes easy to distinguish even if another person impersonates himself.

Further, in the registration / authentication method of the present invention, at least one of the instruction to change the direction of the face and the instruction to blink is randomly given. This makes it possible to more effectively detect spoofing.

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) FIG. 1 is a functional block diagram of an information terminal device having an authentication function according to Embodiment 1 of the present invention. The information terminal device 6 in FIG. 1 includes an input unit 1 for inputting biometric information, a biometric information confirmation unit 5 for confirming the status of biometric information input by a user, an evaluation unit 7 for evaluating the biometric information input, and a user The biometric information confirming means 5 is provided with an instructing means 8 for giving an instruction according to the evaluation result and an authenticating means 4 for authenticating a user registered in advance based on the evaluated biometric information. Is displayed, and index display means 3 for displaying indexes such as two points for designating the size and position of the input biometric information. Here, the index is a mark for aligning the positions of both eyes, and in addition to two points as shown in FIG. 22 (A), a circle as shown in FIG. 23 (A) or a mark as shown in FIG. 23 (B). Although it is a cross as shown or a polygon as shown in FIG. 23C, it is not limited to these as long as it serves as a mark for determining the position.

The information terminal device 6 of the present invention includes, but is not limited to, a personal digital assistant (PDA), a mobile phone, a portable personal computer and the like.

FIG. 2 shows the configuration of a face personal identification registration / authentication system using a mobile phone 1001 as an example of the information terminal device according to the first embodiment of the present invention, which will be described below.

This system has a structure in which a mobile phone 1001 and a registration server 201 are connected by a network 101. The registration server 201 has a function of performing learning using images registered for face authentication, and includes a system management unit 202, a face image registration / update unit 203, a face image database 204, and a data input / output unit 205. ing. The data input / output unit 205 is a mobile phone 1001.
The data sent from the registration server 201
The mobile phone 1001 is provided with a function of transmitting the result processed in (1).

The system management unit 202 has a function of managing personal information relating to registration of a face image and a registration process, and comprises a personal information management unit 206 and a registration log management unit 207. The personal information management unit 206 has a function of managing the owner name, mobile phone number, user name and user ID as personal information, and the registration log management unit 207
It has a function of managing a user ID, a registered image ID, a registration date, an update date, and a learning result ID. The face image registration / update unit 203 has a function of performing learning using the registered face image and obtaining a function for determining whether or not the input image is the person. The face image database 204 has a function of accumulating registered face images and functions obtained by learning. An IC card 50 can be attached to the mobile phone 1001.

FIG. 3 is a schematic view of a camera-equipped mobile phone 1001 as an information terminal device. In FIG. 3, a camera-equipped mobile phone 1001 includes a speaker 11, a display 12, a camera 13 for capturing a face image, a microphone 14, an antenna 15, a button 16, an IC card 50, I.
It is composed of a C card reading interface 51. The entire data processing of the camera-equipped mobile phone 1001 is performed by the data processing unit 17 shown in FIG. Data processing unit 17
Is composed of a device control unit 18 and a data storage unit 19.

FIG. 5 shows a functional configuration of the camera-equipped mobile phone 1001 according to the first embodiment of the present invention.

In FIG. 5, the data processing unit 17 processes the input data from the camera 13, the microphone 14 and the button 16, or the data input from the IC card 50 through the IC card reading interface 51, and the speaker 11 and the display. It has a function of outputting to the antenna 12 and the antenna 15, and is composed of a device control unit 18 and a data storage unit 19. The device control unit 18 not only processes data using various programs, but also controls the device of the mobile phone 1001. The data storage unit 19 can store various programs used by the device control unit 18, data input from the camera 13 or the microphone 14 or the button 16 and result data processed by the device control unit 18.

The face authentication section 20 has a learned function storage section 22 for storing the result of learning the registered image, and the IC card 50.
The authentication unit 21 is configured to authenticate the face image captured from the camera 13 by using the registered image read from the learning function and the learning result read from the learned function storage unit 22.

Mobile phone 1001 with personal authentication function shown in FIG.
3, the camera 13 constitutes the input means 1, the display 12 constitutes the display means 2 and the index display means 3, the speaker 11 or the display 12 constitutes the instruction means 8, and the data processing section 17 constitutes the evaluation means 7. The face authentication unit 20 constitutes the authentication means 4.

Next, the operation of the first embodiment of the present invention will be described.

First, the registration operation will be described with reference to FIGS. 6A, 7 and 8. FIG. 6A shows a face image registration sequence, which is a command between the mobile phone 1001 and the registration server 201, a face image extraction process 601 in the mobile phone 1001, and a face image in the registration server 201. It is composed of a learning process 602.

In the face image extraction processing 601, the face area extraction processing is performed by template matching. The template matching process is as follows. Face regions are extracted in advance from a plurality of images, and an average vector x _m of the feature vectors x, which are gray patterns of the face region images, is prepared as a standard pattern. The center coordinates (X _c , Y _c : 0 <X _c <M, 0 <Y _c <N) of the input image (N × M pixels) have vertical n pixels and horizontal m pixels (N> n, M> m) The input image is cut out so that it becomes the center of the image to be cut out with the size of, and it is converted into the same size as the standard pattern image. After that, a feature vector x _i consisting of a grayscale pattern is calculated, and the similarity between the feature vector x _m of the standard pattern and the feature vector x _i of the input image (for example, the reciprocal of the Euclidean distance described later) is set to a preset threshold value. If it is above, it is output as the face image extraction result. It is also possible to have an eye extraction function after the face area extraction using the same method.

By operating the button 6 of the mobile phone 1001, the device control section 18 reads the registration program from the data storage section 19 and executes it. However, in order to avoid operation by anyone other than the person himself / herself, the registration program is read out only when the number stored by the person himself / herself is input. The device control unit 18 transmits a registration request 603 to the registration server 201, and when receiving a request acceptance response 604 from the registration server 201, starts the face image extraction processing 601.

On the other hand, the registration server 201 sends the registration request 6
When 03 is received, the system management unit 202 collates personal information and determines whether it is new registration or registration information update. In the case of new registration, the received personal information is added and a new registration log is created. When the preparation for registration is completed, the registration server 201 transmits a request reception response 604 including the registration request reception ID to the mobile phone 1001.

Face image extraction processing 601 shown in FIGS.
2 (A) and FIG. 22 (B) will be described.

FIG. 7 is a flow chart of the face image extraction processing 601. The device control unit 18 switches the display of the display 12 (switches the current display to the camera input display).
(Step 1). At the time of switching to the camera input display, the mirror image of the camera input image which is horizontally inverted is displayed on the display 12. The device control unit 18 displays on the display 12 an index 2217 such as two dots for determining the position of the face or eyes.
Is displayed (step 2). In addition, 2 indicating the position of this eye
It is preferable to determine the interval between the dots and the display position on the screen from the average value of the interval between the eyes of the sample face image.

Then, an instruction is issued so that the face of the registrant input from the camera 13 fits in the entire screen (st
ep3). As an instruction method, an instruction is displayed on the display 12, or an instruction is given by voice using the speaker 11.

Next, the device control section 18 takes in the biometric information (step 4) and performs a matching process between the biometric information and a template prepared in advance (step 5).
The similarity calculated by this process is compared with a predetermined threshold value (step 6). If the similarity is small, it is determined that the biometric information is not normal, and the user changes the face direction for improvement. To move up and down or left and right, change the direction of the body, move the position (step
7) Return to step 4.

If the degree of similarity is large, it is determined that the biometric information is normal, and then an instruction is issued to prevent the person from impersonating the person, such as changing the direction of the face or blinking (step 8). As a result of this instruction, if it is detected that it is spoofing, the registration process is ended, and if it is not detected, the process proceeds to step 12. It is preferable that at least one of the instruction to change the direction of the face and the instruction to blink is randomly given as the instruction to prevent spoofing. This is because it is possible to reduce the chances of preparing for impersonation prevention.

FIG. 22A is an example of an input face image, which is an example in which the input face image is small and deviates from the index 2217 such as two points. The device control unit 18 extracts the face area 2218 and the eyes in the face image extraction processing 601 (step 4), and the distance between the center coordinates 2219 of the extracted eyes and the above-mentioned index 2217 is larger than a preset threshold value. If it is detected that it is also large (step 5), the display 12 or the speaker 11 gives an instruction to move the face up and down or left and right as described above (hereinafter referred to as "face (eye) position detection") (step 7). . next,
A face image of the same size as the index as a template, and
A face image larger than the index, for example, a face image with a size of the index plus a threshold, or a face image with a size smaller than the index, for example, a size of the index minus the threshold, is prepared in advance, and a face image extracted from the input image and a template of each size are prepared. Similarity is calculated between the face image and the template in the same manner as in the case of the template matching described above, and the similarity between the input image and the template having the same size as the index is lower than the similarity between the templates having a large size. Case (step6)
, An instruction is issued from the display 12 or the speaker 11 so as to move away from the camera (step 7), and the similarity between the input image and the template of the same size as the index is
If the degree of similarity between templates of a small size is lower (step 6), an instruction is issued to approach the camera (hereinafter referred to as "face size detection") (step 7). In this way, it is possible to give an instruction by feeding back the detection result. As a method not using such a template, the distance between the extracted center coordinates of the left eye and the index 2217 of the left eye, the center coordinates of the right eye, and the index 22 of the right eye are used.
By detecting whether or not the distance to 17 is larger or smaller than a threshold value, or calculating the ratio of the difference with the threshold value, an instruction to move the face up, down, left, right, approach, or far is issued. be able to.

In the above example, the face size is detected after the face (eye) position is detected, but the reverse procedure is also possible.

Then, an instruction is given from the face image capture (step 4) to the matching process (step 6) so that the input face image has an appropriate position and size for the recognition process as shown in FIG. 22B. Repeat step 7). The index 2217 for determining the position of the face or eyes is not limited as long as it is an index for determining the position.

The operation of the device control section 18 described above is performed by the face (eyes)
This is an example of the case where the detection of the position and the face size is applied, but it is also possible to apply the “face direction detection” as follows.

In the face orientation detection, face images extracted from the input image by preparing face images in a plurality of orientations such as the front face and, for example, upper, lower, left, right, upper left, upper right, lower left, lower right in advance as templates. And the template face image in each direction are calculated in the same manner as described above, and the similarity between the template of the input image and the face image of the front is higher than the similarity between the templates of other directions. If it is low, the user similarly instructs to move the face or the body.

Further, the light source direction is in the front and, for example, above,
By preparing multiple face images with different light source directions such as bottom, left, right, upper left, upper right, lower left, and lower right in advance as templates, the light source direction is detected by the same method as face direction detection, and the light source direction is If it is not in front, it is also possible to give an instruction to move the face or move the body (hereinafter, referred to as “light source detection”).

The above face (eye) position detection, face size detection,
Face direction detection and light source direction detection can be performed sequentially, or all of these templates can be compared at the same time and instructions can be issued collectively.

When the face orientation and the light source direction are detected after the face (eye) position detection and the face size detection, the above-mentioned template matching is performed by the similarity comparison by the KL expansion described later. More preferred.

Furthermore, in the case of having an eye extraction function to prevent registration or authentication by impersonating the person,
It is also possible to issue a blink instruction from the speaker 11 or the display 12 (step 8), and the device control unit 18 can detect the blink (step 9). However, this detection needs to be performed after the face (eye) position detection and the face size detection are performed and the user moves to the correct position.

The blink detection (step 9) is performed as follows. That is, an eye image with open eyes and an eye image with closed eyes are prepared in advance as templates, and the eye images extracted from the temporally continuous input images are respectively described above and with the template image. Similarly, the similarity is calculated, and it is determined based on the value of the similarity whether the input image has the eyes open or the eyes closed. Then, when a time sequence in which the open / closed state of the eyes of the input image is “open eyes → closed eyes → open eyes” is detected, it is determined that blinking has occurred. If no blink is detected in this determination, it is determined that the person is impersonating, and the registration is terminated. Further, the face direction detection and the light source direction detection described above are also effective as a method for preventing this spoofing. By these, for example, it is possible to prevent the person from trying to impersonate using a facial photograph.

As described above, by designating the size of the biometric information, the resolution of the input biometric information image can be
It becomes possible to obtain it with a predetermined value required for authentication.
Further, by designating the position of the biometric information, it becomes possible to accurately extract only the biometric information to be authenticated, and it is possible to obtain good information with less noise. Only by specifying both the size and the position, it becomes possible to obtain biometric information with high resolution, low noise, and optimal for authentication.

Furthermore, by issuing an instruction to move the face up and down or left and right, it is possible to prevent the authentication accuracy from deteriorating due to the direction of the face or the direction of the light source.

Further, by these designations, the size and position of the acquired face image can be made to match at the time of registration and at the time of authentication, which can also improve the authentication performance.

Furthermore, by giving at least one of an instruction to blink, an instruction to change the body direction, an instruction to move the face up and down or left and right, and an instruction to move the position, the photographer is given It will also be possible to control impersonation.

Next, the device control section 18 compresses the input face image (step 10). If the data storage section 19 does not have sufficient storage capacity (step 11), the process proceeds to step 13, and if there is sufficient storage capacity. (Step 11) is temporarily stored in the data storage unit 19 (step 12). And
The face image information is transmitted to the registration server 201 together with the personal information required for registration and the registration request reception ID (step 1).
3). Here, the personal information required for registration is information managed by the personal information management unit 206.

The registration server 201 uses the face image information 60.
When 5 is received, the face image registration process is started.

The face image registration process will be described below. The registration server 201 stores face images in the face image database 204.
And the face image reception response 606 is transmitted to the mobile phone 1001. On the other hand, in the registration server 201 that has transmitted the face image reception response 606, the system management unit 202 passes the registered image ID to the face image registration / update unit 203. The face image registration / updating unit 203, which has received the registered image ID, reads the registered image from the face image database 204 and performs a learning process 602.

A flow chart of the learning process 602 is shown in FIG.

In the learning process 602, first, the vector generated from the registered image is read from the face image database 204, the variance-covariance matrix W of the feature vector x _f consisting of the grayscale pattern of a plurality of face images is used, and the expression (1) is used. The eigenvector l _f is calculated.

(W-λ _j I) l _j = 0 (1) where λ is an eigenvalue and I is an identity matrix.

Furthermore, the eigenvalue contribution rate C _j is calculated from (Equation 2), and the matrix A = (l
_{1, l 2, ····, l} n) a transformation matrix (hereinafter, this transformation matrix is called a learned function) obtained as (step1
4).

C _j = λ _j / tr (W) (2) Further, tr (W) means the trace of the variance-covariance matrix W.

This learned function is a discriminant function used for discriminating the user.

Next, the feature vector y _s of the registered image of the person is generated from the vector x _s of the registered image of the person and (Equation 3). The learned function A for mapping to this eigenspace becomes the learning result (step 15).

Y _s = A ^t x _s (3) The processing of these step 14 and step 15 is called KL expansion (Karhunen-Loeve expansion). When the learning process 602 ends, the face image registration / update unit 203 passes the learning result and the threshold value for the determination to the system management unit 202.
The system management unit 202 assigns the learning result ID to the learning result and a threshold value for determination, and the face image database 20
Accumulate in 4. Further, the system management unit 202 transmits the learning result and the threshold value for determination as a registration completion response 607 to the mobile phone 1001 via the data input / output unit 205.

In the mobile phone 1001, the device control unit 1
8 is a face image reception response 606 from the registration server 201
When the is received, the face image recorded in the data storage unit 19 is erased. Further, when the registration completion response 607 is received, the data processing unit 17 records the received learning result and the threshold value for determination in the learned function storage unit 22. Equipment control unit 1
8 notifies the user of the completion of registration using the speaker 11 or the display 12. Device control unit 18
Ends the registration process and returns to the default state. The default state is the same state as the initial state when the mobile phone 1001 is turned on.

The registration server 201 selects the image of the person from the images stored in the face image database 204.
One sheet is extracted and the registered image or the feature vector of the registered image is written in the IC card 50. At this time, personal information is also written in the IC card 50 in addition to the registered image. The IC card 50 is sent to the person.

The registration server 201 writes the newly input image of the person into the IC card 50 after a certain period of time has elapsed since the registration was made before, or via the mobile phone 1001 every time a certain period of time elapses. It has a function to prompt the user to input the registered image.

Next, the authentication operation will be described with reference to FIGS. 9 (A) and 9 (B).

By operating the button 16 of the mobile phone 1001, the device control section 18 reads the recognition program from the data storage section 19 and executes it. The user also inserts the IC card 50 on which the registered image is recorded into the IC card reading interface 51.

Face image extraction processing 901 and face image recognition processing 9
A flow chart of 02 is shown in FIG.

First, the face image extraction processing 90 is performed as in the registration.
1 is performed (step 21).

Next, face image recognition processing 90 using face images
Do 2. The device control unit 18 instructs the face authentication unit 20 to start the face image recognition processing 902 (step 22). This start instruction (step 22) includes the storage position of the extracted face image. The authentication unit 21 generates a vector of the extracted image (step 23).

Similarly, the device control section 18 displays the registered image on the IC.
The vector of the registered image is read out from the card 50 (step 24). However, if the feature vector of the registered image is generated in advance and recorded in the IC card 50, this process is unnecessary.

The device control section 18 uses the learned function storage section 2
The learned function A and the threshold value for determination are read from 2. Using the registered image vector x _s , the extracted image vector x _i, and the learned function A, the registered image feature vector y _s
From (Equation 3), and the feature vector y _{i of the} extracted image
Are calculated from (Equation 4) (step 25).

Y _i = A ^t x _i (4) The similarity is calculated using the obtained registration image feature vector y _s and the extracted image feature vector y _i , and the similarity is larger or smaller than the threshold value. Whether or not the person is the person is judged. The similarity is obtained, for example, as the reciprocal of the Euclidean distance d of the output result using the feature vectors y _s and y _i which are the results of the KL expansion of the registered image and the input image. The authentication unit 21 transmits the determination result to the device control unit 18 (step 26).

Here, the Euclidean distance d is (Equation 5)
Required by.

[0088]

[Equation 1]

When it is determined that the user is the person, the device control section 18 validates all programs of the mobile phone 1001 (st
ep27). If the person is not identified, step2
Return to 1.

In the first embodiment, the registered image of the person and the threshold value are used to determine whether or not the person is the person, but there is a method that does not use the threshold value. As the registered image, a plurality of images of the person and the image of another person are used, and if the similarity between the extracted image and the image of the person is the largest, it is determined as the person, and if the degree of similarity with the image of the other person is the largest, You may decide to be another person.

Further, when the face image extraction processing of step 21 is performed again, it is possible to give an instruction to move the face up and down, or left and right. This instruction can be issued based on the orientation of the face image detected in the previous authentication processing.

The mobile phone with authentication function 1001 uses the result of the face image recognition processing 902 as a recognition success notification 903.
Send to the registration server 201. Further, as shown in FIG. 9B, when the result of the face image recognition processing 902 is authentication failure, the mobile phone with authentication function 1001 transmits a recognition failure notification 904 to the registration server 201.

Although the mobile phone 1001 having a communication function is used as the information terminal device in the first embodiment, the face image extraction processing 601 shown in steps 1 to 12 of FIG. 7 is not limited to this and has a communication function. Even in a non-information processing device, the same function can be realized and an effect can be obtained. That is, in the face image extraction processing 601 shown in FIG. 7, the device control unit 18 gives an instruction to the user or displays an index on the display so that the authentication result by the authentication unit 21 can be accurately performed. The communication function is not related to the realization.

Further, in the first embodiment, the device control section 18 evaluates the face image extracted from the input image as an information terminal for registration and authentication, and gives an instruction to the user based on the result. The treatment was shown. However, in an information terminal that does not have a registration function but has only an authentication function, the authentication unit 21 has an evaluation function that the device control unit 18 has, and the authentication unit 21 fails in authentication, so that re-authentication is required. It is also possible to perform a process of issuing a similar instruction when inputting an image.

In the first embodiment, the device control section 18
The authentication unit 21 has a function of evaluating the input image, but this function is not essential, and the user can simply instruct the user to make a face or eye match with the index and notify the result of the authentication. .

(Second Embodiment) FIG. 11 is a functional block diagram of a mobile phone with a multiple authentication function according to a second embodiment of the present invention. Further, FIG. 12 is a system configuration diagram showing a registration / authentication system according to the second embodiment of the present invention.
The second embodiment is different from the first embodiment in the configuration of the mobile phone 1002 and the configuration of the registration server 301, and the other configurations are the same. Therefore, in the second embodiment, only the configuration different from the first embodiment will be described with reference to FIGS. 11 and 12.

The difference between the configurations of the mobile phone 1002 and the mobile phone 1001 is that the mobile phone 1002 is additionally provided with a speaker authenticating section 23 for authenticating using the voice of the speaker. The speaker authentication unit 23 stores the learned function storage unit 25 for storing the result of learning the registered voice, the registered voice read by the IC card reading interface 51 from the IC card 50, and the learning result read from the learned function storage unit 25. The authentication unit 24 is used to authenticate the voice of the speaker input from the microphone 14.

The difference between the registration server 301 and the registration server 201 is that the registration server 301 stores a face image in the face image database 204 for storing face images and voices. And that a voice registration / updating unit 303 for performing voice learning processing is added.

Next, the operation of the second embodiment of the present invention will be described with reference to FIG. The face image registration operation is the same as in the first embodiment. Here, a voice registration operation will be described.

FIG. 6B shows a voice registration sequence. A command between the mobile phone 1002 and the registration server 301, a voice extraction process 6 in the mobile phone 1002.
08, and voice learning processing 609 in the registration server 301.

By operating the button 16 of the mobile phone 1002, the device control section 18 reads out the registration program from the data storage section 19 and executes the same as in the face image registration. However, to avoid being operated by anyone other than the
The registration program is read out only when the number entered is stored only by the person himself / herself.

The device control section 18 transmits a registration request 610 to the registration server 301 using voice as biometric information, and when receiving a request acceptance response 611 from the registration server 301, starts the voice extraction processing 608. On the other hand, when the registration server 301 receives the registration request, the system management unit 20
2 collates the personal information and judges whether it is a new registration or an update of the registered information. In the case of new registration, the received personal information is added and a new registration log is created. When the preparation for registration is completed, the registration server 301 transmits a request reception response 611 including a registration request reception ID to the mobile phone 1002.

The voice extraction processing 608 will be described with reference to FIG. The device control unit 18 displays an instruction to start registration on the display 12 or gives an instruction by voice using the speaker 11 (step 51).

The user inputs a voice from the microphone 14 according to the instruction. The device control section 18 compresses the input voice (step 52), and if the data storage section 19 has a sufficient capacity, temporarily stores the compressed voice in the data storage section 19.
(Step 53). The voice information 612 along with the personal information required for registration and the registration request acceptance ID are encrypted using the public key cryptosystem (step 54) and transmitted to the registration server 301 (step 55). However, if the data storage unit 19 does not have a sufficient storage capacity, the accumulation process is not performed.

The registration server 301 converts the voice into a face image /
Recorded in the voice database 302 and received as voice response 613
Is transmitted to the mobile phone 1002. On the other hand, in the registration server 301 that has transmitted the reception response, the system management unit 2
02 passes the registered image ID to the voice registration / update unit 303.
Upon receiving the registration image ID, the voice registration / updating unit 303 reads out the registered voice from the face image / voice database 302 and performs a voice learning process 609.

A flow chart of the voice learning process 609 is shown in FIG. First, a voiceprint graph of registered voice read from the face image / voice database 302 is created (step 10).
1). The voiceprint graph is a vector in which time series data of voice is decomposed into frequency components and arranged in time series. The user selects the words to be used for the registered voice from among the prepared words. The voiceprint graph is subjected to KL expansion in the same manner as in the first embodiment, and a transformation matrix composed of eigenvectors is obtained as a learned function A (step 102).

Next, the feature vector y _s of the registered voice of the person is generated from the vector x _s of the registered voice of the person and (Equation 3). The learned function A for mapping to this eigenspace is used as the learning result (step 103).

When the voice learning process 609 is completed, the voice registration / update unit 303 passes the learning result and the threshold value for the determination to the system management unit 202. The system management unit 202 assigns the learning result ID to the learning result and a threshold value for determination, and stores the learning result ID in the face image / voice database 302. Further, the system management unit 202 sets the learning result and the threshold value for the determination via the data input / output unit 205 as a registration completion response 6
14 is transmitted to the mobile phone 1002.

In the mobile phone 1002, the device control unit 1
When receiving the voice reception response 613 from the registration server 301, the reference numeral 8 erases the voice recorded in the data storage unit 19. Further, when the registration completion response 614 is received, the data processing unit 17 records the received learning result and the threshold value for determination in the learned function storage unit 25. The device control unit 18 notifies the user of the completion of registration using the speaker 11 or the display 12. The device control unit 18 ends the registration process and returns to the default state. The default state is the same state as the initial state when the mobile phone 1002 is turned on.

The registration server 301 uses the face image /
One of the voices of the person (one word) is extracted from the voices stored in the voice database 302, and the registered voice or the feature vector of the registered voice is written in the IC card 50. At this time, personal information is also written in the IC card 50 in addition to the registered voice. The IC card 50 is sent to the person. At this time, if there is a face image that has already been registered, the registration image is recorded together with the registration image on a single IC card
It is also possible to write in.

Next, the authentication operation will be described with reference to FIG. By operating the button 16 of the mobile phone 1002, the device control section 18 reads the recognition program from the data storage section 19 and executes it (step 153).
Further, the user inserts the IC card 50 on which the registered image or the registered voice is recorded into the IC card interface 51 (step 152). The user can select which authentication is used (step 151). The selection is made before reading the recognition program.

When the authentication is successful, the device control section 18 enables all the programs of the mobile phone 1002 (step
p154). If the authentication has failed, it is determined whether or not to continue the process (step 155), and if it is to be continued, the process returns to step 151. Since the authentication operation using the face image has been described in the first embodiment, the speaker authentication operation will be described here.

FIG. 16 shows a flow chart of the speaker recognition process.

First, the voice extraction processing 608 is performed as in the registration.
Is performed (step 201). Next, speaker recognition processing is performed.
The device control unit 18 instructs the speaker authentication unit 23 to start the authentication process (step 202). The start instruction includes the storage position of the extracted voice. The authentication unit 24 generates a vector of the extracted voice graph (step 203). Similarly, the device control unit 18 reads out the registered voice from the IC card 50 and generates a vector of the registered voice (step 2).
04). However, if the feature vector of the registered voice is generated in advance and recorded in the IC card 50, this process is unnecessary.

The device control section 18 uses the learned function storage section 2
The learned function A and the threshold value for determination are read from 5, and the registered voice feature vector and the extracted voice feature vector are obtained from the registered voice vector and the extracted voice vector using the learned function A (step 205). . The similarity is calculated using the obtained feature vector of the registered voice and the feature vector of the extracted voice, and it is determined whether or not the person is the person based on whether the similarity is larger or smaller than a threshold value. The calculation of the similarity uses, for example, the reciprocal of the Euclidean distance of the output result. The authentication unit 24 transmits the determination result to the device control unit 18.
(Step 206).

Note that face image recognition as in the present embodiment,
The cost can be reduced by sharing the algorithm related to speaker recognition.

Further, when the authentication fails and continues (re-authentication), the lighting condition or background inconsistency at the time of registration and authentication, which is one of the factors of the authentication failure, is caused by an instruction such as moving the body. It can be expected to be improved, and there is also an effect that authentication failures due to these factors are not repeated.

(Third Embodiment) The configuration of the third embodiment of the present invention will be described with reference to FIG.

The difference in configuration from the first embodiment is that the registration / authentication server 401 has an authentication function.

In FIG. 17, the mobile phone 1003 and the registration / authentication server 401 are connected by the network 101. The registration / authentication server 401 is a system management unit 4 that manages the entire registration / authentication server 401.
02 and registration of face image Registration / authentication unit 40 for learning and authentication
Face image database 4 that stores face images of users 3 and 3
04, the system management unit 402 includes a personal authentication support unit 4 for manually performing face image authentication.
05 and the registered user's address, name, telephone number, registration date, personal information storage unit 406, authentication date and time, authentication log storage unit 407 including authentication determination, and display 408. The registration / authentication unit 403 performs personal authentication. It is composed of a personal authentication unit 409 for performing the above and a face image registration unit 410 for performing a learning process of the face image.

FIG. 4 shows the functional configuration of the mobile phone 1003.

The mobile phone 1003 includes a speaker 11, a display 12, a camera 13 for capturing a face image,
The microphone 14, the antenna 15, the button 16, the IC card reading interface 51, and the data processing unit 17 are provided. The data processing unit 17 is further provided with a device control unit 18 and a data storage unit 19.

Next, the operation of the third embodiment of the present invention will be described. The registration operation is almost the same as that in the first embodiment, and the registration / authentication server 401 is the registration server 201.
It has all the functions of. Here, only the difference in the registration operation from the first embodiment will be described.

In the first embodiment, the registered image is displayed on the IC card 5.
Although the operation of recording 0 is performed, it is not performed in the third embodiment. Further, in the first embodiment, when the device control unit 18 receives the registration completion response, the data processing unit 17 records the received learning result and the threshold value for determination in the learned function storage unit 22, This operation is not performed in the third embodiment.

Next, the authentication operation will be described with reference to FIG. By operating the button 16 of the mobile phone 1001, the device control unit 18 reads the recognition program from the data storage unit 19 and executes it. First, face image extraction processing 1801 is performed as in the case of registration. Next, the device control unit 18
Authentication request 1802 to registration / authentication server 401
To send. The authentication request 1802 includes the extracted face image.

A flow chart of the face image recognition processing 1804 in the registration / authentication server 401 is shown in FIG. The system management unit 402 outputs the received face image to the registration / authentication unit 403 and gives an instruction to start the authentication processing 1804 (step).
301). The personal authentication unit 409 generates a vector of the extracted face image (step 302). In addition, the personal authentication unit 40
Reference numeral 9 reads the registered image from the face image database 404 and generates a vector of the registered image (step 303).
However, if the feature vector of the registered image is generated in advance and recorded in the face image database 404, this process is not necessary.

The personal authentication unit 409 has a face image registration unit 410.
The learned function A and the threshold value for determination are read from the registered image vector and the extracted image vector, and the learned function A is used to calculate the feature vector of the registered image and the feature vector of the extracted image (Equation 3). And (Equation 4), respectively (step 304). The similarity is calculated using the obtained feature vector of the registered image and the feature vector of the extracted image, and it is determined whether or not the person is the person depending on whether the similarity is larger or smaller than a threshold value (step 305). The calculation of the similarity uses, for example, the reciprocal of the Euclidean distance of the output result.

When the face image recognition processing 1804 is completed, the registration / authentication server 401 stores the result thereof in the mobile phone 1.
The recognition response 1803 is transmitted to 001.

If this authentication is successful, the mobile phone 100
The device control unit 18 of No. 1 enables all the programs of the mobile phone 1001. If the authentication fails, the user has three choices. One is face image extraction 1 again
Step 801 is to perform authentication, one is to send an authentication support request to the registration / authentication server 401, and one is to stop the face image authentication 1804 and switch to authentication by ID input. Since the face image authentication 1804 may fail in recognition depending on the lighting condition and the orientation of the face, there is a possibility that the authentication will succeed by changing the lighting condition and performing the authentication again. Although a response time is delayed by making an authentication support request as described below, a third party on the registration / authentication server 401 side is surely authenticated, so the security is high. Further, when switching to authentication by inputting an ID, it takes time for the user, but reliable authentication can be expected.

Here, the operation when making an authentication support request will be described. Mobile phone I for authentication support request
Information such as D, authentication log, and urgency is included. Registration /
When the authentication server 401 receives the authentication support request,
The personal authentication support unit 405 is added to the queue. The personal authentication support unit 405 reads the authentication support request from the queue according to the urgency. The personal authentication support unit 405 displays the registered image and the input image using the authentication log.
Display on 8. The person in charge of personal authentication support visually confirms the image displayed on the display 408, and transmits the determination result to the mobile phone 1003 using the mobile phone ID.

(Embodiment 4) The configuration of Embodiment 4 of the present invention will be described with reference to FIG. The feature of this embodiment is that it is configured only for the mobile phone 1004.

In FIG. 20, a mobile phone 1004 includes a speaker 11, a display 12, a camera 13 for capturing a face image, a microphone 14, an antenna 15, and a button 1.
6, a data processing unit 17, and a face authentication unit 20,
The data processing unit 17 includes a device control unit 18 and a data storage unit 1.
It is composed of nine. The device control section 18 not only processes data using various programs, but also the mobile phone 1
The device control of 004 is also performed.

The data storage unit 19 can store various programs used by the device control unit 18, data input from the camera 13, the microphone 14 and the button 16, and result data processed by the device control unit 18. The face authentication unit 20 includes a learned function storage unit 22 that stores a learning function for authentication,
The authentication unit 21 includes a registration image read from the data storage unit 19 and a learning result read from the learned function storage unit 22 to authenticate the face image captured from the camera 13.

Next, the operation of the fourth embodiment of the present invention will be described. First, the learned function will be described. As the learned function, a default function is recorded in the learned function storage unit 22 in advance at the time of factory shipment. Since the face image of the person himself / herself is not used for learning this learned function, the discriminating ability is low.

Next, the face image registration operation will be described with reference to FIG. When the user operates the button 16, the device control unit 18 reads the registration program from the data storage unit 19 and executes it. However, in order to avoid operation by anyone other than the person himself / herself, the registration program is read out only when the number stored by the person himself / herself is input.

The device control section 18 switches the display of the display 12 (switches the current display to the camera input display).
Is performed (step 401). An index for determining the eye position is displayed on the display 12 (step 402), and an instruction is issued so that the registrant's eye position input from the camera 13 matches two points which are the index (step 403). As an instruction method, an instruction is displayed on the display 12, or an instruction is given by voice using the speaker 11. The instruction contents also include blinking, changing the direction of the face, moving the face up and down, changing the direction of the body, and moving the position. The device control unit 18 displays the input face image on the display 12 so that the user can confirm it.
(Step 404). When the confirmation process is performed by the user's button operation, the device control unit 18 compresses the face image (step
p405), and stores the data in the data storage unit 19 (step 40).
6).

The authentication operation is the same as in the first embodiment.

By combining the first and fourth embodiments of the present invention, the service content can be set in two stages. One is a service that can be authenticated only by a mobile phone, and can update only registered images. A user who wants to further improve the recognition rate can receive a service of performing authentication using the image of the person and performing authentication with the configuration of the first embodiment.

[0139]

As described above, according to the information terminal device and the authentication system of the present invention, when a face image is input, indexes such as two points for determining the position of the face or the eyes are displayed and the face is displayed. By changing the direction, blinking, moving the face up and down, changing the direction of the body, and moving the position, the lighting conditions and the direction of the face change, so the extraction accuracy of the face image improves. . Further, even if another person is masquerading as the person himself / herself by using the photograph, an advantageous effect that the photograph and the living body can be easily distinguished can be obtained.

[Brief description of drawings]

FIG. 1 is a functional configuration diagram of an information processing apparatus having an authentication function according to the present invention.

FIG. 2 is a system configuration diagram of a registration / authentication system according to the first embodiment of the present invention.

FIG. 3 is a schematic view of a camera-equipped mobile phone according to the first embodiment of the present invention.

FIG. 4 is a functional configuration diagram of a camera-equipped mobile phone according to a third embodiment of the present invention.

FIG. 5 is a functional configuration diagram of a mobile phone with an authentication function according to the first embodiment of the present invention.

FIG. 6A is a registration sequence diagram for explaining a face image registration process according to the first embodiment of the present invention. FIG. 6B is a registration sequence diagram for explaining a voice registration process according to the second embodiment of the present invention. Figure

FIG. 7 is a flowchart for explaining face image extraction processing according to Embodiment 1 of the present invention.

FIG. 8 is a flowchart for explaining face image learning processing according to the first embodiment of the present invention.

FIG. 9A is a recognition sequence diagram for explaining a sequence when the recognition processing is successful in the first embodiment of the present invention. FIG. 9B is a sequence when the recognition processing is failed in the first embodiment of the present invention. Recognition sequence diagram to explain

FIG. 10 is a flowchart for explaining face image recognition processing according to the first embodiment of the present invention.

FIG. 11 is a functional configuration diagram of a mobile phone with a multiple authentication function according to the second embodiment of the present invention.

FIG. 12 is a system configuration diagram showing a registration / authentication system according to a second embodiment of the present invention.

FIG. 13 is a flowchart for explaining voice extraction processing according to the second embodiment of the present invention.

FIG. 14 is a flowchart for explaining voice learning processing according to the second embodiment of the present invention.

FIG. 15 is a flowchart for explaining an authentication operation according to the second embodiment of the present invention.

FIG. 16 is a flowchart for explaining speaker recognition processing according to the second embodiment of the present invention.

FIG. 17 is a system configuration diagram showing a registration / authentication system according to a third embodiment of the present invention.

FIG. 18 is a recognition sequence diagram for explaining a recognition process according to the third embodiment of the present invention.

FIG. 19 is a flowchart for explaining face image recognition processing according to the third embodiment of the present invention.

FIG. 20 is a functional configuration diagram of a mobile phone with an authentication function according to a fourth embodiment of the present invention.

FIG. 21 is a flowchart for explaining face image registration processing according to Embodiment 4 of the present invention.

FIG. 22 (A) is an input face image diagram that is the first example in the first embodiment of the present invention; and (B) is an input face image diagram that is the second example in the first embodiment of the present invention.

23A is a diagram of an input face image according to the first embodiment of the present invention; FIG. 23B is a diagram of an input face image according to the first embodiment of the present invention; FIG. 23C is a diagram of the input face image according to the first embodiment of the present invention.

[Explanation of symbols]

1 Input means 2 display means 3 index display means 4 Authentication means 5 Biometric information confirmation means 6 Information terminal equipment 7 Evaluation means 8 instruction means 11 speakers 12 display 13 cameras 14 microphone 15 antenna 16 buttons 17 Data processing unit 18 Equipment control section 19 Data storage 20 Face recognition section 21 Authentication Department 22 Learned function memory 23 Speaker Authentication Unit 24 Authentication Department 25 Learned function memory 29 Authentication function switching unit 50 IC card 51 IC card reading interface 101 network 201 registration server 202 System Management Department 203 Face image registration / update unit 204 face image database 205 Data input / output section 206 Personal Information Management Department 207 Registration Log Management Department 301 registration server 302 Face image / voice database 303 Voice registration / update section 401 Registration / Authentication Server 402 System Management Department 403 Registration / Authentication Department 404 Face image database 405 Personal Authentication Support Department 406 Personal information storage 407 Authentication log storage unit 408 display 409 Personal Authentication Department 410 Face image registration unit 1001 mobile phone 1002 mobile phone 1003 mobile phone 1004 mobile phone 2217 indicators 2218 face area 2219 Eye center coordinates

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04L 9/32 H04L 9/00 673D (72) Inventor Shin Yamada 1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric Industrial Incorporated (72) Inventor Toshiaki Akimoto 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. F term (reference) 5B043 AA09 BA04 DA05 GA02 5B047 AA23 CA12 CB21 5B085 AA08 AE15 AE23 AE25 BC01 BE01 BE04 BG02 5J01 A07 KA16 PA02

Claims (16)

[Claims] 1. A display means for displaying the inputted biometric information of the user, and an authentication means for personally authenticating a user registered in advance based on the biometric information, wherein the display means comprises the biometric information. An information terminal device displaying an index for designating the size and position of information. 2. The information terminal device according to claim 1, wherein the biometric information is a user's face image and one of a user's face image and voice. 3. The information terminal device according to claim 1, wherein the index defines the positions of both eyes. 4. The information terminal device according to claim 1, further comprising an evaluation unit that evaluates the biometric information. 5. The information terminal device according to claim 4, wherein the evaluation unit has at least one of face position detection, face size detection, face orientation detection, and light source direction detection. 6. The method according to claim 1, further comprising instruction means for giving an instruction to a user when the biometric information is input.
The information terminal device according to any one of 1. 7. The information terminal device according to claim 6, wherein the instruction unit gives at least one of an instruction to change the direction of the body, an instruction to move the face up and down or left and right, and an instruction to move the position. . 8. The information terminal device according to claim 6, wherein the instruction given to the user by the instruction means is an instruction to feed back a detection result in the evaluation means. . 9. The information terminal device according to claim 2, wherein the face image is converted into a mirror image and displayed. 10. The information terminal device according to any one of claims 1 to 9, which is any one of a mobile information terminal equipped with a communication unit, a mobile personal computer, and a mobile phone. 11. The information terminal device according to claim 1, wherein the index is two or more points, circles, crosses, or polygons. 12. The information terminal device according to claim 1, and the biometric information input from the information terminal device via a communication network are registered in a database,
A registration server having learning means for learning the identification function of each individual from the biometric information and each biometric information from the already registered database, and a system management means for managing the biometric information, the identification function and the ID. An authentication system equipped with. 13. The authentication system according to claim 12, wherein the biometric information of an individual is updated at regular intervals. 14. The authentication system according to claim 13, wherein the registration server prompts each information terminal device to update the biometric information of an individual at regular intervals. 15. A step of displaying an index for designating the size and position of the biometric information, a step of displaying biometric information of the user input based on the index, and the biometric information being permissible with respect to the index. And if the evaluation result is not within the allowable range, the user is instructed to improve, and if the evaluation result is within the allowable range, the biometric information is registered or based on pre-registered information. Authentication step, and at least one of the instruction to change the direction of the face and the instruction to blink after the registration processing or the authentication processing, and spoofing to the person who recognizes whether the change has been made as instructed. A registration / authentication method comprising: a detecting step; and a step of stopping the registration processing or the authentication processing when the impersonation is detected. 16. The registration / authentication method according to claim 15, wherein at least one of the instruction to change the direction of the face and the instruction to perform the blink is randomly given.