WO2017201675A1

WO2017201675A1 - Robot for performing recognition using biological information and operation method therefor

Info

Publication number: WO2017201675A1
Application number: PCT/CN2016/083190
Authority: WO
Inventors: 叶军; 付骏; 苗森
Original assignee: 深圳普得技术有限公司
Priority date: 2016-05-24
Filing date: 2016-05-24
Publication date: 2017-11-30

Abstract

Provided are a robot for performing recognition using biological information and an operation method therefor, comprising a voice input module (101), a voice recognition module (102), a biological voice feature extraction module (103), a registration instruction extraction module (104), an image input module (105), an biological image feature extraction module (106), and a biological feature comparison and management module (107). When the human-robot interaction starts, the robot can simulate a human-to-human interaction process, and complete, by means of a preset instruction, the automatic registration of a user; during the human-robot interaction, the robot exercises a learning capability, recognizing the identity of the user by means of comparing the biological feature information of the user, and automatically updating and completing the biological information of the user according to the differences of the biological feature recognized each time, such that the robot is more familiar with the user, thereby effectively improving the rate at which the robot may recognize the user. The robot also provides a personalized service for different users.

Description

Title of Invention: A Robot Using Biometric Information Recognition and Method of Using the Same

[0001] The present invention relates to the field of intelligent robot technology, and more particularly to a robot that utilizes biometric information recognition and a method of using the same.

Background technique

[0002] With the continuous development of robots, the degree of intelligence is constantly increasing, and the market demand for artificial intelligence is getting higher and higher. As an important foundation of artificial intelligence, biometrics has received high attention in research fields at home and abroad. In recent years, biometrics have been greatly improved at the algorithm level, enabling biometrics to be implemented faster on civilian CPUs.

[0003] Although biometric algorithms have a variety of technical routes, these algorithms have varying degrees of limitations in practical applications. At present, the identification method used by most robots is to manually enter the user's biological information in advance, such as fingerprint information. Each time a new user has to manually enter user information, the operation is cumbersome; in addition, in order to improve the recognition success rate, the current solution is Repeat the input of the user's fingerprint multiple times before use to ensure that there are enough comparison samples, but this method will keep the sample inconvenience after the fingerprint is entered, and can not automatically update and improve the user's fingerprint information according to the change, so that the identification The correct rate cannot be improved and the user experience is reduced.

technical problem

[0004] At present, the identification method adopted by most robots is to manually input the user's biological information in advance, such as fingerprint information, and each time a new user has to manually enter user information, the operation is cumbersome; in addition, in order to improve the recognition success rate, the current solution The user's fingerprint is repeatedly input many times before use to ensure that there are enough comparison samples, but this method will keep the sample inconvenience after the fingerprint is entered, and can not automatically update and improve the user's fingerprint information according to the change. The recognition accuracy rate cannot be improved, and the user experience is reduced. Problem solution

Technical solution

[0005] The technical problem to be solved by the present invention is to provide a robot that utilizes biometric information recognition and a method of using the same. [0006] The technical solution adopted by the present invention to solve the technical problem thereof is: constructing a robot that utilizes biometric information recognition, including:

a voice input module for receiving user voice information, and an image input module for receiving user image information;

[0008] a voice recognition module connected to the voice input module for identifying and determining voice content of the user voice information;

[0009] a registration instruction extraction module connected to the voice recognition module and configured to issue a registration instruction;

[0010] a voice biometric extraction module coupled to the voice input module for converting the user voice information into a voice feature vector identifying a user identity;

[0011] an image biometric extraction module connected to the image input module for converting the user image information of the user into an image feature vector for identifying a user identity;

[0012] a biometric comparison and management module respectively connected to the registration instruction extraction module, the speech biometrics extraction module, and the image biometrics extraction module, wherein the biometric comparison and management module is used for registration The new user establishes an association relationship between the voice feature vector of the user and the image feature vector, automatically updates the registered user biometric, and manages the user's personal transaction information.

[0013] In the robot for utilizing biometric information according to the present invention, the method further includes: a biometric storage module connected to the biometric comparison and management module for storing a biometric of the user.

[0014] In the robot for identifying biometric information according to the present invention, the method further includes: a user information management module for storing the personal transaction information of the user, which is connected to the biometric feature and the management module.

[0015] Preferably, in the robot using biometric information identification according to the present invention, the user image information includes one or more of a face, a gesture, a gesture, a form, a fingerprint, and an iris.

[0016] In addition, the present invention also discloses a method for using a robot that uses biometric information identification, and includes the following steps.

[0017] receiving user voice information of the user, and identifying voice content of the user voice information;

[0018] determining whether the voice content meets a preset registration condition;

[0019] If not, re-receiving the user's voice information; if yes, converting the user voice information into a voice feature vector for identifying the user identity, and determining whether the user is a registered user;

[0020] If not, a new user registration instruction is issued, a new user account is registered, and the voice feature vector is added to In the new user account, establishing a correspondence between the voice feature vector and the new user account; if yes, obtaining a voice feature vector enhancement amount by comparing with the existing voice feature vector of the user, Whether the speech feature vector enhancement amount has an enhanced value;

[0021] If yes, adding the speech feature vector enhancement amount to the biometric vector database of the user; if not, ending.

[0022] Further, in the method for using the biometric information recognition robot according to the present invention, the user image information of the user is received while receiving the user voice information of the user,

[0023] for a newly registered user account: converting the user image information into an image feature vector for identifying a user identity, adding the image feature vector to the new user account, and establishing the voice feature vector, The image feature vector, and a correspondence between the new user accounts;

[0024] for a registered user account: obtaining an image feature vector enhancement amount by comparing with an existing image feature vector of the user, and determining whether the image feature vector enhancement amount has an enhanced value; if yes, the image is A feature vector enhancement amount is added to the biometric vector database; if not, it ends.

[0025] The method for using the biological information recognition robot according to the present invention, for a registered user, in the process of human-computer interaction:

Receiving biometric information of the user, and extracting a biometric vector corresponding to the biometric information;

[0027] comparing the biometric vector with the biometric vector database to determine whether it can be identified; [0028] if not, continuing to receive and extract the biometric vector of the user, and the biometric The vector database is compared until it is identified.

[0029] The method for using the biometric information recognition robot according to the present invention, after the user is identified: [0030] obtaining a biometric vector enhancement amount by comparing with the biometric vector database;

[0031] determining whether the biometric vector enhancement amount has an enhanced value;

[0032] if yes, adding the biometric vector enhancement amount to the biometric vector database;

, then return to continue extracting the user's biometric vector.

[0033] The method for using the biometric information recognition robot according to the present invention, after the user is identified: [0034] executing pre-stored user personalized information to provide personalized service for the user;

[0035] determining whether an end condition is reached;

[0036] If no, the user biometric information is re-received; if yes, the process ends. [0037] Preferably, the user personalized information includes one or more of user personal information, user social information, user scheduling information, and user daily habits.

Advantageous effects of the invention

Beneficial effect

[0038] A robot using biometric information and a method for using the same according to the present invention have the following beneficial effects: When human-computer interaction starts, the robot can imitate the interaction process between people and complete the user's Automatic registration; in the process of human-computer interaction, the robot has the learning ability to identify the user's identity by comparing the biometric information of the user, and automatically update and improve the user's biological information according to the difference of the biometrics recognized each time. The robot is becoming more and more familiar to the user, thus effectively improving the recognition rate of the robot to the user; the robot also provides different personalized services for different users. Brief description of the drawing

DRAWINGS

[0039] The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

1 is a schematic structural diagram of a system of a robot of the present invention;

2 is a schematic diagram of a biometric information registration application of the robot of the present invention;

3 is a flow chart of biometric registration of a robot of the present invention;

4 is a schematic diagram of a face angle and a biological feature of the robot of the present invention;

[0044] FIG. 5 is a flow chart of automatic updating and perfecting of biometrics of the robot of the present invention;

6 is a schematic diagram showing the data structure of the biometrics of the robot of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

[0046] As shown in FIG. 1, a schematic structural view of a first embodiment of the present invention.

[0047] This embodiment discloses a robot that utilizes biometric information recognition, including: a voice input module 101, a voice recognition module 102, a voice biometric feature extraction module 103, a registration instruction extraction module 104, an image input module 105, and an image biometric feature extraction. The module 106, the biometric comparison and management module 107, the biometric storage module 108, and the user information storage module 109 are respectively described below:

[0048] The voice input module 101 is configured to receive user voice information, and the image input module 105 is configured to receive user images. Information. Preferably, in the robot using the biometric information recognition of the present invention, the user image information includes, but is not limited to, a user's face, gesture, posture, form, fingerprint, iris.

[0049] The voice recognition module 102 of the voice content is connected to the voice input module 101 for identifying and determining the user's language preference;

[0050] The registration instruction extraction module 104 is coupled to the voice recognition module 102 for issuing a registration instruction;

The voice biometric feature extraction module 103 is coupled to the voice input module 101 for converting user voice information into a voice feature vector for identifying a user identity.

The image biometric feature extraction module 106 is coupled to the image input module 105 for converting user image information of the user into an image feature vector for identifying the identity of the user.

[0053] The biometric comparison and management module 107 is connected to the registration instruction extraction module 104, the speech biometric extraction module 103, and the image biometric extraction module 106, respectively, and the biometric comparison and management module 107 is used to register new users and establish The association between the user's speech feature vector and the image feature vector, automatic updating of registered user biometrics, and management of user personal transaction information.

[0054] The biometric storage module 108 is connected to the biometric comparison and management module 107 for storing user biometrics; the user information storage module 109 is connected with the biometric comparison and management module 107 for storing user personal transaction information. .

[0055] The biometric feature comparison algorithm of speech and face is based on the calculation of similarity. In practical applications, changes in human tonality, speech rate, etc., face angle and subtle changes will be accurate for biometric recognition. The resulting P-direction, the more biometric vector templates of the average user, the higher the accuracy of the comparison. Therefore, in the process of human-computer interaction, the biometric comparison and management module 107 is used to identify the identity of the current user. After the user identity is successfully identified, the biometric comparison and management module 107 continuously performs the biometric vector of the user. Update and improve to achieve the effect of improving the accuracy of the comparison.

[0056] As shown in FIG. 2, the biometric registration of the present invention mimics the introduction between a person and a person, initiated by the user's voice. When the user 305 says to the robot 307: "ABC, my name is XYZ" 301, or a similar sentence pattern "ABC, I am X YZ", "ABC, please call me XYZ", etc., the speech recognition module 202 judges the sentence. Register the command for the new user, where "ABC" is the name or logo of the robot and "XYZ" is the name of the new user and the entry to the user information database. At the same time, the robot of the present invention detects the human face 306 through the camera device 205 and the face detection, and converts the face image into a feature vector used by the robot for face recognition, which is compared by the biometric feature. The management module 107 generates a user information database with XYZ as an entry.

[0057] In addition, the present invention also discloses a method for using a robot that uses biometric information identification, which includes the following steps.

Receiving user voice information of the user, and identifying voice content of the user voice information;

[0059] determining whether the voice content meets the preset registration condition; the preset registration condition is a specific statement format, and the statement formats are all abstracting the introduction between people.

[0060] If not, re-receiving the user's voice information; if yes, converting the user voice information into a voice feature vector for identifying the user identity, determining whether the user is a registered user;

[0061] If not, a new user registration instruction is issued, a new user account is registered, a voice feature vector is added to the new user account, and a correspondence between the voice feature vector and the new user account is established; if yes, the user is The existing speech feature vector comparison obtains the speech feature vector enhancement amount, and determines whether the speech feature vector enhancement amount has enhanced value;

[0062] If yes, the speech feature vector enhancement amount is added to the biometric vector database of the user; if not, the process ends.

Further, in the method of using the biometric information recognition robot of the present invention, the user image information of the user is received while receiving the user voice information of the user,

[0064] For a newly registered user account: converting user image information into an image feature vector for identifying a user identity, adding an image feature vector to a new user account, and establishing a voice feature vector, an image feature vector, and a new user account Correspondence between the two; after the association is established, when the robot recognizes the user again, the user can be identified by a single message. Of course, in certain application scenarios, for example, to improve security, it is necessary to simultaneously satisfy the voice information and the image information in order to correctly recognize.

[0065] For the registered user account: obtaining the image feature vector enhancement amount by comparing with the existing image feature vector of the user, and determining whether the image feature vector enhancement amount has an enhanced value; if yes, adding the image feature vector enhancement amount to In the biometric vector database; if not, it ends.

[0066] Further, the method for using the biological information recognition robot of the present invention, for the registered user, in the process of human-computer interaction:

Receiving biological information of the user, extracting a biometric vector corresponding to the biological information; the biometrics herein include but not limited to the user's voice information, image information, face, gesture, posture, shape, fingerprint, rainbow Membrane.

[0068] comparing the biometric vector with the biometric vector database to determine whether the identification is possible;

[0069] If not, the biometric vector of the user is continuously received and extracted and compared with the biometric vector database until identified.

[0070] The method of using the biometric information recognition robot of the present invention, after the user is identified:

[0071] obtaining a biometric vector enhancement amount by comparing with a biometric vector database; biometrics are mainly divided into two categories, one is perfecting existing biometrics, such as facial positive information in a user's face information, When the frontal information of the face changes slightly, the robot can still recognize and add these subtle changes as enhancements to the existing face positive information; the other is features that are not in the biometric vector database, such as the user passing the voice. After the information is recognized, the robot captures the image information of the user, and the image information is not stored in the biometric vector database. Then, the robot automatically extracts the image feature corresponding to the image information, and adds the image feature to the biometric feature. In the vector database.

[0072] It is judged whether the biometric vector enhancement amount has an enhanced value; not all biometrics have an added value, and if each biometric is stored, the data amount is too large, which is not conducive to quickly and accurately identifying the user.

[0073] If yes, the biometric vector enhancement amount is added to the biometric vector database; if not, then returning to extract the biometric vector of the user.

[0074] The method of using the biometric information recognition robot of the present invention, after the user is identified:

[0075] executing pre-stored user personalized information to provide personalized services for the user;

[0076] determining whether an end condition is reached;

[0077] If no, the user biometric information is re-received; if yes, the process ends.

[0078] Preferably, the user personalized information includes one or more of user personal information, user social information, user scheduling information, and user daily habits.

3 is a second embodiment of the present invention.

3 is a flow chart of biometric registration of a robot of the present invention. Specifically, the robot performs voice recognition on the voice input 401; when the voice information is recognized as the new user registration command 402, the robot determines whether it is the registered user 403, that is, whether the XYZ in the above embodiment is already present in the system. If not, initiate a new user profile 405 and add the user biometric 407; if yes, the existing user profile Feature enhancement 404. Here, it is necessary to first determine whether the newly acquired biometric vector has a value 406, and if so, add the user biometric 407 and end 408; if not, end 408.

[0081] There are many methods for computer algorithms for speech recognition and face recognition. The general process mainly includes the collection of speech or graphics, and then converts the original information into machine language. When identifying, the features described in the machine language are similar. Degree comparison. Taking face recognition as an example, mainstream algorithms are mainly divided into two types: feature point based comparison and graph similarity based comparison.

[0082] As shown in FIG. 4, the Local Binary Pattern (LBP) algorithm is taken as an example for description. A1 and B 1 are the original images of the input, and A2 and B2 are the local Binary Pattern.

(LBP) The feature image generated by the algorithm, A1 corresponds to A2, and B1 corresponds to B2. Taking a general comparison method as an example, the feature images B 1 and B2 are divided into N x M block squares, and the feature points in the square are counted to form a vector HIST to represent the features in the square. The feature similarity can be obtained by comparing two feature images by one by one. In general, the number of squares, such as 9x9 or 7x7, will be highly recognizable to the same person, but the redundancy of the face rotation or movement will not be high. On the contrary, the number of squares is small, such as 3x3, 2x2, and the redundancy for the rotation or movement of the face is high, but the recognition for people is low. Some feature-based alignment algorithms deal with the problem of deformation and rotation of graphics, but still can't completely solve the problem of reduced precision caused by face rotation.

[0083] In practical applications, the image may have a large change due to the position, angle, and other reasons of the face.

Taking A1 and B1 in Fig. 4 as an example, when the face angle is rotated, the distribution of feature points and feature points changes greatly. If this change exceeds the threshold judged by the robot, the robot cannot recognize that it is the same user. Therefore, when the robot recognizes the user's identity, it is necessary to continuously collect the user templates and collect facial features at different angles to expand the user's biometric information template to ensure that the robot can recognize the user from different angles.

[0084] It can be understood that the present embodiment only uses the Local Binary Pattern (LBP) algorithm as an example to describe the image recognition process. In actual use, one or more image recognition algorithms may be selected as needed to perform image recognition.

[0085] The process of speech recognition is similar to a human face. Speech recognition is generally divided into text-based Text Dependent and text-independent Text Independent. Since human speech has a certain degree of change, such as intonation and speech rate.

, as well as physical conditions, etc., the biometric identification of speech also has the problem of face recognition, that is, a single special The stencil template is often not able to accurately identify the user identity in the actual application scenario. The system needs to automatically collect valuable templates to enhance the redundancy and accuracy of the comparison.

[0086] After the user completes the registration, the robot starts to interact with the user, the user voice information input module 101 re-receives the user's voice information, and the user voice information biometric extraction module 103 extracts the user's voice feature vector, and the creature The feature comparison and the information stored in the database in the management module 107 are compared to determine whether the identification is possible;

[0087] If not, the speech feature vectors are continuously extracted and compared with the information stored in the database in the biometric comparison and management module 107.

[0088] After the speech feature vector is compared and identified, the first action and the second action are performed:

[0089] The first action: extracting the voice feature vector enhancement amount of the registered user; the biometric comparison and management module

107 determining whether the speech feature vector enhancement amount has an enhanced value; if so, adding the speech feature vector enhancement amount to the first biometric vector template of the registered user; if not, returning to continue extracting the biometric feature.

[0090] The second action: performing pre-stored user personalized information, providing personalized service for the user; after performing the first action and the second action, determining whether the end condition is reached; if yes, ending; if not, the user voice The information input module 101 re-receives the user's user voice information. Preferably, in the method for using the biometric information recognition robot of the present invention, the user personalized information includes: one or more of user personal information, user social information, user inter-office arrangement information, and user daily habits.

[0091] FIG. 5 is a third embodiment of the robot of the present invention.

5 is a flow chart of automatic updating and perfecting of the biometrics of the robot of the present invention; in this embodiment, the robot continuously extracts the biometrics 502 in the human-computer interaction 501, and compares with the biometrics in the biometric vector database. 503. Determine whether the current user identifies 504. If no, continue to extract biometrics 502; if so, make two actions:

[0093] (1) continuously extracting the biometric feature 505 in the interaction, and obtaining the existing user biometric enhancement amount 404. In order to avoid excessively adding unnecessary features, the robot will judge the newly collected feature to see if it is correct. The user's biometric enhancements are valuable and need to be enhanced 406. If not, then return to the continuous extraction of biometrics 505; if so, then add user biometric enhancements to the user biometrics 407 [0094] (2) providing an identity-based personalized service 530 to the current user, such as loading a user name in voice, based on the user's custom service, and the like. When the process meets the end condition 532吋, it ends 533; otherwise the process continues from the human-machine interaction process node.

6 is a schematic diagram of the data structure of the biometrics of the robot of the present invention. In the embodiment, the robot of the present invention, the user identifier 710 takes the above-mentioned XYZ as an example, and the basic structure of the biometric database mainly includes data. The categories are: biometric 720, user information 730, social information 740, daytime and habit 75 0, wherein the biometric 720 category includes, but is not limited to, one or more speech features 725, one or more facial features 721. Under the User Information 730 category, the data includes, but is not limited to, the user's birthday, age, address, phone number, etc. 731. The social information 740 category data includes, but is not limited to, one or more family members and relationships, one or more friends, and relationships, etc. 741. In the event and habits, the 750 category data includes, but is not limited to, the characteristics of the language 751 commonly used by the user, the characteristics of the action 730, and the characteristics of the habit 752. The database structure described in FIG. 6 basically protects the robot by collecting the user information, using the user identifier 710 as the entry, the biometric 720 as the information base for identifying the user, and utilizing the personal user information 730 for the user, the social information 740, Events and habits 750 information processing to provide personalized services for this user. It can be understood that the above personal information of the user can be adjusted according to specific needs according to different application scenarios and user requirements.

[0096] In summary, by implementing the present invention, the robot is more intelligent and has the learning ability. By simulating the interaction between people, the robot can automatically complete the registration of the user and continuously learn in the process of human-computer interaction. Continuously and automatically improve the user's biometrics, making the robot more and more familiar to users, and constantly improve the recognition rate.

The above embodiments are merely illustrative of the technical concept and the features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the contents of the present invention and to implement the present invention without limiting the scope of the present invention. All changes and modifications made within the scope of the claims of the present invention are intended to be included within the scope of the appended claims.

Claims

A robot using biometric information recognition, comprising:

a voice recognition module connected to the voice input module for identifying and determining voice content of the user voice information;

a registration instruction extraction module connected to the voice recognition module for issuing a registration instruction; a voice biometric extraction module connected to the voice input module and configured to convert the user voice information into a voice feature vector for identifying a user identity ;

An image biometric extraction module coupled to the image input module for converting the user image information of the user into an image feature vector identifying the user identity;

a biometric comparison and management module respectively connected to the registration instruction extraction module, the speech biometric extraction module, and the image biometric extraction module, wherein the biometric comparison and management module is used to register a new user, Establishing an association relationship between the voice feature vector of the user and the image feature vector, and automatically updating the registered user biometric

And managing user personal affairs information.

The robot for identifying biometric information according to claim 1, further comprising: a biometric storage module coupled to the biometric alignment and management module for storing biometrics of the user.

The robot for identifying biometric information according to claim 2, further comprising: a user information management module for storing the personal transaction information of the user in connection with the biometric comparison and management module.

The robot using biometric information according to any one of claims 1 to 3, wherein the user image information includes one or more of a face, a gesture, a gesture, a form, a fingerprint, and an iris.

A method of using a robot for recognizing biological information, comprising the steps of:

Receiving user voice information of the user, and identifying voice content of the voice information of the user; Determining whether the voice content meets a preset registration condition;

If not, re-receiving the user's voice information; if yes, converting the user voice information into a voice feature vector for identifying the user identity, and determining whether the user is a registered user;

If not, issuing a new user registration command, registering a new user account, adding the voice feature vector to the new user account, establishing a correspondence between the voice feature vector and the new user account; if yes, And obtaining a speech feature vector enhancement amount by comparing with the existing speech feature vector of the user, and determining whether the speech feature vector enhancement amount has an enhanced value;

If yes, the speech feature vector enhancement amount is added to the biometric vector database of the user; if not, the process ends.

[Claim 6] The method of using a biometric information recognition robot according to claim 5, wherein the user image information of the user is received while receiving the user voice information of the user,

For a newly registered user account: converting the user image information into an image feature vector for identifying a user identity, adding the image feature vector to the new user account, and establishing the voice feature vector, the image a feature vector, and a correspondence between the new user accounts;

For a registered user account: obtaining an image feature vector enhancement amount by comparing with the existing image feature vector of the user, and determining whether the image feature vector enhancement amount has an enhanced value; if yes, enhancing the image feature vector The quantity is added to the biometric vector database; if not, it ends.

[Claim 7] The method for using a biometric information recognition robot according to claim 5 or 6, wherein, for a registered user, in a human-computer interaction process, receiving biometric information of the user, extracting a location Deciding a biometric vector corresponding to the biometric information; comparing the biometric vector with the biometric vector database to determine whether the biometric vector is identifiable;

If not, continuing to receive and extract the biometric vector of the user, and interacting with the biometric The eigenvector database is compared until it is identified.

[Claim 8] A method of using a biometric information recognition robot according to claim 7, wherein after the user is recognized:

Obtaining a biometric vector enhancement amount by comparing with the biometric vector database; determining whether the biometric vector enhancement amount has an enhanced value; if yes, adding the biometric vector enhancement amount to the biometric vector database If no, return to continue extracting the user's biometric vector.

[Claim 9] A method of using a biometric information recognition robot according to claim 7, wherein after the user is recognized:

Execute pre-stored user personalized information to provide personalized services for users;

Determine whether the end condition is reached;

If not, the user biometric information is re-received; if yes, the process ends.

[Claim 10] The method for using a biometric information recognition robot according to claim 9, wherein the user personalization information includes user personal information, user social information, user inter-day scheduling information, and user daily habits. One or more of them.