JP2009223170A - Speech recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy of speech recognition, while reducing time required for speech recognition processing. <P>SOLUTION: A speech recognition system 10 includes a server 20, and the server 20 specifies an article 24 which a person 16 indicates by performing speech recognition processing on speech uttered by the person 16. The server 20 detects a position of the person 16, extracts information required for performing speech recognition on a word about the article 24 located around the person 16, from a speech recognition dictionary 126, and creates a speech recognition local dictionary 205B. Then, the speech uttered by the person 16 is speech-recognized by using the speech recognition local dictionary 205B. As a result, time required for speech recognition processing is reduced, and accuracy of speech recognition is improved. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a speech recognition system, and more particularly to a speech recognition system that recognizes speech produced by a person and identifies an article designated by the person.

  There is no patent document that discloses an example of this type of speech recognition system, but speech recognition is used in various devices as an interface for connecting a person and a device. There is a sorting facility disclosed in Document 1.

According to Patent Document 1, this sorting facility is for picking products, inspecting picked products, and sorting containers in which products are picked. Various sorts of voices performed by picking workers and sorting workers The instructions are recognized by voice and given to the sorting equipment.
JP 2006-36413 A [B65G 1/137]

  However, in order to realize speech recognition, phoneme-based speech recognition dictionaries are generally used. However, in background technology such as this sorter, speech recognition is performed to deal with various utterances produced by humans. The dictionary vocabulary increases (see the fifth line of paragraph number 0019 of Patent Document 1). An increase in the vocabulary of the speech recognition dictionary means that the number of phoneme symbol strings registered in the speech recognition dictionary to be matched with the phoneme symbol string generated by the speech recognition increases. There arises a problem that the processing time becomes long. In addition, when the number of phoneme symbol strings registered in the speech recognition dictionary increases, the number of phoneme symbol strings generated by speech recognition and the candidate phoneme symbol strings to be matched increases, so the rate of correct speech recognition decreases. Also happens.

  Therefore, the main object of the present invention is to provide a novel speech recognition system.

  Another object of the present invention is to provide a speech recognition system capable of reducing the time required for speech recognition processing and increasing the rate of correct speech recognition.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

  A first aspect of the present invention is a voice recognition system for recognizing a voice generated by a human to identify an article designated by the human, a position detecting unit for detecting a human position, and a human position detected by the position detecting unit. Based on, a nearby article specifying unit that specifies an article in the vicinity of a person, a dictionary building unit that builds a speech recognition dictionary related to the article specified by the article specifying unit, and a voice recognition using a voice recognition dictionary built by the dictionary building unit It is a voice recognition system provided with the instruction | indication article | item identification means which identifies the article | item which a person instruct | indicates by performing.

  In the first invention, the speech recognition system (10) identifies an article designated by a person by recognizing the speech produced by the person. The position detection means (124, 200, 208) detects the position of the person who indicates the article by producing a sound. The nearby article specifying means (122, 200) specifies an article in the vicinity of the person based on the position of the person detected by the position detecting means. Then, the dictionary construction means (200, 122, 205A) constructs a speech recognition dictionary (205B) related to the article specified by the article specification means. Furthermore, the designated article specifying means (200) performs voice recognition of a voice uttered by a person using the voice recognition dictionary constructed by the dictionary construction means, and identifies an article designated by the person.

  According to the first invention, since voice recognition is performed based on a voice recognition dictionary that stores only information related to articles in the vicinity of humans, it is possible to reduce the time required for voice recognition and increase the rate of correct voice recognition. it can.

  The second invention is an invention subordinate to the first invention, wherein the dictionary construction means constructs the first speech recognition dictionary based on the second speech recognition dictionary for all articles targeted by the speech recognition system. To do.

  In the second invention, the dictionary construction means (200, 122, 205A) constructs the first speech recognition dictionary based on the second speech recognition dictionary (126) for all articles targeted by the speech recognition system.

  According to the second aspect, since the first speech recognition dictionary is constructed based on the second speech recognition dictionary, the first speech recognition dictionary can be constructed quickly and easily.

  A third invention is a voice recognition system for recognizing voices uttered by humans to identify articles designated by humans, and position information indicating positions where articles exist for each article targeted by the voice recognition system Position information storage means for storing speech recognition information storage means for storing speech recognition information including a word related to the article and a phoneme symbol string of the word for each article targeted by the speech recognition system, and performing speech recognition processing A word specifying means for specifying a word related to the article with reference to the speech recognition information stored in the speech recognition information storage means based on the phoneme symbol string obtained in this manner, and a human being based on the word specified by the word specifying means Stored in the article specifying means for specifying the instructed article, the position detecting means for detecting the position of the person, and the human position detected by the position detecting means and the position information storage means. Based on the position information, and a nearby article specifying means for specifying an article that exists in the vicinity of the person, and the word specifying means refers only to the speech recognition information including the word related to the article specified by the neighboring article specifying means. This is a speech recognition system that identifies words.

  In the third invention, a voice recognition system (10) recognizes a voice uttered by a person and specifies an article designated by the person. The position information storage means (122, 205A) stores position information indicating the position where the article exists for each article targeted by the voice recognition system. The voice recognition information storage means (126) stores voice recognition information including a word related to the article and a phoneme symbol string of the word for each article targeted by the voice recognition system. The word specifying means (200) specifies words related to the article with reference to the speech recognition information stored in the speech recognition information storage means based on the phoneme symbol string obtained by performing the speech recognition process. Further, the article specifying means (200, 205A) specifies an article designated by a person based on the word specified by the word specifying means. The position detecting means (124, 200, 208) detects the position of a person. Further, the nearby article specifying means (200) specifies an article existing in the vicinity of the person based on the position of the person detected by the position detecting means and the position information stored in the position information storage means. Then, the word specifying means specifies the word by referring only to the speech recognition information including the word related to the article specified by the nearby article specifying means.

  According to the third invention, since the speech recognition is performed by referring to only the speech recognition information including the word related to the article existing in the vicinity of the human to identify the word related to the article, the voice recognition process is performed. It is possible to shorten the time and increase the rate of correct voice recognition.

  The fourth invention is an invention subordinate to the third invention, wherein the word specifying means extracts from the voice recognition information storage means only the speech recognition information including the word related to the article specified by the nearby article specifying means. The voice recognition information stored in the voice recognition information storage means is referred to by referring to the sub voice recognition information storage means stored in the above.

  In the fourth invention, the word specifying means (200) is a sub-voice recognition information storage means for extracting and storing only the voice recognition information including the word related to the article specified by the nearby article specifying means from the voice recognition information storage means. Is referred to, the voice recognition information stored in the voice recognition information storage means is referred to.

  According to the fourth aspect of the invention, since the voice recognition is performed using the sub voice recognition information storage unit that stores only the voice recognition information including the word related to the article existing in the vicinity of the person, the time required for the voice recognition process The rate at which correct speech recognition can be achieved can be increased.

  According to the present invention, the time required for the speech recognition process can be shortened.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a communication robot system (hereinafter simply referred to as “system”) 10 of this embodiment includes a communication robot (hereinafter also simply referred to as “robot”) 12. The robot 12 can access a network 14 such as a wireless LAN. The robot 12 cooperates with the server 20 to identify an article that the human 16 indicates with voice, line of sight, and pointing, and performs an operation such as bringing the article to the human 16.

  The human 16 is wearing a wireless tag 18 indicating who the person is, and a marker for motion capture is attached, although not shown. The markers are typically set on the top of the human head, both shoulders, both elbows, the tip of the index finger of both hands, etc., and these markers together with the entire human 16 are controlled by the camera 120 controlled by the server 20. Taken. In the embodiment, three cameras 120 are provided, take a picture of the person 16 from three directions, and supply the camera image to the server 20.

  The server 20 is coupled to a network 14 such as a wireless LAN, and executes a motion capture process that detects the movement of the marker based on the camera video data input as described above, and also detects a skin color area, for example. Thus, the position of the face of the human 16 can be specified.

  In the system 10, as described above, the robot 12 specifies an article designated by the human 16 as an object. In this embodiment, a book (book) 24 is used as an example of an article that can be an object. A radio tag 18 indicating what kind of book the book is attached to the book 24. The book 24 is stored in a bookshelf 26.

  However, the articles that can be the object are not only the books of the embodiment, but any household article can be considered if it is a home system. Naturally, it can be used not only for home use but also in any place (company, office, factory, etc.) that works with people.

  All articles (books 24) targeted by this system 10 are registered in the article dictionary 122 attached to the server 20. The article dictionary 122 will be described later.

  Further, the human 16 targeted by the system 10 is read by the wireless tag reader 208 (see FIG. 4) via any one of the antennas 124 in which a plurality of wireless tags 18 attached to the human 16 are present. The server 20 grasps the position. That is, the position of the person 16 is grasped slightly roughly depending on which antenna 124 has received the radio wave from the wireless tag 18 attached to the person 16.

  In FIG. 1, one robot 12 is shown for simplicity, but two or more robots may be used. Further, the human 16 is not necessarily limited to one person, and can be identified by the wireless tag 18 and may be plural.

  In the embodiment shown in FIG. 1, the positions of the robot 12, the human 16, the article 24, and the like are expressed using the world coordinates of the space where the system 10 is installed. Since it is performed in coordinates, details will not be described, but the robot 12 performs a coordinate conversion process between the robot coordinates and the world coordinates as necessary in the process described later.

  The hardware configuration of the robot 12 will be described with reference to FIG. FIG. 2 is a front view showing the appearance of the robot 12 of this embodiment. The robot 12 includes a carriage 30, and two wheels 32 and one slave wheel 34 for autonomously moving the robot 12 are provided on the lower surface of the carriage 30. The two wheels 32 are independently driven by a wheel motor 36 (see FIG. 3), and the carriage 30, that is, the robot 12 can be moved in any direction, front, back, left, and right. The slave wheel 34 is an auxiliary wheel that assists the wheel 32. Therefore, the robot 12 can move in the arranged space by autonomous control.

  A cylindrical sensor attachment panel 38 is provided on the carriage 30, and a large number of infrared distance sensors 40 are attached to the sensor attachment panel 38. These infrared distance sensors 40 measure the distance to the sensor mounting panel 38, that is, the object (human being, obstacle, etc.) around the robot 12.

  In this embodiment, an infrared distance sensor is used as the distance sensor, but an ultrasonic distance sensor, a millimeter wave radar, or the like can be used instead of the infrared distance sensor.

  A body 42 is provided on the sensor mounting panel 38 so as to stand upright. Further, the above-described infrared distance sensor 40 is further provided in the upper front upper portion of the body 42 (a position corresponding to a human chest), and measures the distance mainly to a human in front of the robot 12. Further, the body 42 is provided with a support column 44 extending from substantially the center of the upper end of the side surface, and an omnidirectional camera 46 is provided on the support column 44. The omnidirectional camera 46 photographs the surroundings of the robot 12 and is distinguished from an eye camera 70 described later. As this omnidirectional camera 46, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, the installation positions of the infrared distance sensor 40 and the omnidirectional camera 46 are not limited to the portions, and can be changed as appropriate.

  An upper arm 50R and an upper arm 50L are provided at upper end portions on both sides of the torso 42 (position corresponding to a human shoulder) by a shoulder joint 48R and a shoulder joint 48L, respectively. Although illustration is omitted, each of the shoulder joint 48R and the shoulder joint 48L has three orthogonal degrees of freedom. That is, the shoulder joint 48R can control the angle of the upper arm 50R around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48R is an axis parallel to the longitudinal direction (or axis) of the upper arm 50R, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do. Similarly, the shoulder joint 48L can control the angle of the upper arm 50L around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48L is an axis parallel to the longitudinal direction (or axis) of the upper arm 50L, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do.

  In addition, an elbow joint 52R and an elbow joint 52L are provided at the respective distal ends of the upper arm 50R and the upper arm 50L. Although illustration is omitted, each of the elbow joint 52R and the elbow joint 52L has one degree of freedom, and the angle of the forearm 54R and the forearm 54L can be controlled around the axis (pitch axis).

  At the tip of each of the forearm 54R and the forearm 54L, a hand 56R and a hand 56L corresponding to a human hand are provided. Although the detailed illustration is omitted, these hands 56R and 56L are configured to be openable and closable so that the robot 12 can grip or hold an object using the hands 56R and 56L. However, the shape of the hands 56R and 56L is not limited to the shape of the embodiment, and may have a shape and a function very similar to a human hand.

  Although not shown, the front surface of the carriage 30, the portion corresponding to the shoulder including the shoulder joint 48R and the shoulder joint 48L, the upper arm 50R, the upper arm 50L, the forearm 54R, the forearm 54L, the sphere 56R, and the sphere 56L, A contact sensor 58 (shown generically in FIG. 3) is provided. A contact sensor 58 on the front surface of the carriage 30 detects contact of a person or another obstacle with the carriage 30. Therefore, the robot 12 can detect the contact with the obstacle during its movement and immediately stop the driving of the wheel 32 to suddenly stop the movement of the robot 12. Further, the other contact sensors 58 detect whether or not the respective parts are touched. In addition, the installation position of the contact sensor 58 is not limited to the said site | part, and may be provided in an appropriate position (position corresponding to a person's chest, abdomen, side, back, and waist).

  A neck joint 60 is provided at the upper center of the body 42 (a position corresponding to a person's neck), and a head 62 is further provided thereon. Although illustration is omitted, the neck joint 60 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 12, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 62 is provided with a speaker 64 at a position corresponding to a human mouth. The speaker 64 is used for the robot 12 to communicate with humans around it by voice or sound. A microphone 66R and a microphone 66L are provided at a position corresponding to a human ear. Hereinafter, the right microphone 66R and the left microphone 66L may be collectively referred to as a microphone 66. The microphone 66 captures ambient sounds, in particular, the voices of humans who are subjects of communication. Furthermore, an eyeball part 68R and an eyeball part 68L are provided at positions corresponding to human eyes. The eyeball portion 68R and the eyeball portion 68L include an eye camera 70R and an eye camera 70L, respectively. Hereinafter, the right eyeball part 68R and the left eyeball part 68L may be collectively referred to as the eyeball part 68. The right eye camera 70R and the left eye camera 70L may be collectively referred to as an eye camera 70.

  The eye camera 70 captures a human face approaching the robot 12, other parts or objects, and captures a corresponding video signal. In this embodiment, the robot 12 detects the line-of-sight directions (vectors) of the left and right eyes of the human 16 from the video signal from the eye camera 70. Specifically, the line-of-sight detection method is disclosed in Japanese Patent Application Laid-Open No. 2004-255074 as using two cameras, and Japanese Patent Application Laid-Open No. 2006-172209 and Japanese Patent Application Laid-Open No. 2006-285531 as using one camera. However, the details are not important here, so only those publications are cited.

  However, a well-known eye mark recorder or the like may be used for detecting the line-of-sight vector of the human 16.

  The eye camera 70 can be the same camera as the omnidirectional camera 46 described above. For example, the eye camera 70 is fixed in the eyeball unit 68, and the eyeball unit 68 is attached to a predetermined position in the head 62 via an eyeball support unit (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 62, and the other is orthogonal to the one axis and goes straight in a direction in which the front side (face) of the head 62 faces. It is an axis (pitch axis) in the direction to be performed. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 68 or the eye camera 70 is displaced, and the camera axis, that is, the line-of-sight direction is moved. Note that the installation positions of the speaker 64, the microphone 66, and the eye camera 70 described above are not limited to those portions, and may be provided at appropriate positions.

  As described above, the robot 12 of this embodiment includes independent two-axis driving of the wheels 32, three degrees of freedom of the shoulder joint 48 (6 degrees of freedom on the left and right), one degree of freedom of the elbow joint 52 (2 degrees of freedom on the left and right), It has a total of 17 degrees of freedom, 3 degrees of freedom for the neck joint 60 and 2 degrees of freedom for the eyeball support (4 degrees of freedom on the left and right).

  FIG. 3 is a block diagram showing the electrical configuration of the robot 12. With reference to FIG. 3, the robot 12 includes a CPU 80. The CPU 80 is also called a microcomputer or a processor, and is connected to the memory 84, the motor control board 86, the sensor input / output board 88 and the audio input / output board 90 via the bus 82.

  The memory 84 includes a ROM, an HDD, and a RAM (not shown). In the ROM and the HDD, a control program for controlling the operation of the robot 12 is stored in advance. For example, a detection program for detecting the output (sensor information) of each sensor, a communication program for transmitting / receiving necessary data and commands to / from an external computer, and the like are recorded. The RAM is used as a work memory or a buffer memory.

  Furthermore, in this embodiment, the robot 12 is configured to be able to speak and make a gesture to communicate with the human 16, but the memory 84 has a speech for such a speech and gesture. / Gesture dictionary 85A is set.

  The motor control board 86 is constituted by, for example, a DSP, and controls driving of each axis motor such as each arm, neck joint, and eyeball unit. That is, the motor control board 86 receives control data from the CPU 80, and controls two motors (collectively indicated as “right eyeball motor 92” in FIG. 3) that control the angles of the two axes of the right eyeball portion 68R. Control the rotation angle. Similarly, the motor control board 86 receives control data from the CPU 80, and controls two angles of the two axes of the left eyeball portion 68L (in FIG. 3, collectively referred to as “left eyeball motor 94”). ) To control the rotation angle.

  The motor control board 86 receives control data from the CPU 80, and includes a total of four motors including three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48R and one motor for controlling the angle of the elbow joint 52R. The rotation angle of two motors (collectively indicated as “right arm motor 96” in FIG. 3) is controlled. Similarly, the motor control board 86 receives control data from the CPU 80, and includes three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48L and one motor for controlling the angle of the elbow joint 52L. The rotation angles of a total of four motors (collectively indicated as “left arm motor 98” in FIG. 3) are controlled.

  Further, the motor control board 86 receives control data from the CPU 80, and controls three motors that control the angles of the three orthogonal axes of the neck joint 60 (in FIG. 3, collectively indicated as “head motor 100”). Control the rotation angle. The motor control board 86 receives control data from the CPU 80 and controls the rotation angles of the two motors (collectively indicated as “wheel motor 36” in FIG. 3) that drive the wheels 32.

  A hand actuator 108 is further coupled to the motor control board 86, and the motor control board 86 receives control data from the CPU 80 and controls the opening and closing of the hands 56R and 56L.

  In this embodiment, a motor other than the wheel motor 36 uses a stepping motor (that is, a pulse motor) in order to simplify the control. However, a DC motor may be used similarly to the wheel motor 36. The actuator that drives the body part of the robot 12 is not limited to a motor that uses a current as a power source, and may be changed as appropriate. For example, in another embodiment, an air actuator may be applied.

  Similar to the motor control board 86, the sensor input / output board 88 is configured by a DSP and takes in signals from each sensor and gives them to the CPU 80. That is, data relating to the reflection time from each of the infrared distance sensors 40 is input to the CPU 80 through the sensor input / output board 88. The video signal from the omnidirectional camera 46 is input to the CPU 80 after being subjected to predetermined processing by the sensor input / output board 88 as necessary. Similarly, the video signal from the eye camera 70 is also input to the CPU 80. Further, signals from the plurality of contact sensors 58 described above (collectively indicated as “contact sensors 58” in FIG. 3) are provided to the CPU 80 via the sensor input / output board 88. Similarly, the voice input / output board 90 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the CPU 80 is output from the speaker 64. In addition, voice input from the microphone 66 is given to the CPU 80 via the voice input / output board 90.

  The CPU 80 is connected to the communication LAN board 102 via the bus 82. The communication LAN board 102 is configured by a DSP, for example, and provides transmission data provided from the CPU 80 to the wireless communication device 104, and the wireless communication device 104 transmits the transmission data to the server 20 via the network 14. In addition, the communication LAN board 102 receives data via the wireless communication device 104 and provides the received data to the CPU 80. For example, the transmission data may be a signal (command) from the robot 12 to the server 20, or operation history information (history data) regarding communication performed by the robot 12. The reason why the history data is transmitted as well as the command is to reduce the capacity of the memory 84 and to reduce power consumption. In this embodiment, the history data is transmitted to the server 20 every time communication is performed. However, the history data may be transmitted to the server 20 in units of a fixed time or a fixed amount.

  Further, the wireless tag reader 106 is connected to the CPU 80 via the bus 82. The wireless tag reader 106 receives a radio wave superimposed with identification information transmitted from the wireless tag 18 (RFID tag) via an antenna (not shown). Then, the RFID tag reader 106 amplifies the received radio wave signal, separates the identification signal from the radio wave signal, demodulates (decodes) the identification information, and supplies the identification information to the CPU 80. According to FIG. 1, the wireless tag 18 is attached to the person 16 in the reception of the company where the robot 12 is disposed or in the living room of a general household, and the wireless tag reader 106 detects the wireless tag 18 within the communicable range. To do. Note that the wireless tag 18 may be an active type or a passive type that is driven according to a radio wave transmitted from the wireless tag reader 106.

  The hardware configuration of the server 20 will be described with reference to FIG. As shown in FIG. 4, the server 20 includes a CPU 200. The CPU 200, also called a processor, is connected to the memory 204, the camera control board 206, the wireless tag reader 208, the LAN control board 210, the input device control board 212, and the monitor control board 214 via the bus 202.

  The CPU 200 governs overall control of the server 20. The memory 204 comprehensively shows ROM, RAM, HDD, and the like, and records a program for operating the server 20 and functions as a work area when the CPU 200 operates. The camera control board 206 is for controlling the camera 120 connected to the control board 206.

  The wireless tag reader 208 receives a radio wave superimposed with identification information transmitted from the wireless tag 18 attached to the person 16 or the article (book) 24 via the antenna 124 connected to the control board 208. Then, the RFID tag reading device 208 amplifies the received radio wave signal, separates the identification signal from the radio wave signal, demodulates (decodes) the identification information, and gives it to the CPU 200. The antenna 124 is placed throughout the reception of the company where the robot 12 is placed, and in each room of a general household, and receives radio waves from all articles (books) 24 targeted by the system 10 and the wireless tag 18 of the human 16. It can be done. Therefore, although there are a plurality of antennas 124, they are comprehensively shown in FIGS.

  The LAN control board 210 controls the wireless communication device 216 connected to the control board 210 so that the server 20 can access the external network 14 wirelessly. Further, the input device control board 212 controls input by, for example, a keyboard or a mouse as an input device connected to the control board 212. The monitor control board 214 controls output to a monitor connected to the control board 214.

  The server 20 is connected to an article dictionary 122 and a voice recognition dictionary 126 (see FIG. 1) attached to the server 20 by an interface (not shown).

  In the memory 204, an article local dictionary 205A, a speech recognition local dictionary 205B, an utterance pattern dictionary 205C, an utterance dictionary 205D, an individual correct / incorrect DB 205E, and a speech recognition rate DB 205F are set.

  The article local dictionary 205A is a dictionary in which contents extracted from an article dictionary 122 described later are registered. When the robot 12 recognizes the person 16, the server 20 extracts only the information of the article (book) 24 existing in the vicinity of the person 16 from the article dictionary 122 and registers it in the article local dictionary 205 </ b> A. The voice recognition local dictionary 205B is a dictionary in which contents extracted from a voice recognition dictionary 126 described later are registered. When the robot 12 recognizes the person 16 and creates the article local dictionary 205A, the server 20 extracts information necessary for voice recognition of words registered in the article local dictionary 205A from the voice recognition dictionary 126. Register in the speech recognition local dictionary 205B. Therefore, the article local dictionary 205A and the speech recognition local dictionary 205B are dynamically rewritten in accordance with the change in the position of the person 16. In this way, the speech recognition local dictionary 205B is created from the speech recognition dictionary 126, and the number of words (phoneme symbol strings) to be speech-recognized is reduced by reducing the dictionary used for speech recognition. It is possible to shorten the time required for the voice recognition and to increase the rate of correct voice recognition.

  The utterance pattern dictionary 205C is a dictionary for determining whether or not the content of the utterance performed by the human 16 is a specific pattern. The utterance dictionary 205D stores information necessary for the server 20 to determine the content of the voice that the robot 12 causes the human 16 to utter. Further, the personal correctness DB 205E stores, for each ID of the person 16, whether or not the system 10 has finally succeeded in specifying the article (book) 24 designated by the person 16. The speech recognition rate DB 205F stores a recognition rate in speech recognition for each word registered in the speech recognition dictionary 126, that is, a rate at which the word can be correctly recognized by actual speech recognition.

  Next, the article dictionary 122 will be described with reference to FIG. The article dictionary 122 shown in FIG. 5 assigns an ID such as a Ucode to one of the articles, and stores necessary information such as the name, attribute, and position (coordinates) for each article as a character string. Registered as. The U-code is specifically a 128-bit number, and can individually identify articles that are 1 trillion times more than 1 trillion times 340 trillion. However, the ID used for the article dictionary 122 does not necessarily need to be such a U-code, and may consist of a combination of appropriate numbers and symbols.

  Such an article dictionary 122 registers all articles, for example, household articles, which are objects to be identified by the system 10 (the robot 12 and the server 20) with IDs and character strings. Equivalent to.

  In the article dictionary 122, information about one article (book) 24 is registered as one record. For example, in addition to the book ID, items such as “name”, “attribute”, “author”, and “publisher” are stored in one record. In the “attribute” item, words that supplementally describe the book such as the color and appearance of the book cover are stored. Information on each item is stored in words in text format. However, information enclosed in double quotation marks (“) in the“ attribute ”item is stored as a phonemic symbol format (phoneme symbol string) word in speech recognition.

  The information of the phoneme symbol string is information generated in the process of the server 20 recognizing the voice uttered by the human 16 picked up by the robot 12 with the microphone 66, and the voice recognition local dictionary 205B (voice recognition dictionary 126). Is not registered, and voice recognition could not be performed. However, when it is possible to determine which book the human 16 is pointing to by the line of sight or pointing of the human 16, the voice that could not be recognized The corresponding phoneme symbol string is stored as the “attribute” of the record of the book.

  Further, what is appended in parentheses after the phoneme symbol string is information on the ID of the person 16 stored in association with the phoneme symbol string. This indicates that the voice indicated by the phonetic symbol string is emitted by the person 16 indicated by the ID.

  As described above, the article dictionary 122 also stores information on the location where each article (book) 24 exists, and one record of the article dictionary 122 contains the article (book) 24 (not shown). It also includes an item that remembers where it exists.

  Next, the speech recognition dictionary 126 will be described with reference to FIG. In general, a speech recognition dictionary includes a word dictionary and a grammar dictionary, but the speech recognition dictionary 126 is a word dictionary. A description of the grammar dictionary is omitted. As shown in FIG. 10, the speech recognition dictionary 126 stores an item for storing a word in a text format, an item for storing a word in a phoneme symbol format (phoneme symbol string) corresponding to the text format word, and a word in the text format. Is a record including an item for storing information indicating whether or not it is the name of an article, for example, the title (name) of a book. In the item storing information indicating whether or not the name of the article is stored in the item storing the text format word, the name of the article is stored in the item storing the text format word. In this case, “1” is stored. Otherwise, “0” is stored.

  In the speech recognition process, the input speech is decomposed into phonemes, and a symbol representing the phoneme is generated for each decomposed phoneme. The symbol string corresponding to the input speech word is a phoneme symbol string stored in the speech recognition dictionary 126. In the speech recognition process, the phoneme symbol string closest to the phoneme symbol string generated from the speech is identified in the speech recognition dictionary 126 (speech recognition local dictionary 205B) and stored in correspondence with the identified phoneme symbol string. The word is output as a speech recognition result.

  As described above, in the system 10, when the person 16 indicates the article (book) 24 by voice, line of sight, and pointing, the robot 12 and the server 20 cooperate to provide the article (book) indicated by the person 16. 24 is specified, and the robot 12 transports the specified article (book) 24 to the person 16. Hereinafter, the exchange between the human 16 and the system 10 may be referred to as communication.

  More specifically, in the system 10, when the human 16 approaches the robot 12, the robot 12 recognizes the human 16 by the wireless tag 18. The server 20 includes an article dictionary 122 in which all articles (books) 24 targeted by the system 10 are registered, and a voice recognition dictionary 126 in which words for identifying the articles (books) 24 by voice recognition are registered. It is attached. When the robot 12 recognizes the human 16, the robot 12 instructs the server 20 to create local dictionaries (article local dictionary 205 </ b> A, voice recognition local dictionary 205 </ b> B) of the article dictionary 122 and the voice recognition dictionary 126.

  Upon receiving an instruction to create a local dictionary, the server 20 specifies the position of the person 16 recognized by the robot 12, and the article (book) 24 within a predetermined range from the specified person 16, for example, within a radius of 5 m. Are extracted from the article dictionary 122 to create the article local dictionary 205A. Then, only the information necessary for voice recognition of the article (book) 24 registered in the article local dictionary 205A is extracted from the voice recognition dictionary 126 to create the voice recognition local dictionary 205B.

  Thereafter, the robot 12 utters, for example, “Let's bring some books?” To the recognized human 16. In response to this utterance, the human 16 replies, for example, “Bring a chemical mystery” while pointing at the article (book) 24 while looking at the article (book) 24 that he / she wants to bring.

  Then, the robot 12 recognizes the voice of the human 16 saying “Please bring a mystery of chemistry”, estimates the gaze of the human 16, and estimates the direction in which the pointing finger is pointing. The article (book) 24 being identified is specified.

  When the article (book) 24 instructed by the person 16 is identified, the server 20 confirms the article (book) 24 identified to the person 16 and the content of the voice uttered by the robot 12, for example, “white book. The robot 12 speaks while pointing to the article (book) 24 (the mysterious book of chemistry 24).

  At this time, the server 20 indicates the content of the utterance to the human 16 when the human 16 next instructs the robot 12 of the article (book) 24 (chemical mysterious book 24). It is assumed that a word to be used as a word for specifying the mysterious book of chemistry 24) is included.

  For example, when the utterance content for confirming the article (book) 24 (the mysterious book of chemistry 24) identified by the robot 12 is “Is it a white book?”, The system 10 When pointing to a book of “Chemical Mystery”, I hope that with the pointing, I will say “Take that white book” instead of saying “Take a chemical mystery”. This is based on the knowledge that humans tend to imitate the utterance content of robots. In this case, the word that the human 16 expects to use next time to identify the article (book) 24 is “white”. The word “white” has no “white” article (book) 24 in addition to the book “chemical mystery” 24 in the vicinity of the human 16, and the recognition rate in the speech recognition of the word “white”. Is selected by the server 20 when it is high.

  By the way, when the human 16 speaks with a pointing hand, “Take a chemical mystery” or the like, the word “chemical mystery” is used instead of the word “chemical mystery”. It is possible to use the word “Bakenazo”, which is an abbreviation for, and say “Bring a Bakenazo”. In this case, if the word “bakenazo” is not registered in the article dictionary 122 and the speech recognition dictionary 126, the system 10 recognizes the voice “bakenazo”, and “bakenazo” points to the book of “chemistry mystery”. I can't be certain.

  Even in such a case, the system 10 can identify the article (book) 24 indicated by the person 16 based on the line of sight of the person 16 or the direction of pointing. Therefore, when there is a word that could not be recognized by speech such as “bakenazo”, the word that could not be recognized by using the result identified based on the direction of the line of sight or the pointing is used. Register in the recognition dictionary 126.

  When the robot 12 speaks, for example, “Is it a white book?” To confirm the article (book) 24 identified by the system 10, the human 16 speaks “Yes” or “No” and the robot Reply to 12. The server 20 recognizes the voice in the response of the human 16 and determines whether or not the article (book) 24 specified by the system 10 is the one designated by the human 16. If the article (book) 24 identified by the system 10 is not designated by the person 16, it is confirmed whether or not the next candidate article (book) 24 is designated by the person 16. To do. On the other hand, when the article (book) 24 specified by the system 10 is the one designated by the person 16, the robot 12 carries the article (book) 24 to the person 16.

  Next, operations of the robot 12 and the server 20 in the embodiment shown in FIG. 1 will be described with reference to flowcharts shown in FIGS.

  In the first step S1 of FIG. 15, the CPU 80 (FIG. 3) of the robot 12 determines whether or not the person 16 (FIG. 1) has been recognized in accordance with the sensor input from the sensor input / output board 88 also illustrated in FIG. To do. Specifically, for example, when the human body is detected by the infrared sensor 40 and the wireless tag reader 106 recognizes the wireless tag 18 worn by the human 16 at that time, it is determined that the human 16 has been recognized. At this time, the robot 12 reads the user ID (for example, (456... 0000004)) of the human 16 from the wireless tag 18.

  If YES is determined in step S1, the CPU 80 of the robot 12 transmits an instruction to create a local dictionary, that is, the article local dictionary 205A and the voice recognition local dictionary 205B to the server 20 via the network 14 in the next step S3. To do. At this time, simultaneously with the local dictionary creation instruction, the user ID (456... 0000004) of the person 16 determined to have been recognized in step S1 is also transmitted to the server 20.

  When the local dictionary creation instruction and the user ID are transmitted from the robot 12, the server 20 determines in step S31 in FIG. 16 that the CPU 200 has received the local dictionary creation instruction. In step S33, the robot 12 The article local dictionary 205A and the voice recognition local dictionary 205B dedicated to the person 16 used for communication with the person 16 recognized by the person 16 are created.

  The process of “create local dictionary” in step S33 is executed according to the procedure shown in the flowchart of FIG. First, in step S81 in FIG. 18, a record of an article (book) 24 near the person 16 specified by the user ID transmitted from the robot 12 is extracted from the article dictionary 122 to create an article local dictionary 205A.

  At this time, the CPU 200 drives the wireless tag reading device 208 and receives radio waves from the wireless tag 18 attached to the person 16 via the antenna 124 to specify the user ID and the position of the person for each person 16. Then, the positions of the humans 16 recognized by the robot 12 are specified based on the user IDs and information on the positions of the humans 16 specified as described above and the user IDs transmitted from the robot 12.

  When the position of the person 16 recognized by the robot 12 is specified, next, an item in which the position of the article (book) 24 of each record stored in the article dictionary 122 is stored is referred to as shown in FIG. All records of articles (books) 24 located near the person 16 recognized by the robot 12 are extracted from the article dictionary 122 to create an article local dictionary 205A as shown in FIG.

  After the article local dictionary 205A is created in this way, the CPU 200 creates a list 300 of “words registered in the article local dictionary 205A” in step S83. FIG. 7 is an illustrative view showing a list 300 of the words. This word list 300 is used when the speech recognition local dictionary 205B is created from the speech recognition dictionary 126. As can be seen from FIGS. 6 and 7, “words registered in the article local dictionary 205A” registered in the word list 300 are “name”, “attribute”, “ It is a word in text format stored in each of “author” and “publisher”. However, as shown in FIG. 7, this word list 300 includes not only words consisting of text strings but also phoneme symbol strings described above (in the example of FIG. 7, “bakenazo”, “saiai” ) Is also registered together with the user ID (in the example of FIG. 7, (456... 0000004), (456... 0000003)). As described above, these phoneme symbol strings (and user IDs) are those in which a phoneme symbol string corresponding to an unregistered word is registered later because no word is registered in the article dictionary 122. .

  In step S85, a phoneme symbol string associated with a user ID other than the user ID of the human 16 recognized by the robot 12 is deleted from the phoneme symbol strings registered in the word list 300. If the user ID of the human 16 recognized by the robot 12 is (456... 0000004), as shown in FIG. 8, the “saiai” associated with the user ID (456. "Is deleted. The name of the book 24 (Chemical Eye) called “saiai” unique to the human 16 whose user ID is (456... 0000003) is the human 16 whose user ID recognized by the robot 12 is (456... 0000004). This is because it is considered unnecessary for the voice recognition in communication with the.

  Next, in step S87, the CPU 200 eliminates redundancy in the word list 300. That is, in the example of FIG. 7, overlapping portions such as “white” and “hard cover” registered in the word list 300 are deleted as shown in FIG.

  When the word list 300 is completed in this manner, the speech recognition local dictionary 205B is created from the speech recognition dictionary 126 using the word list 300.

  First, in step S89, the CPU 200 increments the counter Ct1 set in the memory 204. In the initial state, the counter Ct1 is “0”. The counter Ct 1 counts the number of words registered in the word list 300 and functions as a pointer to the word list 300. Therefore, different words are designated in the word list 300 according to the count value of the counter Ct1. It should be understood that the following operations are performed for each word until the count value of the counter Ct1 is equal to the number of words “L” registered in the word list 300.

  In step S91, it is determined whether or not the word registered in the word list 300 is a phoneme symbol string. For example, if the word registered in the word list 300 is the word “global warming” expressed in a text format as shown in FIG. 8, NO is determined in step S91. Then, in step S93, the speech recognition dictionary 126 (see FIG. 10) is searched using the word “global warming” in text format as a key, and a record including an item storing the word “global warming” is extracted. Then, it is registered in the voice recognition local dictionary 205B as shown in FIG.

  On the other hand, if the word registered in the word list 300 is the word “bakenazo” expressed in the phoneme symbol form as shown in FIG. 8, “YES” is determined in the step S91. Then, in step S95, the speech recognition dictionary 126 is searched using the word “bakenazo” that is a phoneme symbol string as a key, and a record including an item in which the word “bakenazo” is stored is extracted. As shown in FIG. Register in the recognition local dictionary 205B.

  In this way, the article local dictionary 205A is created from the article dictionary 122, and the voice recognition local dictionary 205B is created from the voice recognition dictionary 126.

  On the other hand, returning to FIG. 15, after the robot 12 has transmitted a local dictionary creation instruction to the server 20 in step S <b> 3, the CPU 80 stores the utterance / gesture dictionary 85 </ b> A set in the memory 84 in step S <b> 5. The speaker 64 is used to make a speech such as “Would you like to bring some books?” Thereafter, if the human 16 has made an utterance such as “bring fireworks master”, the robot 12 recognizes the instruction of the human 16, and the CPU 80 determines “YES” in step S7. At this time, the human 16 instructs which book he / she wants to bring by pointing to the book while looking at the book with an utterance “bring fireworks master”.

  If the CPU 80 determines YES in step S7, then in step S9, along with the voice information that the human 16 has instructed, “Give fireworks master”, an instruction to perform voice recognition and estimation of the line of sight and pointing is performed. Send to server 20.

  In the server 20, when the CPU 200 determines in step S 35 of FIG. 16 that the instruction for voice recognition and the estimation of the line of sight and pointing is received from the robot 12 together with the voice information, the “200 The process of “estimation” and the process of “estimation of article by line of sight and pointing” in step S39 are executed in parallel.

  The process of “estimating an article by voice recognition” is executed according to the procedure shown in the flowchart of FIG. First, in step S <b> 111, the CPU 200 recognizes voice information transmitted from the robot 12 as “Take fireworks master”. Specifically, the speech “Take fireworks master” is divided into phonemes, and phoneme symbols corresponding to each phoneme are generated. Then, a phoneme symbol string “hanabinotakumi” in the generated phoneme symbol string “hanabinotakumi mottekite” is identified as an object with reference to a grammar dictionary (not shown). Next, the phonetic symbol string “hanabinotakumi” is used as a key to search the speech recognition local dictionary 205B (see FIG. 11), and a record having an item storing the word of the nearest phoneme symbol string is specified. Then, “fireworks master”, which is a text-format word stored in the specified record, is obtained. At this time, based on the information stored in the “article name” item of the specified record, is the name of the article that has been voice-recognized, that is, the title (name) of the book 24? It can be determined whether or not. Based on this determination, the flag tw is set later. However, setting the flag tw later is convenient for the representation of the flowchart, and the flag tw may be set in step S111.

  For example, when the human 16 instructs “bring a fireworks master” instead of “bring a black firework”, “black” is similarly obtained as a result of the speech recognition in step S111. The voice recognition method is not particularly limited, and an existing method can be adopted.

  Also, instead of saying "Take Fireworks Takumi", human 16 uttered "Take Hanatak" using the word "Hanataku" which is a unique abbreviation for the word "Fireworks Takumi". In this case, since the phonetic symbol string “hanataku” corresponding to the word “hanataku” is not registered in the speech recognition local dictionary 205B, speech recognition cannot be performed.

  As described above, when there is a word that cannot be recognized by the voice, the CPU 200 determines YES in step S113, and in step S115, determines whether or not the utterance content of the person 16 who has recognized the voice is a specific pattern. To do. Here, the specific pattern of the utterance content is, for example, “Bring XX” or “Request XX”, and will include a word (XX) that specifies an article designated by the human 16. It is an utterance pattern. Whether or not the content of the speech uttered by the human 16 corresponds to these specific patterns is determined by the utterance pattern set in the memory 204 together with the grammatical analysis of the utterance content using a grammar dictionary (not shown) in the speech recognition processing. This can be determined by referring to the dictionary 205C.

  If YES is determined in step S115, it is set on the memory 204 in step S117 to indicate that there is a word that could not be recognized by voice and that would specify an article designated by the person 16. The numerical value “1” is stored in the flag nw, and the flag nw is turned on. The initial state of the flag nw is an off state (numerical value “0”). In step S119, the phoneme symbol string of the word that cannot be recognized by the speech recognition process in step S111 is stored in the work area wa set in the memory 204. If NO is determined in step S115, steps S117 and S119 are skipped.

  On the other hand, if NO is determined in step S113, that is, if it is determined that there is no word that cannot be voice-recognized, in step S121, the CPU 200 determines whether the voice-recognized word is the name of the article, that is, the title (name) of the book 24. Judging. Whether or not the speech-recognized word is the title (name) of the book 24 is determined based on information recorded in the “article name” item included in each record of the speech recognition local dictionary 205B.

  If YES is determined in the step S121, a numerical value “1” is stored in the flag tw set in the memory 204 in order to indicate that the title (name) of the book 24 is recognized in step S123, and the flag tw is set. Turn on. The initial state of the flag tw is an off state (numerical value “0”). If NO is determined in step S121, step S123 is skipped.

  In step S125, the CPU 200 refers to the article local dictionary 205A based on the text-formatted word obtained as a result of the speech recognition in step S111, for example, “Fireworks Master”, and the article (book) indicated by the human 16 Articles (books) 24 as 24 candidates are selected. Specifically, the text-format word obtained in step S111, for example, “article of fireworks” is used as a key to search the article local dictionary 205A, and the text-format word of “artificial fireworks” is stored in the item. Identify the records that are In the example of FIG. 6, since the record with the article ID “123... 0000046” stores the word “fireworks master” in the item “name”, the article with the article ID “123. Book) 24 is identified. The article (book) 24 having the article ID specified in this way is selected as a candidate for the article designated by the human 16, and the ID of the article (book) 24 is registered in the article list A (not shown).

  Here, if the word obtained as a result of the speech recognition in step S111 is “white”, for example, in the case of the article local dictionary 205A in FIG. Since there are a plurality of records storing the word “N”, the article IDs included in these records, that is, articles with the article IDs “123… 0000035”, “123… 0000091”, and “123… 0000102” (books) ) 24 is selected as a candidate for the article designated by the human 16. In this case, the IDs of the plurality of articles selected in this way are registered in the article list A (not shown).

  In step S127, it is determined whether or not the candidate for the article (book) 24 thus selected exists in the article list A. If YES is determined in step S127, in step S129, the CPU 200 indicates a numerical value “1” in the flag ca set in the memory 204 to indicate that there is a candidate for the article (book) 24 as a result of the speech recognition. Is stored and the flag ca is turned on. Note that the initial state of the flag ca is an off state (numerical value “0”). If NO is determined in step S127, step S129 is skipped. In this manner, the article (book) 24 instructed by the person 16 is estimated by voice recognition.

  Returning to FIG. 16, the processing of “estimation of article by line of sight and pointing” in step S39 is executed according to the procedure shown in the flowchart of FIG. First, the CPU 200 increments the counter Ct2 set in the memory 204 in step S141. In the initial state, the counter Ct2 is set to “0”. The counter Ct2 counts the number of articles existing in the vicinity of the person 16, and functions as a pointer for the articles local dictionary 205A (see FIG. 6). Therefore, different articles are designated in the article local dictionary 205A by the count value of the counter Ct2. It should be understood that the following operations are performed for each article until the count value of counter Ct2 is equal to the number of articles “m” listed in article local dictionary 205A.

  Subsequent to step S141, the CPU 200 executes in parallel the operation of estimating the line of sight of the human 16 to obtain the certainty thereof and the operation of estimating the pointing direction and obtaining the certainty thereof. For the sake of convenience, description will be made in the order of estimating the line of sight first and then estimating the pointing direction.

  The processing from step S141 to step S153 in FIG. 20 is performed at a predetermined repetition time (t1, t2, t3) after receiving an instruction for performing speech recognition and gaze and pointing estimation from the robot 12 in step S35 in FIG. ,..., Tm), but in the embodiment, it is executed at 50 Hz (50 times per second). Note that the processing from step S141 to step S153 is performed, for example, when the robot 12 speaks in step S5 in FIG. 15, a notification informing the fact is transmitted from the robot 12 to the server 20, and the server that has received the notification. 20 may be executed at regular repetition times (t1, t2, t3,..., Tm) after receiving the notification.

  In step S <b> 143, the CPU 200 receives line-of-sight vector information indicating the direction of the line of sight of the human 16 from the robot 12. Although not explicitly shown in the flow diagram, in the robot 12, the CPU 80 processes the camera video from the eye camera 70, for example, according to any of the methods described in the above-mentioned publications, respectively, Estimate the eye gaze vector. The line-of-sight directions of the left and right eyes are indicated by straight lines L1 and L2 in FIG. In this way, the robot 12 repeatedly estimates each line of sight L1 and L2 at predetermined time intervals, and transmits line-of-sight vector information indicating the direction of the line of sight to the server 20.

  In the server 20, in step S145, the CPU 200 calculates the distance between the line of sight L1 and L2 and the article that the counter Ct2 is currently pointing in the article local dictionary 205A. In this calculation, the CPU 200 uses the line-of-sight vector information received from the robot 12 by converting the coordinate from the robot coordinate system to the world coordinate system. In calculating the distance, information on the position of the article (book) 24 recorded in the record of the article (book) 24 of the article local dictionary 205A is used.

  On the other hand, in order to estimate the pointing direction, in step S147, first, the CPU 200 identifies the arm on which the human 16 has performed the pointing operation. Specifically, referring to the motion capture data, for example, the arm on the side where the difference between the fingertip and shoulder height of the human 16 is small is estimated as the pointing arm. This is because, when performing a pointing operation, it is considered that the operation of lifting the arm first. In this way, after estimating which arm is used to perform the pointing operation in step S147, the CPU 200 estimates the pointing direction in the next step S149.

  In this embodiment, as shown in FIG. 13, a straight line L3 passing through the fingertip of the pointing arm and the center (center of gravity) of the face and a straight line L4 passing through the fingertip of the pointing arm and the elbow of the arm are assumed. Then, the straight lines L3 and L4 are estimated with reference to the motion capture data. In the next step S151, the distance between each straight line L3 and L4 and each article is calculated.

  The above-described steps S143 to S145 and steps S147 to S151 are performed every repetition time (t1, t2, t3,..., Tm). Then, for each repetition time (t1, t2, t3,..., Tm), an article having a minimum distance from the lines L1, L2, L3, and L4 is obtained. In each line, a high certainty factor (“◯” in FIG. 14) is assigned to the minimized article. In this way, for example, a certainty factor table as shown in FIG. 14 is created.

  If the certainty factor table is created by calculating the article having the shortest distance for each straight line as described above, the certainty factor (○) may be given to two or more straight lines for one article. As a result, an article list B (not shown) as will be described later in step S155 can be created.

  In the certainty factor table of FIG. 14, the certainty factor evaluated for each of the lines of sight L1 and L2 can be referred to as “gaze certainty factor”, and the certainty factor evaluated for each of the pointing direction lines L3 and L4 is It can be said that it is the “direction direction certainty”.

  Referring to the example shown in FIG. 14, an article having an ID of “123... 0000001”, in the example, a book named “global warming” shown in FIG. Although the distance to the book is minimized, it can be determined that no line is closest to the book in other time intervals. For the next article having an ID of “123 ... 0000046”, in the example, the book named “Fireworks Takumi” shown in FIG. It can be seen that this article has been approached again. In this way, the certainty factor table shown in FIG. 14 is created in steps S145 and S151.

  In step S <b> 155, the CPU 200 refers to the certainty factor table shown in FIG. 14 and specifies the article (book) 24 that the person 16 thinks instructed at that time. Specifically, the number of times that ○ is included in both the line of sight (L1 or L2) and the pointing (L3 or L4) in the order in which the confidence evaluations (circles in FIG. 14) are simply large or the repetition time. An article list B (not shown), which is a list of article IDs, is created in the descending order.

  This confidence evaluation will be described with reference to the example shown in FIG. 14. For example, if the book named “global warming” having the ID “123... The degree of certainty of the book “Fireworks Takumi” with ID “123... 0000046” is “3”. Accordingly, in this case, ID “123... 0000046” and ID “123... 0000001” are registered in the article list B in this order.

  However, when the number of certainty factors (circles) is the same, or when the number of certainty factors (circles) is smaller than a predetermined threshold, for example, each repetition shown in FIG. What is necessary is just to specify the articles | goods to which confidence is provided in the half or more of all the sections of time as a target object.

  In this manner, the article list A is created by the process of “estimating articles by voice recognition” in step S37, and the article list B is created by the process of “estimating articles by line of sight and pointing” in step S39. Next, in step S41 of FIG. 16, the CPU 200 creates a candidate article list C (not shown), which is a list of article candidates designated by the human 16. The process of creating the candidate article list C is executed according to the procedure shown in the flowchart of FIG.

  First, in step S161, the CPU 200 determines whether or not the flag ca set in the memory 204 is on, that is, whether or not there is an article estimated from the result of speech recognition, in other words, in the article list A. It is determined whether or not the article ID is registered.

  If NO is determined in step S161, there is no article estimated based on the result of the voice recognition. Therefore, in step S163, a candidate article list C is created based on the line-of-sight certainty and the pointing direction certainty. That is, the candidate article list C is created with the contents of the article list B as they are.

  On the other hand, if “YES” is determined in the step S161, the CPU 200 next determines whether the flag tw is in an on state in step S165, that is, whether the title (name) (name of the article) of the book 24 is recognized by voice recognition. No, in other words, whether the ID of the article registered in the article list A obtained as a result of the speech recognition is determined based on the title (name) of the book 24 included in the utterance content of the person 16 Judging.

  If YES is determined in step S165, in step S167, the CPU 200 prioritizes the voice recognition result over the line-of-sight certainty and the pointing direction certainty, and creates the candidate article list C. That is, the candidate ID is registered first in the candidate article list C so that the ID of the article registered in the article list A is higher, and then the ID of the article registered in the article list B is lower. To do. It is more reliable to estimate the article (book) 24 by voice recognition of the title (name) of the book 24 named by the human 16 than to estimate the article (book) 24 based on the line of sight or pointing. Because it is considered. Note that if there are duplicate product IDs in the product list A and the product list B, the product IDs registered in the product list B are not registered in the candidate product list C.

  On the other hand, if NO is determined in step S165, the CPU 200 creates the candidate article list C in step S169 by prioritizing the line-of-sight certainty and the pointing direction certainty over the result of voice recognition. In other words, the product ID registered in the product list B is first registered in the candidate product list C so that the ID of the product registered in the product list B is higher, and then the product ID registered in the product list A is registered lower. To do. Note that if there are duplicate product IDs in the product list A and the product list B, the product IDs registered in the product list A are not registered in the candidate product list C.

  When the candidate article list C is created in this way, the CPU 200 next selects the article (book) 24 that is the first candidate from the candidate article list C in step S43 of FIG. In the first step S43, the ID of the article (book) 24 registered at the top of the candidate article list C is selected. In step S43 after the second time, the ID of the article (book) 24 registered after the second candidate article list C is selected.

  In step S45, it is determined whether or not an article (its ID) is selected in step S43, that is, whether or not a candidate article (book) 24 exists. If YES is determined in the step S45, the CPU 200 next acquires the position information of the article selected in the step S43 in a step S47. That is, the CPU 200 searches the article local dictionary 205A using the ID of the article (book) 24 selected in step S43 (hereinafter referred to as “selected article (book) 24”) as a key, whereby the article ID is obtained. Information on the position of the article (book) 24 to be shown is acquired.

  Next, in step S49, the CPU 200 causes the robot 12 to speak to confirm to the human 16 whether or not the “selected article (book) 24” is the article (book) 24 instructed by the human 16. Determine the utterance content. The content of this utterance performed by the robot 12 includes a word (hereinafter referred to as a “specific word”) that identifies the article (book) 24 that the server 20 presumes that the human 16 has instructed. Referring to FIG. 6, it is assumed that the article (book) 24 estimated by the server 20 to be instructed by the human 16 is a book “Fireworks Master” whose article ID is “123... 0000046”. In this case, in step S49, for example, utterance contents such as “It is a book of fireworks master”, “It is a black book”, “It is a book of hard cover”, etc. are determined. In this example, “artificial fireworks”, “black”, and “hard cover” are the above-mentioned “specific words”, respectively. The process of “determination of utterance content” is executed according to the procedure shown in the flowchart of FIG.

  Referring to FIG. 22, first, in step S181, CPU 200 refers to personal correctness DB 205E set in memory 204 to determine whether or not communication with human 16 recognized by robot 12 is the first time. That is, it is determined whether or not it is the first time that the article (book) 24 indicated by the person 16 is specified. The personal correctness DB 205E is a record of communication results between the robot 12 and the human 16. That is, the personal correctness DB 205E includes success or failure indicating whether the system 10 can correctly estimate the ID of the person 16 who communicated with the robot 12 and the article (book) 24 indicated by the person 16. The success is recorded for each communication. If the ID of the person 16 is not recorded in the personal correctness DB 205E, it is determined that the person 16 is the first meeting.

  If YES is determined in step S181, that is, if it is determined that the human 16 is the first meeting, the CPU 200 determines the “specific word” of the article (book) 24 as the title (name) of the book 24 in step S183. The contents of the title (name) of the book 24 can be obtained by searching the article local dictionary 205A for the ID of the article (book) 24 selected in step S43 in FIG. 16 ("selected article (book) 24"). it can.

  In step S203, using the utterance dictionary 205D set in the memory 204, based on the title (name) of the book 24, which is the "specific word" determined in step S183, for example, "fireworks master". Decide the utterance content such as "It is a fireworks master's book". In step S203, utterance content information is also generated for the robot 12 to utter the determined utterance content accordingly.

  On the other hand, if NO is determined in step S181, that is, if the ID of the person 16 is recorded in the personal correctness DB 205E, in step S185, the CPU 200 refers to the personal correctness DB 205E in the same way. The success rate in communication is calculated, and it is determined whether or not the success rate is 70% or more, for example. If NO is determined in step S185, as described above, the "specific word" of the article (book) 24 is determined as the title (name) of the book 24 in step S183, and the utterance content is determined in step S203. .

  On the other hand, if YES is determined in the step S185, a word list D (not shown) relating to the “selected article (book) 24” is created from the article local dictionary 205A in a step S187. For example, if the “selected article (book) 24” is a book 24 of “artist of fireworks” whose article ID is “123... 0000046” with reference to FIG. The words “Fireworks Takumi”, “Black”, “Hardcover”, “Shiro Kondo”, and “ATR Publishing Co., Ltd.” are registered in the record with the article ID “123 ... 0000046” in the dictionary 205A. Is done.

  After the word list D is created, the CPU 200 next selects words related to the article (book) 24 existing in the vicinity of the person 16 among the words registered in the word list D in step S189. Delete the same word. An article (book) 24 registered in the article local dictionary 205A exists in the vicinity of the person 16. Assuming that the “selected article (book) 24” is a book 24 called “artisan of fireworks” whose article ID is “123... 0000046” with reference to FIG. The word “hard cover” and the word “ATR Publishing Co., Ltd.” are registered.

  Here, the “specific word” of the article (book) 24 is determined to be the word “hardcover” or the word “ATR Publishing Co., Ltd.”, and the robot 12 “is a hardcover book” or “ATR Publishing Co., Ltd.” It ’s a book. ” Then, as can be seen from the article local dictionary 205 </ b> A in FIG. 6, there are a plurality of “hardcover books” and “books of ATR Publishing Co.” in the vicinity of the person 16, so the person 16 is based on the utterance of the robot 12. Thus, it is not known which article (book) 24 the robot 12 indicates. Therefore, the same word as the word recorded in the record other than the “selected article (book) 24” recorded in the article local dictionary 205A is deleted from the word list D.

  Next, in step S191, the CPU 200 deletes words composed of less than six phonemes from the word list D, that is, words that are decomposed into less than six phonemes in the speech recognition process. This is because words with a small number of phonemes are difficult to recognize correctly. Note that the number of phonemes to be decomposed can be determined based on the number of phoneme symbols constituting the phoneme symbol string corresponding to the word that is found by referring to the word in the speech recognition local dictionary 205B. However, the phoneme symbol string shown in the speech recognition local dictionary 205B in FIG. 11 is expressed in Roman letters using alphabets for convenience of explanation, and the alphabet shown in FIG. 11 does not represent phoneme symbols. Absent. In step S193, words starting with “u” in Hiragana are further deleted from the word list D. This is because words beginning with “u” are difficult to recognize correctly.

  Next, in step S195, the CPU 200 determines whether or not a word exists in the word list D. If NO is determined in step S195, a word to be “specific word” does not exist in the word list D, and therefore the word “that” as an instructional adjective is determined as “specific word” in step S197. In step S203, the utterance dictionary 205D is used to determine the utterance content such as “That book” based on “that” which is the “specific word” determined in step S197. Alternatively, the content of the utterance may be “that white book” using the instruction adjective and any word registered in the word list D created in step S187. However, in this example, the article (book) 24 cannot be uniquely specified by the word “white book”. In step S203, an utterance content such as “It's white (this)” may be used by using a pronoun, or an utterance content may be simply “that (this)”.

  On the other hand, if “YES” is determined in the step S195, the speech recognition rate DB 205F set in the memory 204 is referred to in the step S199, the respective speech recognition rates of the words registered in the word list D are acquired, and the word The words registered in the list D are sorted in descending order of speech recognition rate. In step S201, the word having the highest speech recognition rate registered at the top of the word list D is determined as the “specific word”. In step S203, the utterance content is determined based on the “specific word” determined in step S201 using the utterance dictionary 205D. In this way, the content that the robot 12 speaks to confirm the article (book) 24 instructed by the person 16 estimated by the system 10 with the person 16 is determined.

  As described above, when a “specific word” having a high voice recognition rate is used as the content spoken by the robot 12 when the human body 16 confirms the article (book) 24 that the system 10 has estimated that the human 16 has instructed, If this "specific word" is used when 16 imitates this and indicates the article (book) 24 next time, the system 10 can easily identify the article (book) 24 by voice recognition. It becomes like this.

  Returning to FIG. 16, when the utterance content is determined in step S49, the CPU 200 next determines the position information of “selected article (book) 24” acquired in step S47 and step S203 (FIG. 22) in step S51. And the utterance content information generated in step (1) are transmitted to the robot 12.

  When the robot 12 receives the position information of the “selected article (book) 24” and the utterance content information, the CPU 80 determines YES in step S11 of FIG. Then, in the next step S13, the CPU 80 refers to the utterance / gesture dictionary 85A set in the memory 84, and selects the “selected article (book) 24” based on the position information of the “selected article (book) 24”. Pointing and speaking based on the utterance content information. At this time, the CPU 80 uses the received position information of the “selected article (book) 24” by converting the coordinates from the world coordinate system to the robot coordinate system.

  When the “selected article (book) 24” is the book 24 with the title (name) “Fireworks Master”, in this step S13, the book placed at the second name from the right in FIG. And utterances such as “It ’s a fireworks master ’s book”.

  In response to such pointing and utterance of the robot 12, the human 16 makes a response such as “Yes” or “No”. Then, in the robot 12, the CPU 80 takes in the voice of the human 16 input through the microphone 66, and determines that the human 16 has responded in step S15. In step S17, an instruction for determining whether the content is to be affirmed by recognizing the speech and the content to be affirmed or not is transmitted to the server 20 together with the captured speech information of the human 16.

  In server 20, CPU 200 determines YES in step S53 of FIG. 16 when it receives an instruction to determine whether it is affirmative or negative by voice recognition and information of voice to be voice-recognized. Next, in step S55, the CPU 200 performs voice recognition processing on the received voice information to determine whether the voice content indicates affirmation or denial, and in step S57. The determination result is transmitted to the robot 12.

  When the robot 12 receives the determination result whether the voice recognition is affirmative or negative, the CPU 80 determines YES in step S19. Further, in step S21, it is determined whether or not the result of the speech recognition in the server 20 is affirmative. If NO is determined in step S21, the CPU 80 determines whether or not the position information of the “selected article (book) 24” and the utterance content information are further received from the server 20 in step S11.

  On the other hand, if “YES” is determined in the step S <b> 21, the robot 12 moves in the direction of the corresponding article (book) 24, holds the corresponding article (book) 24, and carries it to the position of the person 16. That is, since the coordinates of the position where the article (book) 24 exists are already known, the CPU 80 of the robot 12 controls the wheel motor 36 to move the robot 12 to the position of the article (book) 24, and then By controlling the actuator 108 (FIG. 3), the hand 56R (or 56L) is opened and closed, the article (book) 24 is gripped by the hand 56R (or 56L: FIG. 2), and the wheel motor 36 is controlled again in this state. The robot 12 is moved to the position of the human 16. In this manner, in step S23, the “selected article (book) 24” selected by the server 20 in step S43 of FIG.

  On the other hand, in step S57 in FIG. 16, the server 20 transmits a determination result of whether the content of the voice of the human 16 by voice recognition is affirmative or negative to the robot 12, and then in step S61 in FIG. It is determined whether or not the result of the determination is affirmative. If NO is determined in step S61, the CPU 200 determines in step S43 in FIG. 16 the next candidate article (C) from the candidate article list C created from the voice recognition result, the line-of-sight reliability and the pointing direction reliability. Book) Select 24.

  At this time, if the ID of the other article (book) 24 is not registered in the candidate article list C, it is determined that no article has been selected in step S45 (NO in step S45). Then, the CPU 200 transmits a notification that there is no candidate article to the robot 12 in step S59.

  When the robot 12 receives a notification that there is no candidate article, the CPU 200 determines YES in step S25 of FIG. 15 and ends the process assuming that the article (book) 24 designated by the human 16 cannot be specified. .

  On the other hand, returning to FIG. 17, if YES is determined in the step S61, the CPU 200 next stores a memory (memory) in order to indicate that the article (book) 24 instructed by the human 16 can be specified in the step S63. The numerical value “1” is stored in the flag sc set in 204 to turn on the flag sc. The initial state of the flag sc is an off state (numerical value “0”).

  Next, in step S65, the CPU 200 determines whether or not the flag nw set in the memory 204 is on. As described above, this flag nw is a word that cannot be recognized as a result of speech recognition of the speech uttered by the human 16, and the word is a word that will specify the article designated by the human 16. In some cases it is set to the on state.

  If YES is determined in the step S65, the CPU 200 registers a word that cannot be recognized in the speech in the article dictionary 122 and updates the dictionary in a step S67. Specifically, in step S119 in FIG. 19, the phoneme symbol string of the word that could not be recognized by speech stored in the work area wa of the memory 204 is registered in the article dictionary 122. For example, human 16 speaks “bring Hanatak” and “Hanataku” could not be recognized by voice, but human 16 pointed to the book “Fireworks Takumi” based on gaze and pointing. Suppose that it turns out. In this case, since the phoneme symbol string “hanataku” of the word “hanataku” is stored in the work area wa, the ID of the article in the article dictionary 122 is “123... “0000046” is stored in the “attribute” item of the book 24 record “Fireworks Master” together with the ID of the person 16 who spoke the word “Hanataku”.

  Next, in step S69, the CPU 200 registers the word that could not be recognized as a new record in the speech recognition dictionary 126 and updates the dictionary. Specifically, if the example in step S67 is used as it is, the article (book) 24 ("selected article (book) 24" selected in step S43 in FIG. ”) Is added to the record that includes an item that stores the word“ Fireworks Master ”and an item that stores the phoneme symbol string“ hanataku ”stored in the work area wa of the memory 204. At this time, a value “1” indicating the name of the article (title (name) of the book 24) is stored in the “article name” item of the record.

  If the word “fireworks master” and the phoneme symbol string “hanataku” are registered in the speech recognition dictionary 122 in this way, the next time, for example, when the human 16 speaks, “Bring Hanataku”, “Hanataku” can be recognized by speech, and “artificial fireworks”, the name of the article (book) 24, can be obtained as a result of the speech recognition.

  In step S71, the personal correctness / error DB 205E set in the memory 204 is updated. That is, it is determined whether or not the flag sc set in the memory 204 is in the on state. If the flag sc is in the on state, the article (book) 24 instructed by the person 16 has been identified. Therefore, information indicating that the communication with the person 16 has been successful is stored in the personal correctness DB 205E. On the other hand, when the flag sc is in the off state, information indicating that the communication with the person 16 is unsuccessful is stored.

  If NO is determined in step S65, steps S67 and S69 are skipped.

  Thus, in this system 10, the position of the person 16 pointing to the article (book) 24 is detected, and the article (book) 24 existing in the vicinity of the person 16 is specified. Further, only a record for a word related to the article (book) 24 existing in the vicinity of the person 16 is extracted from the records included in the speech recognition dictionary 126 to create the speech recognition local dictionary 205B. Then, using this voice recognition local dictionary 205B, the voice uttered by the person 16 is recognized, and the article (book) 24 indicated by the person 16 is specified. That is, in this case, the words that are expected to be included in the speech of the human 16 are limited to the articles (books) 24 that exist in the space near the human 16, so that the speech recognition dictionary (speech used for speech recognition) is used. The words included in the recognition local dictionary 205B) are optimized. Therefore, the speech recognition processing is performed as compared with the case where the speech recognition processing is executed using the speech recognition dictionary 126 in which records for words related to all articles (books) 24 targeted by the system 10 are registered. The time can be shortened and the accuracy of voice recognition can be improved.

  In the above-described embodiment, the time required for the speech recognition process is shortened by creating a separate speech recognition local dictionary 205B from the speech recognition dictionary 126. Instead, the speech recognition local dictionary 205B is used instead. For example, among the records registered in the speech recognition dictionary 126, only the record for the word related to the article (book) 24 existing in the vicinity of the person 16 is flagged, Even if the voice recognition process is performed with reference to only the records with “”, the time required for the voice recognition process can be shortened and the accuracy of the voice recognition can be improved.

FIG. 1 is an illustrative view showing an outline of a communication robot system showing an embodiment of the present invention. FIG. 2 is an illustrative view showing the appearance of the robot shown in FIG. 1 from the front. FIG. 3 is an illustrative view showing an electrical configuration of the robot shown in FIG. FIG. 4 is an illustrative view showing an electrical configuration of the server shown in FIG. FIG. 5 is an illustrative view showing one example of an article dictionary used in the embodiment of FIG. FIG. 6 is an illustrative view showing one example of an article local dictionary used in the embodiment of FIG. FIG. 7 is an illustrative view showing one example of a list of words used in the embodiment of FIG. FIG. 8 is an illustrative view showing one example of a list of words used in the embodiment of FIG. FIG. 9 is an illustrative view showing one example of an article dictionary used in the embodiment of FIG. FIG. 10 is an illustrative view showing one example of a speech recognition dictionary used in the embodiment of FIG. FIG. 11 is an illustrative view showing one example of a speech recognition local dictionary used in the embodiment of FIG. FIG. 12 is an illustrative view showing one example of a speech recognition dictionary used in the embodiment of FIG. FIG. 13 is an illustrative view showing a human gaze and pointing direction. FIG. 14 is an illustrative view showing one example of a certainty factor table used in the embodiment of FIG. FIG. 15 is a flowchart showing the operation of the robot in the embodiment of FIG. FIG. 16 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 17 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 18 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 19 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 20 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 21 is a flowchart showing the operation of the server in the embodiment of FIG. FIG. 22 is a flowchart showing the operation of the server in the embodiment of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Communication robot system 12 ... Communication robot 14 ... Network 18 ... Wireless tag 20 ... Server 24 ... Goods (book)
80 ... CPU
120 ... Camera 124 ... Antenna 200 ... CPU
208... RFID tag reader

Claims (4)

A voice recognition system for recognizing human-generated voice and identifying an article designated by the human,
Position detecting means for detecting a human position,
Based on the position of the person detected by the position detection means, a nearby article specifying means for specifying an article in the vicinity of the person,
Dictionary construction means for constructing a first speech recognition dictionary related to the article identified by the neighboring article identification means, and articles designated by a human by performing speech recognition using the first speech recognition dictionary constructed by the dictionary construction means A voice recognition system comprising a specified article specifying means for specifying.   The speech recognition system according to claim 1, wherein the dictionary construction unit constructs the first speech recognition dictionary based on a second speech recognition dictionary for all articles targeted by the speech recognition system. A voice recognition system for recognizing human-generated voice and identifying an article designated by the human,
Position information storage means for storing position information indicating the position of the article for each article targeted by the voice recognition system;
Voice recognition information storage means for storing voice recognition information including a word related to the article and a phoneme symbol string of the word for each article targeted by the voice recognition system;
Word specifying means for specifying a word related to an article with reference to the speech recognition information stored in the speech recognition information storage means based on a phoneme symbol string obtained by performing speech recognition processing;
Article specifying means for specifying an article designated by the person based on the word specified by the word specifying means;
Based on position detection means for detecting the position of a person, and the position of the person detected by the position detection means and position information stored in the position information storage means, an article existing in the vicinity of the person is specified. Providing a nearby article specifying means,
The said word specific | specification means is a speech recognition system which specifies a word only with reference to the said speech recognition information containing the word relevant to the articles | goods which the said adjacent goods specific | specification means specified.   The word specifying means refers to the sub-voice recognition information storage means that extracts and stores only the voice recognition information including the word related to the article specified by the neighboring article specification means from the voice recognition information storage means. The speech recognition system according to claim 3, wherein the speech recognition information stored in the speech recognition information storage means is referred to.

Images