JP2022103472A - Information processor, information processing method, and program - Google Patents

Abstract

To provide an information processor capable of improving the usability of a voice user interface.SOLUTION: An information processor in an embodiment includes: an extraction unit that extracts unique expressions from utterances of each of multiple target users; a determination unit that determines whether or not the target user has visited a specific point where the number of visits has increased sharply based on the behavior history of the target user for each of the target users; a first generation unit that generates a feature value as a combination of the unique expression extracted by the extraction unit and the determination result by the determination unit for each of the target users; an analysis unit that performs clustering of the multiple target users for which the feature value is generated by the first generation unit; and a second generation unit that generates a dictionary of at least one of a speech recognition and a natural language understanding for each cluster generated by the clustering.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing apparatus, an information processing method, and a program.

A voice user interface using voice recognition technology is known (see, for example, Patent Document 1-4).

Japanese Unexamined Patent Publication No. 2017-58674 Japanese Patent Publication No. 2019-536172 Japanese Unexamined Patent Publication No. 2014-106523 Japanese Patent Publication No. 2004-502985

The content of the user's utterance to the voice user interface and the unique expressions included in the utterance change according to the user's community and fashion. Although the user who is the subject of the utterance and the information source of the utterance are diversified, the conventional technique cannot cope with the diversification, and the usability of the voice user interface may not be sufficient.

Aspects of the present invention have been made in consideration of such circumstances, and one of the objects of the present invention is to provide an information processing device, an information processing method, and a program capable of improving the usability of a voice user interface. do.

The information processing apparatus, information processing method, and program according to the present invention have the following configurations.
(1) The first aspect of the present invention is an extraction unit that extracts a unique expression from each speech of a plurality of target users, and a specific point where the number of visits has rapidly increased based on the behavior history of the target user. A feature amount that combines a determination unit that determines whether or not a user has visited for each target user, the unique expression extracted by the extraction unit, and a determination result by the determination unit is determined for each target user. Speech recognition and natural language for each of the first generation unit generated in, the analysis unit that clusters the plurality of target users whose feature quantities are generated by the first generation unit, and the clusters generated by the clustering. It is an information processing apparatus including a second generation unit that generates a dictionary for at least one of understanding.

(2) The second aspect of the present invention is, in the first aspect, a predetermined period of the number of times the specific point is visited by another user is equal to or greater than the threshold value, or the number of times the specific point is visited by the other user. It is an information processing device whose rate of increase per hit is equal to or higher than the threshold value.

(3) In the third aspect of the present invention, in the first aspect or the second aspect, the co-occurrence expression of the named entity extracted from the utterance of the target user belonging to the cluster is collected for each cluster. The second generation unit is an information processing device that generates the dictionary including the co-occurrence expression collected by the collection unit for each cluster.

(4) A fourth aspect of the present invention is, in any one of the first to third aspects, to the target user belonging to the specific cluster among the plurality of the clusters, to the target user among the plurality of dictionaries. It is an information processing apparatus further provided with a providing unit that provides usage guidance information of the dictionary corresponding to a specific cluster.

(5) In the fifth aspect of the present invention, in the fourth aspect, the second generation unit generates a new dictionary in which the dictionary generated for each cluster and the existing dictionary are combined, and the providing unit generates a new dictionary. An information processing device that provides the target user belonging to the specific cluster with usage guidance information of the new dictionary in which the dictionary corresponding to the specific cluster and the existing dictionary are combined.

(6) In the sixth aspect of the present invention, in any one of the first to fifth aspects, a verification unit that verifies the dictionary based on the utterance of the target user in a predetermined user group is provided. It is an information processing device to be further provided.

(7) In the seventh aspect of the present invention, in any one of the first to sixth aspects, the first generation unit has the first feature amount based on the named entity and the determination result by the determination unit. This is an information processing apparatus that generates a combination with a second feature amount based on the above as the feature amount.

(8) In the eighth aspect of the present invention, in the seventh aspect, the second feature amount includes one or both of the presence or absence of a visit to the specific point and the number of visits to the specific point. It is an information processing device including a feature amount representing.

(9) A ninth aspect of the present invention is an information processing apparatus in which, in any one of the first to eighth aspects, the named entity includes the wording of a place name or a mark.

(10) In the tenth aspect of the present invention, the computer extracts a unique expression from each speech of a plurality of target users, and based on the behavior history of the target user, the target is a specific point where the number of visits has rapidly increased. Whether or not a user has visited is determined for each target user, and a feature amount that combines the extracted unique expression and the determination result is generated for each target user, and the feature amount is generated. It is an information processing method that performs clustering of a plurality of target users and generates a dictionary for at least one of speech recognition and natural language understanding for each cluster generated by the clustering.

(11) The eleventh aspect of the present invention is to extract a unique expression from each speech of a plurality of target users on a computer, and to determine a specific point where the number of visits has rapidly increased based on the behavior history of the target user. It is determined for each target user whether or not the target user has visited, and a feature amount that combines the extracted unique expression and the determination result is generated for each target user. It is a program for executing the clustering of the plurality of target users who generated the above, and the generation of a dictionary for at least one of speech recognition and natural language understanding for each cluster generated by the clustering.

According to the above aspect, the usability of the voice user interface can be improved.

It is a block diagram of the information provision system 1 of an embodiment. It is a figure for demonstrating the content of the user authentication information 132. It is a figure for demonstrating the content of personal utterance history information 134A. It is a figure for demonstrating the content of group speech history information 134B. It is a figure for demonstrating the content of personal action history information 136A. It is a figure for demonstrating the content of group action history information 136B. It is a block diagram of the communication terminal 300 of an embodiment. It is a figure which shows an example of the schematic structure of the vehicle M equipped with the agent device 500. It is a flowchart which shows the flow of a series of processing by the information providing apparatus 100 of an embodiment. It is a flowchart which shows the flow of a series of processing by the information providing apparatus 100 of an embodiment. It is a figure for explaining the point where the number of visits increased sharply and the point where the number of visits did not increase rapidly. It is a figure which shows an example of the clustering result of a user vector. It is a figure for demonstrating the generation method of an information processing dictionary. It is a figure which shows typically the scene where the usage guidance information of an information processing dictionary is provided.

Hereinafter, embodiments of the information processing apparatus, information processing method, and program of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of the information providing system 1 of the embodiment. The information providing system 1 includes, for example, an information providing device 100, a communication terminal 300 used by the user U1 of the information providing system 1, and a vehicle M used by the user U2 of the information providing system 1. These components are communicable with each other via the network NW. The network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a telephone line, a public line, a dedicated line, a provider device, a radio base station, and the like. The information providing system 1 may include one or more of the communication terminal 300 and the vehicle M. The vehicle M includes, for example, an agent device 500. The information providing device 100 is an example of an “information processing device”.

The information providing device 100 receives an inquiry or request of the user U1 from the communication terminal 300, performs processing according to the received inquiry or request, and transmits the processing result to the communication terminal 300. Further, the information providing device 100 receives an inquiry or request of the user U2 from the agent device 500 mounted on the vehicle M, performs processing according to the received inquiry or request, and transmits the processing result to the agent device 500. The information providing device 100 may function as, for example, a cloud server that communicates with the communication terminal 300 and the agent device 500 via the network NW and transmits / receives various data.

The communication terminal 300 is, for example, a portable terminal such as a smartphone or a tablet terminal. The communication terminal 300 receives information such as inquiries and requests from the user U1. The communication terminal 300 transmits the information received from the user U1 to the information providing device 100, and outputs the information obtained as a reply to the transmitted information. That is, the communication terminal 300 functions as a voice user interface.

The vehicle M on which the agent device 500 is mounted is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the electric power generated by the generator connected to the internal combustion engine or the electric power generated by the secondary battery or the fuel cell. Further, the vehicle M may be an autonomous driving vehicle. Autonomous driving is, for example, the automatic control of one or both of the steering and speed of a vehicle. The vehicle operation control described above may include various operation controls such as ACC (Adaptive Cruise Control), ALC (Auto Lane Changing), and LKAS (Lane Keeping Assistance System). The driving of the self-driving vehicle may be controlled by the manual driving of the occupant (driver).

The agent device 500 interacts with the occupant of the vehicle M (for example, the user U2), and provides information for inquiries, requests, and the like from the occupant. The agent device 500 receives, for example, information such as an inquiry or request from the user U2, transmits the received information to the information providing device 100, and outputs the information obtained as a reply to the transmitted information. That is, the agent device 500 functions as a voice user interface, similarly to the communication terminal 300.

[Information provider]
Hereinafter, the configuration of the information providing device 100 will be described. The information providing device 100 is, for example, a communication unit 102, an authentication unit 104, an acquisition unit 106, a voice recognition unit 108, a natural language processing unit 110, a determination unit 112, a user vector generation unit 114, and an analysis unit. It includes 116, a collection unit 118, a dictionary generation unit 120, a verification unit 122, a provision unit 124, and a storage unit 130. The combination of the voice recognition unit 108 and the natural language processing unit 110 is an example of the "extraction unit". The user vector generation unit 114 is an example of the “first generation unit”. The dictionary generation unit 120 is an example of the “second generation unit”.

The authentication unit 104, the acquisition unit 106, the voice recognition unit 108, the natural language processing unit 110, the determination unit 112, the user vector generation unit 114, the analysis unit 116, the collection unit 118, and the dictionary generation unit 120. The verification unit 122 and the provision unit 124 are each realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium), and may be installed in the storage device of the information providing device 100 by mounting the storage medium on a drive device or the like.

The storage unit 130 is realized by the above-mentioned various storage devices, EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), RAM (Random Access Memory), or the like. In addition to the program referenced by the processor, the storage unit 130 stores, for example, user authentication information 132, speech history information 134, action history information 136, and the like.

The user authentication information 132 includes, for example, information for identifying a user who uses the information providing device 100, information used at the time of authentication by the authentication unit 104, and the like. The user authentication information 132 is, for example, a user ID, a password, an address, a name, an age, a gender, a hobby, a special skill, orientation information, and the like. The directivity information is information indicating the directivity of the user, for example, information indicating the user's way of thinking, information indicating preference (information on preference), information indicating items that the user attaches importance to, and the like.

The utterance history information 134 is history information of words (that is, utterances) spoken by the user to the communication terminal 300 or the agent device 500 that functions as a voice user interface. The utterance history information 134 includes individual utterance history information 134A, which is the utterance history of one user, and group utterance history information 134B, which is the utterance history of a plurality of users. For example, when only one user is in the vehicle M on which the agent device 500 is mounted (when only one user's utterance is picked up by the agent device 500), the utterance history of that user is an individual utterance. It is recorded as history information 134A. On the other hand, when a plurality of users are in a group on the vehicle M (when the utterances of the plurality of users are picked up by the agent device 500), the utterance histories of the plurality of users are recorded as the group utterance history information 134B. Will be done.

The behavior history information 136 is history information of the user's behavior such as a visit to a tourist spot or an Internet search. The action history information 136 includes individual action history information 136A, which is the action history of one user, and group action history information 136B, which is the action history of a plurality of users. For example, when only one user is on the vehicle M on which the agent device 500 is mounted, the transition history (movement history) of the position of the vehicle M on which the user is placed is recorded as the personal utterance history information 134A. .. On the other hand, when a plurality of users are on the vehicle M in a group, the transition history (movement history) of the position of the vehicle M on which the plurality of users are placed is recorded as the group utterance history information 134B. Further, when one user carries and moves the communication terminal 300, the transition history (movement history) of the position of the communication terminal 300 is recorded as the personal utterance history information 134A. On the other hand, when each of the plurality of users carries and moves the communication terminal 300, the transition history (movement history) of the positions of the plurality of communication terminals 300 is recorded as the group utterance history information 134B.

The communication unit 102 is an interface for communicating with the communication terminal 300, the agent device 500, and other external devices via the network NW. For example, the communication unit 102 includes a NIC (Network Interface Card), an antenna for wireless communication, and the like.

The authentication unit 104 registers information about a user (for example, users U1 and U2) who uses the information providing system 1 in the storage unit 130 as user authentication information 132. For example, when the authentication unit 104 receives a user registration request from the communication terminal 300 or the agent device 500, the authentication unit 104 accepts a GUI (Graphical User Interface) for inputting various information included in the user authentication information 132. Display on the device. When the user inputs various information into the GUI, the authentication unit 104 acquires information about the user from the device. Then, the authentication unit 104 registers the information about the user acquired from the communication terminal 300 or the agent device 500 in the storage unit 130 as the user authentication information 132.

FIG. 2 is a diagram for explaining the contents of the user authentication information 132. The user authentication information 132 is, for example, associated with the user's authentication information by information such as the user's address, name, age, gender, contact information, and orientation information. The authentication information includes, for example, a user ID, a password, and the like, which are identification information for identifying the user. Further, the authentication information may include biometric authentication information such as fingerprint information and iris information. The contact information may be, for example, address information for communicating with the voice user interface (communication terminal 300 or agent device 500) used by the user, the user's telephone number, e-mail address, terminal identification information, or the like. May be. The information providing device 100 communicates with each mobile communication device based on the contact information and provides various information.

The authentication unit 104 authenticates the user of the service of the information providing system 1 based on the user authentication information 132 registered in advance. For example, the authentication unit 104 authenticates the user at the timing when the service use request is received from the communication terminal 300 or the agent device 500. Specifically, when the authentication unit 104 receives the usage request, the authentication unit 104 displays the GUI for inputting the authentication information such as the user ID and the password on the requested terminal device, and inputs the GUI on the GUI. The input authentication information and the authentication information of the user authentication information 132 are compared. The authentication unit 104 determines whether or not the authentication information matching the input authentication information is stored in the user authentication information 132, and if the authentication information matching the input authentication information is stored, the service is used. Allow. On the other hand, when the authentication information matching the input authentication information is not stored, the authentication unit 104 performs a process for prohibiting the use of the service or for making a new registration.

The acquisition unit 106 acquires utterances of one or a plurality of users from the communication terminal 300 or the agent device 500 via the communication unit 102 (via the network NW), and stores the utterances as the utterance history information 134 in the storage unit 130. Store in. The user's utterance may be voice data (also referred to as acoustic data or acoustic stream), or may be text data recognized from the voice data. Further, the acquisition unit 106 acquires the action history of one or a plurality of users from the communication terminal 300 or the agent device 500 via the communication unit 102 (via the network NW), and uses it as the action history information 136. It is stored in the storage unit 130.

FIG. 3 is a diagram for explaining the contents of the personal utterance history information 134A. In the personal utterance history information 134A, for example, the place where the utterance was made, the content utterance of the utterance, and the provided information are associated with the date and time when the user utters the utterance. The utterance content may be a voice spoken by the user, or may be a text obtained by voice recognition by the voice recognition unit 108, which will be described later. The provided information is information provided by the providing unit 124 as a response to the user's utterance. The provided information includes, for example, voice information for dialogue and display information such as images and actions.

FIG. 4 is a diagram for explaining the contents of the group utterance history information 134B. The group utterance history information 134B is, for example, associated with the date and time when the user utters, in addition to information such as the place where the utterance was made, the content of the utterance, and the provided information, and further, the group member information is associated with the information. Is. The group member information is, for example, information about other users who got on the same vehicle M, other users who accompanied the same place, and other users who can be regarded as being at the same place at the same time from the position information (for example, user ID, etc.). ).

FIG. 5 is a diagram for explaining the contents of the personal behavior history information 136A. The personal action history information 136A has, for example, a user ID and a date and time associated with the action history. The action history includes, for example, a visit destination by the user and a means of transportation thereof. As described above, the action history may include the action history on the Internet.

FIG. 6 is a diagram for explaining the contents of the group action history information 136B. In the group action history information 136B, for example, the action history and the group member information are associated with each other with respect to the user ID and the date and time.

The voice recognition unit 108 performs voice recognition (process of converting the voice into text) for recognizing the voice spoken by the user. For example, the voice recognition unit 108 performs voice recognition on the voice data representing the user's utterance acquired by the acquisition unit 106, and generates text data obtained by converting the voice data into text. The text data includes a character string in which the content of the utterance is described as a character.

For example, the voice recognition unit 108 may convert voice data into text using an acoustic model and a dictionary for automatic voice recognition (hereinafter referred to as an ASR dictionary). The acoustic model is a model that is pre-trained or adjusted so as to separate the input voice according to the frequency and convert each separated voice into a phoneme (spectrogram). For example, in a neural network or a hidden Markov model. be. The ASR dictionary is a database in which character strings are associated with a combination of a plurality of phonemes, and the positions where the character strings are separated are defined by a syntax. The ASR dictionary is a so-called pattern matching dictionary. For example, the voice recognition unit 108 inputs voice data to an acoustic model, searches the ASR dictionary for a set of phonemes output by the acoustic model, and acquires a character string corresponding to the set of phonemes. The voice recognition unit 108 generates the combination of the character strings thus obtained as text data. Instead of using the ASR dictionary, the voice recognition unit 108 may generate text data from the output result of the acoustic model by using, for example, a language model implemented by an n-gram model or the like.

The natural language processing unit 110 performs natural language understanding to understand the structure and meaning of text. For example, the natural language processing unit 110 interprets the meaning of the text data generated by the speech recognition unit 108 while referring to a dictionary (hereinafter, NLU dictionary) prepared in advance for meaning interpretation. The NLU dictionary is a database in which abstract semantic information is associated with text data. For example, the NLU dictionary defines that the words "I" and "colleagues" are highly related to each other, and the words "hamburger" and "eat" are highly related to each other. As a result, for example, the sentence "I ate a hamburger with a colleague" performed the act of "eating" to two objects, "colleague" and "hamburger", by a single subject "I". It is not interpreted in the sense that it is interpreted in the sense that the two subjects "I" and "colleague" performed the act of "eating" on a single object called "hamburger". Become. The NLU dictionary may include synonyms, synonyms, and the like. Speech recognition and natural language understanding do not necessarily have to be clearly separated into stages, and may be performed by interacting with each other, such as modifying the result of speech recognition based on the result of natural language understanding.

Further, the natural language processing unit 110 extracts the named entity from the text data generated by the voice recognition unit 108. For example, the natural language processing unit 110 may use TF (Term Frequency) -IDF (Inverse Document Frequency) or the like to extract a named entity.

The proper expression may be, for example, one word (word) such as a noun, or one phrase (phrase) in which a noun and a noun are connected by another part of speech (for example, an auxiliary verb). It may be a single sentence containing various parts of speech such as nouns, verbs, auxiliary verbs, and auxiliary verbs.

For example, named entity includes a place name, the wording of the place name, a mark, and the wording of the mark. The mark includes, for example, a company name, a brand name, a store name, and the like. For example, the official name of a company is "Honda Motor Co., Ltd.", and users in the area where the head office of the company "Honda Motor Co., Ltd." is located are familiar with the company and "Honda Motor Co., Ltd." "And so on. In this case, the named entity "Honda Motor Co., Ltd." is treated as the phrase "Honda Motor Co., Ltd.". Also, when the official name of a restaurant is "ABCDEF", users in one area abbreviate the restaurant as "ABC", while users in another area call it "ABC". It is assumed that the restaurant is abbreviated as "DEF". In this case, named entities that are called differently in different regions, such as "ABC" and "DEF", are treated as the phrase "ABCDEF". In addition, the relationship between named entity recognition and its wording may exist not only in regions but also in differences in age and generation, such as young people and adults, and also in differences in communities and factions. You can do it. Such a relationship may also exist, for example, between a widely known major name and a minor name that is used interchangeably with the major name. In this way, the natural language processing unit 110 extracts expressions that are different from each other in terms of character strings but have the same meaning, as unique expressions.

The determination unit 112 determines whether or not each user has visited a point where the number of visits has increased rapidly, based on the action history of each of the plurality of users included in the action history information 136. The "point where the number of visits has increased sharply" is, for example, a point where the number of visits by other users is equal to or higher than the threshold value, or a point where the rate of increase in the number of visits by other users per predetermined period is equal to or higher than the threshold value. ..

The user vector generation unit 114 generates a multidimensional feature amount for each user, which is a combination of the named entity extracted by the natural language processing unit 110 and the determination result by the determination unit 112. For example, the user vector generation unit 114 has a vector of named entity obtained by using TF-IDF or the like (hereinafter, referred to as “utterance vector”) and a vector based on the determination result by the determination unit 112 (hereinafter, “action vector””. ) And the combination is generated as one vector. Hereinafter, a vector that combines an utterance vector and an action vector will be referred to as a “user vector”. The utterance vector is an example of the "first feature amount", and the action vector is an example of the "second feature amount".

The analysis unit 116 clusters a plurality of users for which a user vector is generated, and classifies users having similar characteristics such as utterance contents and visited points into the same cluster. At this time, the analysis unit 116 may compress the dimension of the user vector. For dimensional compression, for example, principal component analysis, a topic model represented by LDA (Latent Dirichlet Allocation), a neural network such as Word2Vec, or the like may be used. Further, for dimension compression, LASSO (Least Absolute Shrinkage and Selection Operator), which is one of regularization regression, NMF (Nonnegative Matrix Factorization), and the like may be used.

The collecting unit 118 collects the co-occurrence expression of the named entity extracted from the utterance of the user belonging to the cluster for each cluster generated by the clustering. The co-occurrence expression is, for example, a word that easily appears together with a named entity extracted from a user's utterance on a website or the like. For example, the collecting unit 118 accesses a web server or the like via the communication unit 102, crawls the website provided by the web server, and collects the content. The collection unit 118 extracts co-occurrence expressions from the collected content by using association analysis or the like. In this way, the collecting unit 118 may function like a Web crawler.

The dictionary generation unit 120 generates a dictionary (hereinafter referred to as an information processing dictionary) for each information processing such as speech recognition and natural language understanding for each cluster generated by clustering. The information processing dictionary is either an ASR dictionary referred to in the above-mentioned speech recognition (speech text conversion) or an NLU dictionary referred to in natural language understanding (text meaning interpretation). Or both dictionaries. For example, when a language model is used instead of an ASR dictionary for speech recognition, the information processing dictionary includes only an NLU dictionary. The information processing dictionary includes co-occurrence expressions collected by the collecting unit 118.

The verification unit 122 verifies the accuracy of the information processing dictionary generated by the dictionary generation unit 120. The details of the verification method will be described later.

The providing unit 124 provides (transmits) various information to the communication terminal 300 or the agent device 500, which is a voice user interface, via the communication unit 102. For example, when the acquisition unit 106 acquires an inquiry or request as an utterance from the communication terminal 300 or the agent device 500, the provision unit 124 generates information that is a response to the inquiry or request. For example, when an utterance meaning "tell me the weather today" is acquired, the provider 124 will provide content corresponding to the unique expressions "today" and "weather" (images and videos showing the results of weather forecasts, etc.). Voice etc.) may be generated. Then, the providing unit 124 returns the generated information to the voice user interface inquired or requested via the communication unit 102.

Further, the providing unit 124 provides the communication terminal 300 or the agent device 500 with the usage guidance information of the information processing dictionary generated by the dictionary generation unit 120. The usage guidance information is, for example, recommended to the user to newly refer (use) the ASR dictionary during voice recognition, or to newly refer (use) the NLU dictionary when understanding natural language. This is information that recommends the user to set it.

[Communication terminal]
Next, the configuration of the communication terminal 300 will be described. FIG. 7 is a configuration diagram of the communication terminal 300 of the embodiment. The communication terminal 300 includes, for example, a terminal-side communication unit 310, an input unit 320, a display 330, a speaker 340, a microphone (hereinafter, microphone) 350, a position acquisition unit 355, a camera 360, and an application execution unit 370. And an output control unit 380 and a terminal side storage unit 390. The position acquisition unit 355, the application execution unit 370, and the output control unit 380 are realized by, for example, a hardware processor such as a CPU executing a program (software). Further, some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit unit; including circuitry), or realized by collaboration between software and hardware. May be done. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium (non-transient) such as a DVD or a CD-ROM. It is stored in a sex storage medium), and may be installed in the storage device of the communication terminal 300 by mounting the storage medium in a drive device, a card slot, or the like.

The terminal-side storage unit 390 may be realized by the above-mentioned various storage devices, EEPROM, ROM, RAM, or the like. The terminal-side storage unit 390 stores, for example, the above program, the information providing application 392, and various other information.

The terminal-side communication unit 310 uses, for example, the network NW to communicate with the information providing device 100, the agent device 500, and other external devices.

The input unit 320 accepts the input of the user U1 by operating various keys, buttons, or the like, for example. The display 330 is, for example, an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like. The input unit 320 may be integrally configured with the display 330 as a touch panel. The display 330 displays various information in the embodiment under the control of the output control unit 380. The speaker 340 outputs a predetermined voice under the control of the output control unit 380, for example. The microphone 350 receives the voice input of the user U1 under the control of the output control unit 380, for example.

The position acquisition unit 355 acquires the position information of the communication terminal 300. For example, the position acquisition unit 355 includes a GNSS (Global Navigation Satellite System) receiver represented by GPS (Global Positioning System) or the like. The position information may be, for example, two-dimensional map coordinates or latitude / longitude information. The position acquisition unit 355 may transmit the acquired position information to the information providing device 100 via the terminal-side communication unit 310.

The camera 360 is, for example, a digital camera using a solid-state image sensor (image sensor) such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). For example, when the communication terminal 300 is attached to the instrument panel of the vehicle M as a substitute for a navigation device or the like, the camera 360 of the communication terminal 300 automatically or in response to the operation of the user U1 is the vehicle of the vehicle M. The room may be imaged.

The application execution unit 370 executes the information providing application 392 stored in the terminal side storage unit 390. The information providing application 392 outputs an image provided by the information providing device 100 to the display 330, and outputs a voice corresponding to the information provided by the information providing device 100 from the speaker 340. It is an application program for controlling 380. Further, the application execution unit 370 transmits the information input by the input unit 320 to the information providing device 100 via the terminal-side communication unit 310. As the information providing application 392, for example, one downloaded from an external device via the network NW may be installed in the communication terminal 300.

The output control unit 380 displays an image on the display 330 and outputs sound to the speaker 340 under the control of the application execution unit 370. At that time, the output control unit 380 may control the content or mode of the image to be displayed on the display 330, or may control the content or mode of the sound to be output to the speaker 340.

[vehicle]
Next, a schematic configuration of the vehicle M on which the agent device 500 is mounted will be described. FIG. 8 is a diagram showing an example of a schematic configuration of a vehicle M on which the agent device 500 is mounted. The vehicle M shown in FIG. 8 includes an agent device 500, a microphone 610, a display / operation device 620, a speaker unit 630, a navigation device 640, an MPU (Map Positioning Unit) 650, a vehicle device 660, and an in-vehicle device. A communication device 670, an occupant recognition device 690, and an automatic operation control device 700 are mounted. In addition, a general-purpose communication device 680 such as a smartphone may be brought into the vehicle interior and used as a communication device. The general-purpose communication device 680 is, for example, a communication terminal 300. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like.

First, configurations other than the agent device 500 will be described. The microphone 610 collects the voice emitted in the passenger compartment. The display / operation device 620 is a device (or device group) capable of displaying an image and accepting an input operation. The display / operation device 620 is typically a touch panel. The display / operation device 620 may further include a HUD (Head Up Display) or a mechanical input device. The speaker unit 630 outputs, for example, a voice, an alarm sound, or the like inside or outside the vehicle. The display / operation device 620 may be shared by the agent device 500 and the navigation device 640.

The navigation device 640 includes a navigation HMI (Human machine Interface), a positioning device such as GPS, a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. A part or all of the microphone 610, the display / operation device 620, and the speaker unit 630 may be used as the navigation HMI. The navigation device 640 refers to the map information based on the position of the vehicle M specified by the positioning device, and is a route (navigation route) for moving from the position of the vehicle M to the destination input by the user from the map information. Is searched, and guidance information is output using the navigation HMI so that the vehicle M can travel along the route. The route search function may be provided in the information providing device 100 or the navigation server accessible via the network NW. In this case, the navigation device 640 acquires a route from the information providing device 100 or the navigation server and outputs guidance information. The agent device 500 may be constructed based on the navigation controller, and in that case, the navigation controller and the agent device 500 are integrally configured on the hardware.

The MPU 650, for example, divides the map route provided by the navigation device 640 into a plurality of blocks (for example, divides the route into 100 [m] units with respect to the vehicle traveling direction), and determines a recommended lane for each block. For example, the MPU 650 determines which lane from the left to drive. Further, the MPU 650 may determine the recommended lane by using the map information (high-precision map) having higher accuracy than the map information stored in the storage device of the navigation device 640. The high-precision map may be stored in the storage device of the MPU 650, for example, or may be stored in the storage device of the navigation device 640 or the vehicle side storage unit 560 of the agent device 500. The high-precision map may include information on the center of the lane or information on the boundary of the lane, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like.

The vehicle device 660 is, for example, a camera, a radar device, a LIDAR (Light Detection and Ranging), and an object recognition device. The camera is, for example, a digital camera using a solid-state image sensor such as a CCD or CMOS. The camera is attached to any part of the vehicle M. The radar device radiates radio waves such as millimeter waves around the vehicle M, and detects radio waves (reflected waves) reflected by the object to at least detect the position (distance and orientation) of the object. LIDAR irradiates the periphery of the vehicle M with light and measures the scattered light. LIDAR detects the distance to the target based on the time from light emission to light reception. The object recognition device performs sensor fusion processing on the detection result of a part or all of the camera, the radar device, and the LIDAR, and recognizes the position, type, speed, and the like of the object existing around the vehicle M. The object recognition device outputs the recognition result to the agent device 500 and the automatic operation control device 700.

Further, the vehicle equipment 660 includes, for example, a driving operator, a traveling driving force output device, a braking device, a steering device, and the like. Driving controls include, for example, accelerator pedals, brake pedals, shift levers, steering wheels, odd-shaped steers, joysticks and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation controller, and the detection result is the agent device 500, the automatic operation control device 700, or the traveling drive force output device, the brake device, and the steering. It is output to some or all of the devices. The traveling driving force output device outputs a traveling driving force (torque) for the vehicle M to travel to the drive wheels. The brake device includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the automatic operation control device 700 or the information input from the operation operator so that the brake torque corresponding to the braking operation is output to each wheel. The steering device includes, for example, a steering ECU and an electric motor. The electric motor, for example, exerts a force on the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the automatic driving control device 700 or the information input from the driving operator, and changes the direction of the steering wheel.

Further, the vehicle equipment 660 includes, for example, a door lock device, a door opening / closing device, a window, a window opening / closing device and a window opening / closing control device, a seat, a seat position control device, a room mirror and its angle position control device, and inside and outside the vehicle. Lighting devices and their control devices, wipers and defoggers and their respective control devices, direction indicator lights and their control devices, vehicle information devices such as air conditioners, and the like may be included.

The in-vehicle communication device 670 is a wireless communication device that can access the network NW using, for example, a cellular network or a Wi-Fi network.

The occupant recognition device 690 includes, for example, a seating sensor, a vehicle interior camera, an image recognition device, and the like. The seating sensor includes a pressure sensor provided at the bottom of the seat, a tension sensor attached to the seat belt, and the like. The vehicle interior camera is a CCD camera or a CMOS camera provided in the vehicle interior. The image recognition device analyzes the image of the vehicle interior camera, recognizes the presence / absence of the user for each seat, the face of the user, and the like, and recognizes the seating position of the user. Further, the occupant recognition device 690 may specify a user who is seated in a driver's seat, a passenger's seat, or the like included in the image by performing a matching process with a face image registered in advance.

The automatic operation control device 700 performs processing by, for example, a hardware processor such as a CPU executing a program (software). A part or all of the components of the automatic operation control device 700 may be realized by hardware (circuit unit; including circuitry) such as LSI, ASIC, FPGA, GPU, or cooperation between software and hardware. May be realized by. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the automatic operation control device 700, or may be detachable such as a DVD or a CD-ROM. It is stored in a storage medium, and may be installed in the HDD or flash memory of the automatic operation control device 700 by mounting the storage medium (non-transient storage medium) in the drive device.

The automatic driving control device 700 recognizes the position, speed, acceleration, and other states of objects in the vicinity of the vehicle M based on the information input via the object recognition device of the vehicle equipment 660. In principle, the automatic driving control device 700 travels in the recommended lane determined by the MPU 650, and the vehicle M automatically (without depending on the driver's operation) so as to be able to respond to the surrounding conditions of the vehicle M. ) Generate a target track to run in the future. The target trajectory contains, for example, a speed element. For example, the target track is expressed as an arrangement of points (track points) to be reached by the vehicle M in order.

The automatic driving control device 700 may set an event of automatic driving in generating a target trajectory. Autonomous driving events include constant speed driving events, low speed following driving events, lane change events, branching events, merging events, takeover events, automatic parking events, and the like. The automatic operation control device 700 generates a target trajectory according to the activated event. Further, the automatic driving control device 700 controls the traveling driving force output device, the braking device, and the steering device of the vehicle equipment 660 so that the vehicle M passes the generated target track at the scheduled time. For example, the automatic driving control device 700 controls a traveling driving force output device or a braking device based on a speed element associated with a target track (track point), or controls a steering device according to the degree of bending of the target track. do.

Next, the agent device 500 will be described. The agent device 500 is a device that interacts with the occupant of the vehicle M. For example, the agent device 500 transmits the utterance of the occupant to the information providing device 100, and receives the answer to the utterance from the information providing device 100. The agent device 500 presents the received answer to the occupant using voice or an image.

The agent device 500 includes, for example, a management unit 520, an agent function unit 540, and a vehicle-side storage unit 560. The management unit 520 includes, for example, an acoustic processing unit 522, a display control unit 524, and a voice control unit 526. In FIG. 8, the arrangement of these components is simply shown for the sake of explanation, and in reality, for example, a management unit 520 may intervene between the agent function unit 540 and the vehicle-mounted communication device 60. The arrangement can be modified arbitrarily.

Each component other than the vehicle-side storage unit 560 of the agent device 500 is realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit part; including circuitry), or realized by the cooperation of software and hardware. May be good. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device.

The vehicle-side storage unit 560 may be realized by the above-mentioned various storage devices, EEPROM, ROM, RAM, or the like. For example, a program and various other information are stored in the vehicle side storage unit 560.

The management unit 520 functions by executing a program such as an OS (Operating System) or middleware.

The sound processing unit 522 receives input sound from the occupant of the vehicle M (for example, user U2) so as to be in a state suitable for recognizing information related to an inquiry, a request, or the like. Perform sound processing. Specifically, the acoustic processing unit 522 may perform acoustic processing such as noise reduction.

The display control unit 524 generates an image regarding the response result to the inquiry or request from the occupant of the vehicle M to the output device such as the display / operation device 620 in response to the instruction from the agent function unit 540. The image related to the answer result is, for example, an image showing a list of stores and facilities showing the answer result to inquiries and requests, an image about each store and facility, an image showing the driving route to the destination, and other recommendation information. An image or the like showing the start or end of processing. Further, the display control unit 524 may generate an anthropomorphic character image (hereinafter referred to as an agent image) that communicates with the occupant in response to an instruction from the agent function unit 540. The agent image is, for example, an image of a mode of talking to an occupant. The agent image may include, for example, a facial image such that the facial expression and the facial orientation are recognized by the viewer (occupant) at least. The display control unit 524 outputs the generated image to the display / operation device 620.

The voice control unit 526 causes some or all of the speakers included in the speaker 630 to output voice in response to an instruction from the agent function unit 540. The voice includes, for example, a voice for the agent image to have a dialogue with the occupant, and a voice corresponding to the image output to the display / operation device 620 by the display control unit 524. Further, the voice control unit 526 may use a plurality of speakers 630 to control the localization of the sound image of the agent voice at a position corresponding to the display position of the agent image. The position corresponding to the display position of the agent image is, for example, a position where the occupant is expected to feel that the agent image is speaking the agent voice, and specifically, a position near the display position of the agent image (for example, 2). It is within ~ 3 [cm]). Further, the localization of the sound image means that, for example, the spatial position of the sound source felt by the occupant is determined by adjusting the loudness of the sound transmitted to the left and right ears of the user.

The agent function unit 540 causes an agent image or the like to appear in cooperation with the information providing device 100 based on various information acquired by the management unit 520, and includes a voice response according to the utterance of the occupant of the vehicle M. Provide services. For example, the agent function unit 540 activates the agent based on the start word included in the voice processed by the sound processing unit 522, or terminates the agent based on the end word. Further, the agent function unit 540 transmits the voice data processed by the sound processing unit 522 to the information providing device 100 via the in-vehicle communication device 670, and provides the information obtained from the information providing device 100 to the occupant. do. Further, the agent function unit 540 may have a function of cooperating with the general-purpose communication device 680 and communicating with the information providing device 100. In this case, the agent function unit 540 pairs with the general-purpose communication device 680 by, for example, Bluetooth (registered trademark), and connects the agent function unit 540 and the general-purpose communication device 680. Further, the agent function unit 540 may be connected to the general-purpose communication device 680 by wired communication using USB (Universal Serial Bus) or the like.

[Processing flow of information processing device]
Next, a flow of a series of processes by the information providing device 100 will be described with reference to a flowchart. 9 and 10 are flowcharts showing the flow of a series of processes by the information providing device 100 of the embodiment.

First, the acquisition unit 106 acquires the utterances and action histories of a plurality of users from the communication terminal 300 or the agent device 500 via the communication unit 102 (step S100). When the acquisition unit 106 acquires the user's utterance and action history, it stores them in the storage unit 130 as the utterance history information 134 and the action history information 136.

Next, the voice recognition unit 108 generates text data from the utterances of each of the plurality of users by voice recognition (step S102). If the utterance is already converted into text in the communication terminal 300 or the agent device 500, that is, if the user's utterance acquired by the acquisition unit 106 is text data, the process of S102 may be omitted.

Next, the natural language processing unit 110 selects text data derived from the utterance of one target user from the text data of the utterances of each user generated by the voice recognition unit 108, and is unique from the selected text data. The expression is extracted (step S104). That is, the natural language processing unit 110 selects a target user from an unspecified number of users, and extracts a named entity from the utterance of the target user.

Next, the determination unit 112 determines whether or not there is a point where the number of visits has increased sharply among the points visited by the plurality of users based on the action history of the plurality of users (step S106).

FIG. 11 is a diagram for explaining a point where the number of visits has increased sharply and a point where the number of visits has not increased. In the illustrated example, there are three candidate sites X1 to X3 such as tourist spots on the map. For example, in the candidate site X1, the number of visits is rapidly increasing, and the number of visits by the user is equal to or greater than the threshold value, or the rate of increase in the number of visits by the user per predetermined period is equal to or greater than the threshold value. On the other hand, at the candidate sites X2 and X3, the number of visits does not change, the number of visits by the user is less than the threshold value, or the rate of increase in the number of visits by the user per predetermined period is less than the threshold value. In such a case, the determination unit 112 determines that the candidate site X1 is a point where the number of visits has increased sharply, and the candidate sites X2 and X3 are points where the number of visits has not increased sharply.

Further, the determination unit 112 visits a point where the number of visits by the user is equal to or greater than the threshold value or the rate of increase in the number of visits by the user is equal to or greater than the threshold value, even if the location is not a place where people tend to gather, such as a tourist spot. May be determined as a point where the number has increased sharply. For example, the determination unit 112 refers to the position information of an unspecified number of users, and a large number of users are gathered at a certain point, and the number of visits by the user at that point is equal to or greater than the threshold value, or the rate of increase thereof is equal to or greater than the threshold value. In some cases, the point may be determined as a point where the number of visits has increased sharply. That is, if there is a point that inspires a large number of users for some reason, that point is determined as a point where the number of visits has increased sharply.

Returning to the description of the flowcharts of FIGS. 9 and 10. Next, the determination unit 112 determines whether or not the target user has actually visited a point where the number of visits has increased rapidly (step S108).

For example, the determination unit 112 compares the position coordinates of the target user with the position coordinates of other users who have visited the point where the number of visits has increased rapidly on the map, and if the position coordinates of the users are the same, the target user. May determine that he has visited a point where the number of visits has increased sharply. In addition, communication services such as free Wi-Fi are provided in facilities (for example, commercial buildings and parking lots) that are determined to be points where the number of visits has increased sharply, and the target users and other users can use the communication services together. Suppose that you used. In this case, the location information of the target user and the other users includes the location information of the Wi-Fi access point. Therefore, when the position information of the common access point is included in the position information of the target user and other users, the determination unit 112 may determine that the target user has visited the point where the number of visits has increased rapidly. ..

When the determination unit 112 determines that the target user has visited a point where the number of visits has increased rapidly, the user vector generation unit 114 generates an action vector of the target user indicating "visited" (step S110).

On the other hand, the user vector generation unit 114 "does not visit" when the determination unit 112 determines that there is no point where the number of visits has increased rapidly or the target user has not visited the point where the number of visits has increased rapidly. The action vector of the target user indicating that is generated (step S112).

For example, the user vector generation unit 114 may generate a one-dimensional vector (scalar) in which "visited" is set to "1" and "not visited" is set to "0" as an action vector. .. Further, when the target user repeatedly visits a point where the number of visits has rapidly increased, the user vector generation unit 114 may generate an action vector having the number of visits N as an element.

Next, the user vector generation unit 114 generates a user vector of the target user by combining the utterance vector in which the named entity extracted from the utterance of the target user is vectorized and the action vector of the target user (step S114). ). For example, if the utterance vector is 10-dimensional and the action vector is 1-dimensional, the user vector is an 11-dimensional vector. The user vector may be associated with the user ID of the target user or the like.

Next, the natural language processing unit 110 determines whether or not a user vector has been generated for all the users whose utterances and action histories have been acquired (step S116). When the user vector is not generated for all users, the natural language processing unit 110 returns the processing to S104, reselects another user different from the user selected as the target user last time, and reselects the new target user. Extract unique expressions from the utterances of the target user. After that, the processes of S106 to S114 are performed for the new target user, and the user vector of the new target user is generated. In this way, the user vector is repeatedly generated.

On the other hand, when the user vector is generated for all users, the analysis unit 116 compresses each dimension of the plurality of user vectors (step S118). For example, when the user vector is 11 dimensions, the analysis unit 116 compresses it to 10 dimensions or less (reduces the number of elements of the vector).

Next, the analysis unit 116 performs clustering of a plurality of user vectors with compressed dimensions, and classifies users having similar characteristics such as utterance contents and visiting points into the same cluster (step S120).

Next, the collecting unit 118 collects the co-occurrence expression of the named entity extracted from the utterance of the user belonging to the cluster for each cluster generated by the clustering (step S122).

FIG. 12 is a diagram showing an example of the clustering result of the user vector. In the illustrated example, the plurality of user vectors are classified into three clusters A, B, and C. In this case, the collecting unit 118 collects the co-occurrence expression for the named entity of the user (hereinafter referred to as the user group A) to which the user vector belongs to the cluster A from the website or the like. Any one of clusters A, B, and C is an example of a "specific cluster".

Similarly, the collecting unit 118 has a co-occurrence expression for the unique expression of the user to which the user vector belongs to the cluster B (hereinafter referred to as the user group B) and the user having the user vector to the cluster C (hereinafter referred to as the user group C). Collect co-occurrence expressions for unique expressions from websites.

For example, in the user group A, it is assumed that "ABC", which is the phrase of the restaurant "ABCDEF", is frequently spoken. In this case, the collecting unit 118 collects expressions that are likely to appear together with the expression "ABC" on a web page or the like as co-occurrence expressions of the user group A. On the other hand, in the user group B, it is assumed that "DEF", which is the phrase of the restaurant "ABCDEF", is frequently spoken. In this case, the collecting unit 118 collects expressions that are likely to appear together with the expression "DEF" on a web page or the like as co-occurrence expressions of the user group B.

Returning to the description of the flowcharts of FIGS. 9 and 10. Next, the dictionary generation unit 120 generates an information processing dictionary including an ASR dictionary for speech recognition and an NLU dictionary for natural language understanding for each cluster generated by clustering (step S124).

FIG. 13 is a diagram for explaining a method of generating an information processing dictionary. As shown in the figure, the dictionary generation unit 120 may generate the information processing dictionary DICT_A in which the named entities and co-occurrence expressions of the user group A are associated with each other for the cluster A. Similarly, the dictionary generation unit 120 generates an information processing dictionary DICT_B in which the eigenexpressions and co-occurrence expressions of the user group B are associated with each other for the cluster B, and the eigenexpressions and co-occurrence of the user group C for the cluster C. The information processing dictionary DICT_C in which the expressions are associated with each other may be generated. In this way, the dictionary generation unit 120 generates an information processing dictionary for each cluster.

Returning to the description of the flowcharts of FIGS. 9 and 10. Next, the dictionary generation unit 120 determines whether or not the existing dictionary exists in each storage device of the communication terminal 300 or the agent device 500 (step S126). The existing dictionary is, for example, various dictionaries for realizing a character prediction conversion function and an input prediction function (suggest function) used in mobile phones, personal computers, and the like.

When it is determined that the existing dictionary exists, the dictionary generation unit 120 generates a new information processing dictionary by combining the information processing dictionary and the existing dictionary (step S128). The new information processing dictionary includes an existing dictionary in addition to an ASR dictionary for speech recognition and / or an NLU dictionary for natural language understanding. The new information processing dictionary is an example of a "new dictionary".

Next, the verification unit 122 verifies the accuracy of the information processing dictionary (including the new information processing dictionary) generated by the dictionary generation unit 120 (step S130). For example, the verification unit 122 verifies the accuracy of the information processing dictionary of the cluster A based on the utterance of the user group A. More specifically, in the verification unit 122, when the coverage (coverage) for the utterance frequency and the coverage for the user in the user group A (users to which the user vector belongs to the cluster A) are equal to or higher than a preset threshold value. , It is determined that the accuracy of the information processing dictionary of cluster A is equal to or higher than the threshold value. Similarly, the verification unit 122 verifies the accuracy of the information processing dictionary of the cluster B based on the utterance of the user group B, and verifies the accuracy of the information processing dictionary of the cluster C based on the utterance of the user group C.

Next, when the accuracy of the information processing dictionary is equal to or higher than the threshold value, the providing unit 124 provides the usage guidance information of the information processing dictionary to the communication terminal 300 or the agent device 500 (step S132). This ends the processing of this flowchart.

FIG. 14 is a diagram schematically showing a scene in which usage guidance information of an information processing dictionary is provided. U3 in the figure is a user whose user vector is close to the cluster A in the clustering feature space. That is, the user U3 is a user whose characteristics such as utterance content and action history are similar to those of the user group A. It is recommended that such a user U3 set, for example, a new information processing dictionary DICT_NEW which is a combination of the information processing dictionary DICT_A including the unique expression and the co-occurrence expression of the user group A and the existing dictionary DICT_X on the voice user interface. Will be done. For example, it is assumed that the user U3 sets the use of the recommended new information processing dictionary DICT_NEW on the voice user interface. In this case, the voice user interface transmits information to the effect that the use of the new information processing dictionary DICT_NEW is permitted to the information providing device 100. Upon receiving this permission information, the information providing device 100 uses the new information processing dictionary DICT_NEW permitted by the user U3 to perform voice recognition for the utterance of the user U3 and interpret the meaning of the recognized voice. do. As a result, the user U3 can interpret the meaning of the wording of the named entity that is used on a daily basis, and can provide an appropriate answer to the inquiry or request of the wording. As a result, the user can use the voice user interface in a familiar manner.

According to the embodiment described above, the information providing device 100 converts the utterances of a plurality of users into text and extracts a unique expression from the text data. The information providing device 100 determines whether or not each user has visited a point where the number of visits has increased rapidly, based on the behavior history of each of the plurality of users. The information providing device 100 includes an utterance vector obtained by vectorizing a unique expression extracted from each user's text data, and an action vector vectorized by the presence / absence of each user's visit to a point where the number of visits has increased and the result of the number of visits. To generate a user vector for each user. The information providing device 100 clusters a plurality of users for which a user vector is generated, and classifies users having similar characteristics such as utterance contents and visiting points into the same cluster. The information providing device 100 generates an information processing dictionary including an ASR dictionary for speech recognition and an NLU dictionary for natural language understanding for each cluster generated by clustering. Then, the information providing device 100 provides, for example, the user of the cluster A among the plurality of clusters with the usage guidance information of the information processing dictionary corresponding to the cluster A.

This makes it possible to respond to the variety of utterances that can change depending on the region, age, fashion, and the like. As a result, the usability of the voice user interface is improved, and the voice user interface can be used, for example, by using a phrase familiar to the user. In addition, even if a word registered in the dictionary becomes obsolete or a new word not registered in the dictionary becomes popular, the dictionary can be automatically updated.

The embodiment described above can be expressed as follows.
The memory that stores the program and
With a processor,
When the processor executes the program,
Extract unique expressions from the utterances of multiple target users
Based on the behavior history of the target user, it is determined for each target user whether or not the target user has visited a specific point where the number of visits has increased sharply.
A multidimensional feature amount that combines the extracted named entity and the determined result is generated for each target user.
Clustering of the plurality of target users who generated the feature amount was performed.
For each cluster generated by the clustering, a dictionary for at least one of speech recognition and natural language understanding is generated.
An information processing device that is configured as such.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Information providing system, 100 ... Information providing device, 102 ... Communication unit, 104 ... Authentication unit, 106 ... Acquisition unit, 108 ... Voice recognition unit, 110 ... Natural language processing unit, 112 ... Judgment unit, 114 ... User vector generation Unit, 116 ... Analysis unit, 118 ... Collection unit, 120 ... Dictionary generation unit, 122 ... Verification unit, 124 ... Providing unit, 130 ... Storage unit, 300 ... Communication terminal, 310 ... Terminal side communication unit, 320 ... Input unit, 330 ... Display, 340, 630 ... Speaker, 350, 610 ... Mike, 355 ... Position acquisition unit, 360 ... Camera, 370 ... App execution unit, 380 ... Output control unit, 390 ... Terminal side storage unit, 500 ... Agent device, 520 ... Management unit, 540 ... Agent function unit, 560 ... Vehicle side storage unit, 620 ... Display / operation device, 640 ... Navigation device 640 ... MPU, 660 ... Vehicle equipment, 670 ... In-vehicle communication device, 680 ... General-purpose communication device, 690 ... Crew recognition device, 700 ... Automatic operation control device, M ... Vehicle

Claims (11)

An extractor that extracts named entities from the utterances of multiple target users,
Based on the behavior history of the target user, a determination unit that determines for each target user whether or not the target user has visited a specific point where the number of visits has increased sharply.
A first generation unit that generates a feature amount that combines the named entity extracted by the extraction unit and the determination result by the determination unit for each target user.
An analysis unit that clusters the plurality of target users whose features are generated by the first generation unit, and an analysis unit.
A second generator that generates a dictionary for at least one of speech recognition and natural language understanding for each cluster generated by the clustering.
Information processing device equipped with. The specific point is a point where the number of visits by another user is equal to or greater than the threshold value, or the number of visits by the other user is a point where the rate of increase per predetermined period is equal to or greater than the threshold value.
The information processing apparatus according to claim 1. Further, a collecting unit is provided for collecting the co-occurrence expression of the named entity extracted from the utterance of the target user belonging to the cluster for each cluster.
The second generation unit generates the dictionary containing the co-occurrence expressions collected by the collection unit for each cluster.
The information processing apparatus according to claim 1 or 2. Further, a providing unit that provides usage guidance information of the dictionary corresponding to the specific cluster among the plurality of dictionaries is further provided to the target user belonging to the specific cluster among the plurality of the clusters.
The information processing apparatus according to any one of claims 1 to 3. The second generation unit generates a new dictionary by combining the dictionary generated for each cluster and the existing dictionary.
The providing unit provides the target user belonging to the specific cluster with usage guidance information of the new dictionary in which the dictionary corresponding to the specific cluster and the existing dictionary are combined.
The information processing apparatus according to claim 4. A verification unit that verifies the dictionary based on the utterance of the target user in a predetermined group of users is further provided.
The information processing apparatus according to any one of claims 1 to 5. The first generation unit generates a combination of a first feature amount based on the named entity and a second feature amount based on a determination result by the determination unit as the feature amount.
The information processing apparatus according to any one of claims 1 to 6. The second feature amount includes a feature amount representing one or both of the presence / absence of a visit to the specific point and the number of visits to the specific point.
The information processing apparatus according to claim 7. The named entity includes the wording of a place name or mark,
The information processing apparatus according to any one of claims 1 to 8. The computer
Extract unique expressions from the utterances of multiple target users
Based on the behavior history of the target user, it is determined for each target user whether or not the target user has visited a specific point where the number of visits has increased sharply.
A feature amount that combines the extracted named entity and the determined result is generated for each target user.
Clustering of the plurality of target users who generated the feature amount was performed.
For each cluster generated by the clustering, a dictionary for at least one of speech recognition and natural language understanding is generated.
Information processing method. On the computer
Extracting named entity from each utterance of multiple target users,
Based on the behavior history of the target user, it is determined for each target user whether or not the target user has visited a specific point where the number of visits has increased sharply.
To generate a feature amount that combines the extracted named entity and the determined result for each target user.
Performing clustering of the plurality of target users who generated the feature amount,
To generate a dictionary for at least one of speech recognition and natural language understanding for each cluster generated by the clustering.
A program to execute.

Images