JP2022542415A - Systems and methods for managing spoken queries using pronunciation information - Google Patents

Abstract

The system receives the spoken query at the audio interface and converts the spoken query to text. During conversion, the system may determine phonetic information, generate metadata indicating the pronunciation of one or more words of the query, include phonemic information within the text query, or both. A query includes one or more entities that can be more accurately identified based on pronunciation. The system searches for information, content, or both in one or more databases based on generated text queries, phonetic information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, retrieves the identified information, and provides it to the user.

Description

TECHNICAL FIELD This disclosure relates to systems for managing phonetic queries and, more particularly, to systems for managing phonetic queries based on pronunciation information.

In conversational systems, when a user issues a spoken query to the system, the speech is converted to text using an automatic speech recognition (ASR) module. This text then forms the input to the conversation system, which determines the responses to the text. For example, when a user says, "Show me a Tom Cruise movie," the ASR module converts the user's speech to text and emits it to the conversation system. The conversation system only acts on the text it receives from the ASR module. Sometimes, in this process, the conversation system loses details of the pronunciation of words or sounds contained in the user's query. Pronunciation details can provide information that can be useful in searching, especially when the same word has two or more pronunciations and the pronunciations correspond to different meanings.

This disclosure describes systems and methods for conducting a search and predicting the user's intended search query based on multiple contextual inputs as the user speaks query words. Searches may be based on multiple contextual inputs including, for example, user search history, user likes and dislikes, general trends, query word pronunciation details, and any other suitable information. An application receives a spoken query and generates a text query representing the spoken query. Applications use phonetic information, which may be included in the metadata associated with the text query included in the text query, or included in the metadata of entities in the database, to more accurately retrieve search results. In some embodiments, the application generates metadata based on text-to-speech and speech-to-text conversions to improve reachability of entities from search queries.

The above and other objects and advantages of the present disclosure will become apparent upon consideration of the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout.

FIG. 1 depicts a block diagram of an illustrative system for generating text queries, according to some embodiments of the present disclosure.

FIG. 2 shows a block diagram of an illustrative system for retrieving content in response to voice queries, according to some embodiments of the present disclosure.

FIG. 3 shows a block diagram of an illustrative system for generating pronunciation information, according to some embodiments of the disclosure.

FIG. 4 is a block diagram of illustrative user equipment, according to some embodiments of the present disclosure.

FIG. 5 shows a block diagram of an illustrative system for responding to voice queries, according to some embodiments of the disclosure.

FIG. 6 shows a flowchart of an illustrative process for responding to spoken queries based on pronunciation information, according to some embodiments of the present disclosure.

FIG. 7 shows a flowchart of an illustrative process for responding to spoken queries based on alternative representations, according to some embodiments of the present disclosure.

FIG. 8 shows a flowchart of an illustrative process for generating metadata about entities based on pronunciation, according to some embodiments of the present disclosure.

FIG. 9 depicts a flowchart of an illustrative process for retrieving content associated with an entity of a voice query, according to some embodiments of the present disclosure.

例証するために、ユーザは、「Ｌｏｕｉｓのインタビューを見せて」とシステムのオーディオインターフェースに対して声に出し得る。システムは、以下等の例証的テキストクエリを生成し得る。
オプション１）「Ｆｒａｕｄ ＭａｇａｚｉｎｅとのＬｏｕｉｓ Ｆｒｅｅｈのインタビューを見せて」
オプション２）「ＣＢＳで放送されたＬｅｗｉｓ Ｂｌａｃｋのインタビューを見せて」
結果として生じるテキストクエリは、ユーザが単語「Ｌｏｕｉｓ」を発話した方法に依存する。ユーザが、「ＬＯＯ－ｅｅ」と発音した場合、システムは、オプション１を選択するか、または、より重い重みをオプション１に適用する。ユーザが、「ＬＯＯ－ｈｉｓ」と発音した場合、システムは、オプション２を選択するか、または、より重い重みをオプション２に適用する。発音が考慮されないと、システムは、音声クエリに正確に応答することが可能ではないであろう可能性が高い。 In some embodiments, the present disclosure provides a system configured to receive spoken queries from users, analyze the spoken queries, and generate text queries (e.g., diversions) for retrieving content or information. for The system responds to voice queries based in part on the pronunciation of one or more keywords. For example, in the English language, there are multiple words with the same spelling but different pronunciations. This can be especially true for people's names. Some examples include:

To illustrate, a user may say "Show me an interview with Louis" into the system's audio interface. The system may generate an illustrative text query such as the following.
Option 1) "Show me Louis Freeh's interview with Fraud Magazine"
Option 2) "Show me the Lewis Black interview that aired on CBS."
The resulting text query depends on how the user pronounced the word "Louis." If the user pronounces "LOO-ee", the system selects option 1 or applies a heavier weight to option 1. If the user pronounces "LOO-his", the system selects option 2 or applies a heavier weight to option 2. If pronunciation is not considered, it is likely that the system will not be able to respond accurately to spoken queries.

In some situations, a spoken query containing a partial name of a person can cause ambiguity in correctly detecting that person (eg, referred to as a "non-deterministic person search query"). For example, if the user says, "Show me a movie starring Tom" or "Show me an interview with Louis," the system asks if it is Tom or Louis/Louie/Lewis that the user is asking. would need to be determined. In addition to pronunciation information, the system can, for example, collect user search history (e.g., previous queries and search results), user likes/dislikes/preferences (e.g., from user profile information), (e.g., multiple user ), popularity (eg, among multiple users), any other suitable information, or any combination thereof. The system retains the pronunciation information in a suitable form (eg, in the text query itself or in metadata associated with the text query) so that it is not lost after the automatic speech recognition (ASR) process.

In some embodiments, for phonetic information to be used by the system, the information fields in which the system searches should contain the phonetic information for comparison with the query. For example, an information field may contain information about an entity that includes phonetic metadata. The system may implement a phoneme conversion process, which takes the user's spoken query as input and converts it into text that, when read back, sounds phonetically correct. The system may be configured to use the output of the phoneme conversion process and pronunciation metadata to determine search results. In an illustrative example, phonetic metadata stored for an entity may include:

In some embodiments, the present disclosure provides a system configured to receive spoken queries from users, analyze the spoken queries, and generate text queries (e.g., diversions) for retrieving content or information. target. Information fields that the system retrieves include phonetic metadata, alternative textual representations of entities, or both. For example, when a user issues a spoken query to the system, the system first converts the speech to text using the ASR module. The resulting text then forms the input to a conversation system (eg, to perform actions in response to queries). To illustrate, if a user says, "Show me a Tom Cruise movie," the ASR module converts the user's utterance to text and issues a text query to the conversation system. If an entity corresponding to "Tom Cruise" exists in the data, the system will match it with the text "Tom Cruise" and display the appropriate results (e.g., information about Tom Cruise, content featuring Tom Cruise , or its content identifier). An entity may be referred to as 'reachable' when it exists within the data (eg, in an information field) and can be directly accessed using the entity title. Reachability is of utmost importance for the system to perform search operations. For example, if some data (e.g., movie, artist, television series, or other entity) exists within the system and the associated data is stored, the user cannot access that information. If so, the entity may be referred to as "unreachable." An unreachable entity in the data system represents a retrieval system failure.

The system may identify one or more entities or content items among the plurality of stored information. In some embodiments, the system generates an audio file based on the first text string representing the entity or content item. Based on the first text string and at least one speech criterion, the system may use the speech to text module to generate a second text string based on the audio file. The system compares the text strings and stores the second text string if the second text string is not identical to the first text string. In some embodiments, the system generates metadata that includes results from text-to-speech-to-text conversions to anticipate possible misidentifications when responding to spoken queries during search operations. Metadata may include alternative representations of entities to improve reachability.

FIG. 1 shows a block diagram of an illustrative system 100 for generating text queries, according to some embodiments of the disclosure. System 100 includes ASR module 110 , conversation system 120 , pronunciation metadata 150 , user profile information 160 and one or more databases 170 . For example, ASR module 110 and conversation system 120, which together may be included in system 199, may be used to implement query applications.

A user may vocalize a query 101 containing the utterance "Show me that Louis interview from last week" into the system 199 audio interface. ASR module 110 is configured to sample, condition, and digitize received audio input, analyze the resulting audio file, and generate text queries. In some embodiments, ASR module 110 reads information from user profile information 160 to help generate text queries. For example, speech recognition information about the user may be stored in user profile information 160, and ASR module 110 may use the speech recognition information to identify the speaking user. In a further example, system 199 may include user profile information 160 stored in suitable memory. ASR module 110 may determine phonetic information for the spoken word "Louis." Because there is more than one pronunciation for the text word "Louis," system 199 generates a text query based on the pronunciation information. Furthermore, the sound "Loo-his" can be translated into text as "Louis" or "Lewis", thus the context information is the correct entity of the spoken query (e.g., Louis as opposed to Louis Farrakhan, Lewis as in Lewis Black). In some embodiments, conversation system 120 uses recognized words from ASR module 110, contextual information, user profile information 160, pronunciation metadata 150, one or more databases 170, any other information, or is configured to generate text queries, respond to text queries, or both, based on any combination of For example, conversation system 120 may generate a text query and then compare the text query to pronunciation metadata 150 for multiple entities to determine matches. In a further example, conversation system 120 compares one or more recognized words to pronunciation metadata 150 for multiple entities to determine matches, and then generates text queries based on the identified entities. obtain. In some embodiments, conversation system 120 generates text queries with accompanying phonetic information. In some embodiments, conversation system 120 generates text queries with embedded phonetic information. For example, a text query may include the phoneme representation of a word such as "loo-ee" rather than the correct grammatical representation "Louis". In a further example, pronunciation metadata 150 may include one or more reference phoneme representations against which text queries may be compared.

User profile information 160 includes user identification information (e.g., name, identifier, address, contact information), user search history (e.g., previous spoken queries, previous text queries, previous search results, previous search results or queries user preferences (e.g., search preferences, favorite entities, keywords included in more than one query), user likes/dislikes (e.g., entities followed by the user within a social media application, user input information), other users connected to the user (e.g., friends, family, contacts in social networking applications, contacts stored on the user device), user voice data (e.g., audio samples, signatures, utterances) patterns, or files for identifying the user's voice), any other suitable information about the user, or any combination thereof.

One or more databases 170 contain any suitable information for generating text queries, responding to text queries, or both. In some embodiments, pronunciation metadata 150 , user profile information 160 , or both may be contained in one or more databases 170 . In some embodiments, one or more databases 170 contain statistical information (eg, search histories, content consumption histories, consumption patterns) about multiple users. In some embodiments, one or more databases 170 store information about multiple entities, including people, places, objects, events, content items, media content associated with one or more entities, or combinations thereof. including.

FIG. 2 shows a block diagram of an illustrative system 200 for retrieving content in response to voice queries, according to some embodiments of the present disclosure. System 200 includes a speech processing system 210 , a search engine 220 , an entity database 250 and user profile information 240 . Speech processing system 210 may identify audio files and analyze them for phonemes, patterns, words, or other elements for which keywords may be identified. In some embodiments, speech processing system 210 may analyze audio input and identify words in the temporal domain, the spectral domain, or both. For example, speech processing system 210 may analyze the audio input in the time domain and determine periods during which speech occurs (eg, to eliminate periods of pause or silence). Speech processing system 210 may then analyze each period in the spectral domain to identify phonemes, patterns, words, or other elements from which keywords may be identified. Speech processing system 210 may output the generated text query, one or more words, pronunciation information, or a combination thereof. In some embodiments, speech processing system 210 may retrieve data from user profile information 240 for speech recognition, speech recognition, or both.

Search engine 220 receives the output from speech processing system 210 and combines it with search settings 221 and contextual information 222 to generate a response to a text query. Search engine 220 may use user profile information 240 to generate, modify, or respond to text queries. The search engine 220 searches through data in the database of entities 250 using text queries. The database of entities 250 may include metadata associated with multiple entities, content associated with multiple entities, or both. For example, the data may include an identifier for the entity, details describing the entity, a title referring to the entity (eg, which may include phonemic representations or alternate representations), phrases associated with the entity (eg, which may include phonemic representations or alternate representations). , links associated with the entity (e.g., IP addresses, URLs, hardware addresses), keywords associated with the entity (e.g., which may include phonemic expressions or alternate expressions), any other suitable information, or any combination thereof. When the search engine 220 identifies one or more entities that match the keywords of the text query, identifies one or more content items that match the keywords of the text query, or both, the search engine 220 may then provide information, content, or both to the user as a response 270 to the text query. In some embodiments, search settings 221 are databases, entities, types of entities, types of content, other search criteria, or any of these that affect the generation of text queries, the retrieval of search results, or both. Including combinations. In some embodiments, the contextual information 222 includes genre information (eg, to further refine the search field), keywords, database identification (eg, databases likely to contain the target information or content), type of content (eg, , date, genre, title, format), any other suitable information, or any combination thereof. Response 270 may include, for example, content (eg, video to be displayed), information, a list of search results, links to content, any other suitable search results, or any combination thereof.

FIG. 3 shows a block diagram of an illustrative system 300 for generating pronunciation information, according to some embodiments of the present disclosure. System 300 includes a text-to-speech engine 310 and a speech-to-text engine 320 . In some embodiments, the system 300 determines phonetic information independently from text or phonetic queries. For example, system 300 may generate metadata about one or more entities (eg, metadata stored in pronunciation metadata 150 of system 100 or metadata stored in database of entities 250 of system 200). The text-to-speech engine 310 may identify a first text string 302 that may include entity names or other identifiers that are likely to be included in the spoken query. For example, the text-to-speech engine 310 may allow the user to speak a spoken query that includes a name rather than a numeric or alphanumeric identifier (e.g., the user speaks "Louis" instead of "WIKI04556"). Since it is higher, it may identify the 'name' field of the entity metadata rather than the 'id' field. Text-to-speech engine 310 generates audio output 312 on a speaker or other audio device based on the first text string. For example, the text-to-speech engine 310 uses one or more settings to adjust audio details (e.g., male/female voice, accent, or other details), playback speed, or any other suitable setting may be defined. Speech-to-text engine 320 receives audio input 313 from audio output 312 at a microphone or other suitable device (eg, in addition to or instead of an audio file that may be stored) and performs text conversion of audio input 313. Generate (eg, in addition to or instead of storing an audio file of recorded audio). Speech-to-text engine 320 may use processing settings to generate new text strings 322 . New text string 322 is compared to first text string 302 . If the new text string 322 is identical to the text string 302, no metadata needs to be generated as the spoken query can result in a conversion to an exact text query. If the new text string 322 is not identical to the text string 302, this indicates that the spoken query may have been incorrectly converted to a text query. Thus, if the new text string 322 is not identical to the text string 302, the utterance-to-text engine 320 includes the new text string 322 in the metadata associated with the entity with which the text string 302 is associated. The system 300 identifies multiple entities and generates metadata for each entity that includes the resulting text strings from the text-to-speech engine 310 and the speech-to-text engine 320 (e.g., new text string 322, etc.). can. In some embodiments, for a given entity, Text->Speech engine 310, Speech->Text engine 320, or both use two or more settings to generate two or more new text strings. obtain. Thus, since two or more text strings differ from text string 302, each new text string can then be stored in the metadata. For example, different pronunciations or pronunciation interpretations resulting from different settings can generate different new text strings, which can be stored for spoken queries from different users. By generating and storing alternate representations (e.g., text strings 302 and new text strings 322), system 300 may update metadata and enable more precise searches (e.g., entity reachability and improve search accuracy).

In the illustrative example, for entities, system 300 identifies titles and related phrases, runs each phrase through text-to-speech engine 310, saves respective audio files, and then passes each respective audio file to speech-to-text. It is passed through engine 320 to obtain an ASR transcription record (eg, new text string 322). If the ASR transcription record differs from the original phrase (eg, text string 302), system 300 adds the ASR transcription record to the entity's related phrases (eg, as stored in metadata). In some embodiments, the system 300 does not require any manual work and can be fully automated (eg, no user input is required). In some embodiments, system 300 is alerted when a user issues a query and does not get the desired results. In response, a human manually identifies what should be the correct entity for the query. Incorrect results are stored to provide information for future queries. System 300 addresses potential inaccuracies at the metadata level rather than the system level. Analysis of the text string 302 for many entities can be exhaustive and automatic such that all false examples are identified and resolved in advance (eg, prior to the user's voice query). System 300 does not require the user to provide a spoken query to generate false examples (eg, alternative expressions). The system 300 can be used to emulate user interaction with the query system and anticipate potential sources of error in conducting searches.

A user may access content, an application (e.g., for interpreting voice queries), and, for example, its device (i.e., user equipment or audio equipment), one or more network-connected devices, one or more electronic devices with displays. Other features from one or more of the devices, or combinations thereof, may be accessed. Any of the illustrative techniques of this disclosure may be implemented in a user device, a device providing a display to a user, or any other suitable control circuitry configured to respond to voice queries and generate display content to a user. can be implemented by

FIG. 4 shows a generalized embodiment of an illustrative user device. User equipment system 401 includes or may include set-top box 416 communicatively coupled to display 412 , audio equipment 414 , and user input interface 410 . In some embodiments, display 412 may include a television display or computer display. In some embodiments, user input interface 410 is a remote control device. Set-top box 416 may include one or more circuit boards. In some embodiments, one or more circuit boards include processing circuitry, control circuitry, and storage devices (eg, RAM, ROM, hard disks, removable disks, etc.). In some embodiments, the circuit board includes input/output paths. User equipment device 400 and user equipment system 401 may each receive content and data via input/output (hereinafter “I/O”) path 402 . I/O paths 402 may provide content and data to control circuitry 404 , which includes processing circuitry 406 and storage device 408 . Control circuitry 404 may be used to send and receive commands, requests, and other suitable data using I/O paths 402 . I/O path 402 may connect control circuitry 404 (specifically, processing circuitry 406) to one or more communication paths (described below). An I/O function may be provided by one or more of these communication paths, but is shown as a single path in FIG. 4 to avoid overcomplicating the drawing. A set-top box 416 is shown in FIG. 4 for purposes of illustration, but any suitable computing device having processing circuitry, control circuitry, and storage may be used in accordance with the present disclosure. For example, set-top box 416 may be a personal computer (eg, notebook, laptop, desktop), network-based server hosting user-accessible client devices, non-user-owned device, any other suitable device, or any other suitable device. can be substituted or complemented by any combination of

Control circuitry 404 may be based on any suitable processing circuitry, such as processing circuitry 406 . As referred to herein, processing circuitry may be one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc. and may include multi-core processors (eg, dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputers. In some embodiments, the processing circuitry includes multiple separate processors or processing units, such as multiple processing units of the same type (e.g., two Intel Core i7 processors) or multiple different processors (e.g., Intel Core i7 processors). i5 processor and Intel Core i7 processor). In some embodiments, control circuitry 404 executes instructions for applications stored in memory (eg, storage device 408). Specifically, control circuitry 404 may be instructed by an application to perform the functions discussed above and below. For example, an application may provide instructions to control circuitry 404 to generate a media guidance display. In some implementations, any action performed by control circuitry 404 may be based on instructions received from an application.

In some client/server-based embodiments, control circuitry 404 includes communication circuitry suitable for communicating with application servers or other networks or servers. Instructions for performing the functionality described above may be stored on the application server. The communication circuitry may be a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet (RTM) modem, to communicate with other equipment or any other suitable communication circuitry. ) card, or a wireless modem. Such communication may involve the Internet or any other suitable communication network or pathway. Additionally, communication circuitry may include circuitry (discussed in more detail below) that enables peer-to-peer communication of user equipment devices or communication of user equipment devices located remotely from one another.

The memory may be an electronic storage device, such as storage device 408 that is part of control circuitry 404 . As referred to herein, the phrase "electronic storage device" or "storage device" includes random access memory, read-only memory, hard drive, optical drive, solid state device, quantum storage device, game console, game media , or any other suitable fixed or removable storage device, or any combination thereof, for storing electronic data, computer software, or firmware. Storage device 408 may be used to store the various types of content described herein and the media guidance data described above. Non-volatile memory may also be used (eg, to launch boot-up routines and other instructions). Cloud-based storage may be used to supplement or replace storage 408, for example.

A user may use user input interface 410 to send commands to control circuitry 404 . User input interface 410, display 412, or both may include a touch screen configured to provide display and receive tactile input. For example, touch screens may be configured to receive tactile input from a finger, a stylus, or both. In some embodiments, the equipment device 400 may include a front facing screen and a rear facing screen, multiple front screens, or multiple angled screens. In some embodiments, user input interface 410 is a remote control device having one or more microphones, buttons, keypads, any other components configured to receive user input, or combinations thereof. include. For example, user input interface 410 may include a handheld remote control device with an alphanumeric keypad and options. In a further example, user input interface 410 may include a handheld remote control device having a microphone and control circuitry configured to receive and identify voice commands and transmit information to set top box 416 .

Audio equipment 414 may be provided as integrated with other elements of each of user device 400 and user equipment system 401, or may be a stand-alone unit. The audio component of videos and other content displayed on display 412 may be played through speakers of audio device 414 . In some embodiments, the audio may be distributed to a receiver (not shown), which processes the audio and outputs it through speakers of audio device 414 . In some embodiments, for example, control circuitry 404 is configured to provide audio cues to the user or other audio feedback to the user using speakers of audio device 414 . Audio device 414 may include a microphone configured to receive audio input such as voice commands and utterances (eg, including voice queries). For example, a user may speak letters or words that are received by a microphone and converted to text by control circuit 404 . In a further example, the user may speak a command, which is received by the microphone and recognized by control circuit 404 .

Applications (eg, for managing voice queries) may be implemented using any suitable architecture. For example, a standalone application may be fully implemented on each of user device 400 and user equipment system 401 . In some such embodiments, instructions for the application are stored locally (e.g., in storage device 408) and data for use by the application are sent on a periodic basis (e.g., out-of-band feed , from Internet resources, or using another suitable approach). Control circuitry 404 may retrieve instructions for an application from storage device 408, process the instructions, and generate any of the displays discussed herein. Based on the processed instructions, control circuitry 404 may determine what actions to perform when input is received from user input interface 410 . For example, movement of the cursor on the display up/down may be indicated by the processed instructions when the input interface 410 indicates that the up/down button has been selected. An application and/or any instructions for implementing any of the embodiments discussed herein may be encoded on a computer-readable medium. Computer-readable media include any medium that can store data. Computer readable media include, but are not limited to, propagating electrical or electromagnetic signals and may be transient or include, but are not limited to hard disks, floppy disks, USB drives, DVDs, CDs , media cards, register memory, processor cache, random access memory (RAM), etc., and may be non-transitory.

In some embodiments the application is a client/server based application. Data for use by a thick or thin client implemented on each of user device 400 and user equipment system 401 issues a request to a server remote from each of user equipment device 400 and user equipment system 401. read on demand by For example, a remote server may store instructions for an application in a storage device. The remote server may use circuitry (eg, control circuitry 404) to process stored instructions and generate the displays discussed above and below. The client device may receive the display generated by the remote server and display the content of the display locally on the user device 400 . In this manner, the processing of instructions is performed remotely by the server, while the resulting display, which may include text, keyboard, or other visuals, is provided locally on user device 400 . User device 400 may receive inputs from a user via input interface 410 and transmit those inputs to a remote server for processing and generating a corresponding display. For example, user device 400 may transmit a communication to a remote server indicating that the up/down button was selected via input interface 410 . The remote server may process the instructions according to the input and generate a display of the application corresponding to the input (eg, a display that moves the cursor up/down). The generated display is then transmitted to user device 400 for presentation to the user.

In some embodiments, the application is downloaded, interpreted by an interpreter or virtual machine (eg, launched by control circuitry 404), or otherwise launched. In some embodiments, applications may be encoded in ETV Binary Interchange Format (EBIF), received by control circuitry as part of a preferred feed, and interpreted by a user agent running on control circuitry 404 . For example, the application can be an EBIF application. In some embodiments, an application may be defined by a series of JAVA-based files that are received and launched by a local virtual machine or other suitable middleware executed by control circuitry 404 .

FIG. 5 shows a block diagram of an illustrative network arrangement 500 for responding to voice queries, according to some embodiments of the disclosure. Illustrative system 500 may represent a situation in which a user provides a voice query at user device 550, views content on a display of user device 550, or both. Although there may be more than one type of user device in system 500, only one is shown in FIG. 5 to avoid overcomplicating the drawing. Additionally, each user may utilize more than one type of user device and may also utilize more than one type of user device. User device 550 may be the same as user device 400 of FIG. 4, user equipment system 401, any other suitable device, or any combination thereof.

User device 550, illustrated as a wireless-enabled device, may be coupled to communication network 510 (eg, connected to the Internet). For example, user device 550 is coupled to communication network 510 via a communication path (eg, which may include an access point). In some embodiments, user device 550 may be a computing device coupled to communication network 510 via a wired connection. For example, user device 550 may also include a wired connection to a LAN or any other suitable communication link to network 510 . Communication network 510 may include one or more of the Internet, a cellular network, a mobile voice or data network (eg, a 4G or LTE network), a cable network, a public switched telephone network, or any other type of communication network or combination of communication networks. can be a network of A communication path may be a satellite path, a fiber optic path, a cable path, a path supporting Internet communications, a free space connection (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communication path. or may include one or more communication paths, such as a combination of such paths. Although communication paths are not drawn between user device 550 and network device 520, these devices can include communication paths such as those described above, as well as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth ( (trademark), infrared, IEEE 802-11x, etc.) or other short range communication via wired or wireless paths. BLUETOOTH (registered trademark) is a trademark of Bluetooth (registered trademark) SIG, Inc. is a certification mark owned by Devices may also communicate with each other directly through indirect paths through communication network 510 .

System 500 as illustrated includes network device 520 (eg, a server or other suitable computing device) coupled to communication network 510 via a suitable communication path. Communications between network device 520 and user device 550 may be exchanged via one or more communication paths, but to avoid overcomplicating the drawing, FIG. is shown as Network device 520 may include a database and one or more applications (eg, as an application server, host server). Multiple network entities may exist and communicate with network 510, but only one is shown in FIG. 5 to avoid overcomplicating the drawing. In some embodiments, network device 520 may include one source device. In some embodiments, network device 520 implements an application that communicates with instances of the application on many user devices (eg, user device 550). For example, an instance of a social media application may be implemented on user device 550 and application information communicated to and from network device 520 that may store profile information about the user (e.g., the current social media feed is , as available on devices other than user device 550). In a further example, an instance of a search application may be implemented on the user device 550, and the application information may include profile information about the user, search history from multiple users, entity information (e.g., content and metadata), any other is communicated to and from network device 520, which may store suitable information for the , or any combination thereof.

In some embodiments, network device 520 stores, for example, entity information, metadata, content, historical communication and search records, user preferences, user profile information, any other suitable information, or any combination thereof. including one or more types of stored information. network device 520 provides an application host database or server, plug-ins, software developer kits (SDKs), application programming interfaces (APIs), or software (e.g., as downloaded to user devices); For example, other software tools configured to remotely launch software (hosting applications accessed by the user device) or otherwise provide application support to applications on the user device 550 can include In some embodiments, information from network device 520 is provided to user device 550 using a client/server approach. For example, user device 550 may pull information from a server, or a server may push information to user device 550 . In some embodiments, an application client residing on user device 550 initiates a session with network device 520 and updates as needed (e.g., as data becomes stale or as the user device receives data). information may be obtained upon receiving a request from a user for In some embodiments, information may include user information (eg, user profile information, user-generated content). For example, user information may include content transactions the user has engaged in, searches the user has conducted, content the user has consumed, whether the user interacts with social networks, any other suitable information, or any combination thereof, etc. current and/or historical user activity information. In some embodiments, user information may identify patterns for a given user over time. As shown, network device 520 includes entity information for multiple entities. Entity information 521, 522, and 523 contain metadata about each entity. Entities for which metadata is stored in the network device 520 may be linked to each other, referenced to each other, described by one or more tags in the metadata, or a combination thereof. .

In some embodiments, applications may be implemented on user device 550, network device 520, or both. For example, an application may be implemented as software or a set of executable instructions, which may be stored in the storage of user device 550, network device 520, or both, and executed by control circuitry of the respective devices. In some embodiments, the applications may include audio recording applications, speech-to-text applications, text-to-speech applications, voice-recognition applications, or combinations thereof implemented as client/server-based applications; resides on the user device 550 and the server application resides on a remote server (eg, network device 520). For example, an application may run partly as a client application on user device 550 (e.g., by control circuitry of user device 550) and partly on a remote server, by control circuitry of a remote server (e.g., network device 520). can be implemented as a server application running on the control circuit of the When executed by the control circuitry of the remote server, the application may instruct the control circuitry to generate a display and transmit the generated display to user device 550 . The server application may instruct the remote device's control circuitry to transmit the data for storage on the user device 550 . The client application may instruct control circuitry of the receiving user device to generate the application display.

In some embodiments, the deployment of system 500 is a cloud-based deployment. The cloud provides access to services such as information storage, retrieval, messaging, or social networking services, among other examples, and access to any of the content described above with respect to user devices. Services can be offered in the cloud through cloud-computing service providers or through other providers of online services. For example, cloud-based services may include storage services, sharing sites, social networking sites, search engines, or other services delivered for viewing by others on devices to which User Sourced Content is connected. can. These cloud-based services may allow user devices to store information in the cloud and receive information from the cloud, rather than storing information locally and accessing information stored locally. . Cloud resources can be accessed by user devices using, for example, web browsers, messaging applications, social media applications, desktop applications, or mobile applications, such as audio recording applications, speech-to-text applications, text-to-speech applications, voice- It may include any combination of recognition applications and/or access applications thereof. User device 550 may be a cloud client that relies on cloud computing for application delivery, or user device 550 may have some functionality without access to cloud resources. For example, some applications running on user device 550 may be cloud applications (eg, applications delivered as a service over the Internet), while other applications are stored and running on user device 550. can be In some embodiments, user device 550 may receive information from multiple cloud resources simultaneously.

In the illustrative example, a user may speak a voice query to user device 550 . The voice query is recorded by the audio interface of user device 550, sampled and digitized by application 560, and converted by application 560 into a text query. Application 560 may also include pronunciations as well as text queries. For example, one or more words of a text query may be represented by phonemic symbols rather than proper spelling. In a further example, pronunciation metadata may be stored with a text query including phonemic representations of one or more words of the text query. In some embodiments, application 560 transmits text queries and any suitable pronunciation information to network device 520 for searching in a database of entities, content, metadata, or combinations thereof. Network device 520 may identify entities associated with the text query, content associated with the text query, or both, and provide that information to user device 550 .

For example, a user may speak into the microphone of user device 550, "Show me a Tom Cruise movie." Application 560 may generate the text query “Tom Cruise movies” and transmit the text query to network device 520 . Network device 520 may identify the entity “Tom Cruise” and then identify the movies linked to the entity. The network device 520 then sends content (eg, video files, trailers, or clips), content identifiers (eg, movie titles and images), content addresses (eg, URLs, websites, or IP addresses), any other Suitable information, or any combination thereof, may be transmitted to user device 550 . Since the pronunciations of "Tom" and "Cruise" are generally unambiguous, application 560 need not generate pronunciation information in this situation.

In a further example, a user may say "Show me an interview with Louis" into the microphone of user device 550, and the user pronounces the name Louis as "loo-ee" instead of "loo-ihs." In some embodiments, application 560 may generate a text query “interview with Louis” and transmit the text query to network device 520 along with metadata including the phoneme representation as “loo-ee”. In some embodiments, application 560 may generate a text query “interview with Loo-ee” and transmit the text query to network device 520, where the text query itself contains phonetic information (e.g., phoneme expression). Since the name Louis is generic, there can be many entities containing this identifier. In some embodiments, network device 520 may identify entities that have metadata that includes phonetic tags that have “loo-ee” as a phoneme representation. In some embodiments, network device 520 may retrieve trending searches, a user's search history, or other contextual information to identify entities that users are likely to point to. For example, the user may have previously searched for "FBI," and the entity Louis Freeh (eg, a former director of the FBI) may contain metadata that includes tags for "FBI." Once the entity is identified, the network device 520 then retrieves the content (eg, a video file or clip of an interview), a content identifier (eg, file title and still image from the interview), a content address (eg, one of the interviews). URL, website, or IP address for streaming one or more video files), any other suitable information related to Louis Freeh, or any combination thereof, may be transmitted to user device 550 . Since the pronunciation of "Louis" may be ambiguous, application 560 may generate pronunciation information in such situations.

テキストクエリは、正しくないスペルを含み得るが、正しいエンティティに関連付けられたメタデータが、変形例を含むので、正しいエンティティが、識別され得る。故に、ネットワークデバイス５２０は、代替表現を含むエンティティ情報を含み得、したがって、語句「Ｗｉｌｌｉａｍ ｇｊｏｋａ」を含むテキストクエリに応答して、正しいエンティティを識別し得る。エンティティが、識別されると、ネットワークデバイス５２０は、次いで、コンテンツ（例えば、オーディオまたはビデオファイルクリップ）、コンテンツ識別子（例えば、曲またはアルバムタイトルおよびコンサートからの静止画像）、コンテンツアドレス（例えば、音楽の１つ以上のオーディオファイルをストリーミングするためのＵＲＬ、ウェブサイト、またはＩＰアドレス）、Ｗｉｌｌｉａｍ Ｄｊｏｋｏに関連する任意の他の好適な情報、またはそれらの任意の組み合わせをユーザデバイス５５０に伝送し得る。名前「Ｄｊｏｋｏ」は、発話から正しくなく変換され得るので、アプリケーション５６０は、そのような状況では、正しいエンティティを識別するための発音情報をメタデータ内への記憶のために生成し得る。 In an illustrative example, a user may speak “William Djoko” into the microphone of user device 550 . Application 560 may generate text queries that may not correspond to the correct spelling of the entity. For example, the spoken query "William Djoko" may be converted to text as "William gjoka". This incorrect text conversion can lead to difficulties in identifying the correct entity. In some embodiments, the metadata associated with the entity William Djoko includes pronunciation-based alternative representations. Metadata for the entity “William Djoko” may include phonetic tags (eg, “related phrases”), as shown in Table 1.

A text query may contain incorrect spelling, but the correct entity may be identified because the metadata associated with the correct entity contains variations. Thus, network device 520 may include entity information that includes alternative representations, and thus may identify the correct entity in response to a text query that includes the phrase "William gjoka." Once the entity is identified, the network device 520 then retrieves the content (e.g., audio or video file clip), content identifier (e.g., song or album title and still image from a concert), content address (e.g., music URL, website, or IP address for streaming one or more audio files), any other suitable information related to William Djoko, or any combination thereof, may be transmitted to user device 550 . Since the name "Djoko" may be incorrectly translated from speech, application 560 may generate phonetic information for storage in metadata to identify the correct entity in such situations.

In the illustrative example above, the reachability of the entity William Djoko is improved by storing alternate representations, especially since the ASR process may result in a grammatically incorrect text translation of the entity name.

In an illustrative example, metadata may be generated based on pronunciation for later reference (eg, by text queries or other search and retrieval processes) rather than in response to a user's spoken query. In some embodiments, network device 520, user device 550, or both may generate metadata based on the pronunciation information. For example, user device 550 may receive user input of alternate representations of entities (eg, based on previous search results or speech-to-text conversions). In some embodiments, network device 520, user device 550, or both may automatically generate metadata about entities using text-to-speech and speech-to-text modules. For example, application 560 may identify a textual representation of an entity (eg, a text string of the entity's name), input the textual representation into a text-to-speech module, and generate an audio file. In some embodiments, the text-to-speech module includes one or more settings or criteria with which the audio file is generated. For example, settings or criteria can include language (e.g., English, Spanish, Mandarin), accent (e.g., local or language-based), voice (e.g., specific human voice, male voice, female voice), speed (e.g., playback time of the relevant portion of the audio file), pronunciation (eg, for multiple phoneme variants), any other suitable setting or criteria, or any combination thereof. Application 560 then inputs the audio file into the Speech to Text module and produces a resulting textual representation. If the resulting textual representation is not identical to the original textual representation, application 560 may store the resulting textual representation in metadata associated with the entity. In some embodiments, application 560 may repeat this process for different settings or criteria, thus generating different textual representations that may be stored in metadata. The resulting metadata contains the original textual representation along with the variations generated using text-to-speech-to-text transformations to predict likely variations. Thus, when application 560 receives a spoken query from a user and the translation to text does not exactly match the entity identifier, application 560 may still identify the correct entity. Further, application 560 does not need to analyze text queries for phonetic information because the metadata includes variations (eg, analysis is performed a priori rather than in real time).

Applications 560 may include, for example, audio recording, speech recognition, speech-to-text conversion, text-to-speech conversion, query generation, search engine functionality, content retrieval, display generation, content presentation, metadata generation, database functionality, or any of these. Any suitable functionality, such as combination, may be included. In some embodiments, aspects of application 560 are implemented across two or more devices. In some embodiments, application 560 is implemented on a single device. For example, entity information 521 , 522 , and 523 may be stored in memory storage of user device 550 and accessed by application 560 .

FIG. 6 shows a flowchart of an illustrative process 600 for responding to spoken queries based on phonetic information, according to some embodiments of the present disclosure. For example, the query application may use user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof, etc. process 600 may be implemented on any suitable hardware. In a further example, the query application may be an instance of application 560 of FIG.

At step 602, a query application receives a spoken query. In some embodiments, an audio interface (eg, audio device 414, user input interface 410, or a combination thereof) may include a microphone or other sensor that receives audio input and generates electronic signals. In some embodiments, audio input is received at an analog sensor that provides an analog signal that is conditioned, sampled and digitized to generate an audio file. The audio file may then be analyzed by a query application in steps 604 and 606. In some embodiments, the audio files are stored in memory (eg, storage device 408). In some embodiments, the query application includes a user interface (eg, user input interface 410) that allows users to record, play, modify, crop, visualize, or otherwise manage audio recordings. to enable. For example, in some embodiments the audio interface is configured to receive audio input at all times. By way of further example, in some embodiments, the audio interface may provide audio input when the user provides instructions to the input interface (e.g., by selecting a soft button on the touch screen to initiate audio recording). configured to receive As a further example, in some embodiments the audio interface is configured to receive audio input and initiate recording when speech or other suitable audio signal is detected. The query application may include any suitable conditioning software or hardware for converting audio input into stored audio files. For example, the query application may apply one or more filters (eg, lowpass, highpass, notch filters, or bandpass filters), amplifiers, digitizers, or other adjustments to generate the audio file. In further examples, the query application may be compression, transformation (e.g., spectral transform, wavelet transform), normalization, equalization, truncation (e.g., in the time or spectral domain), any other suitable processing, or any of them. Any suitable processing may be applied to the conditioned signal to generate an audio file, such as a combination of . In some embodiments, at step 602, the control circuit receives an audio file from a separate application, from a separate module of the query application, based on user input, or any combination thereof. For example, at step 602, control circuitry may receive the spoken query as an audio file that is stored in a storage device (eg, storage device 408) for further processing (eg, steps 604-612 of process 600). .

At step 604 , the query application extracts one or more keywords from the spoken query of step 602 . In some embodiments, one or more keywords may represent a complete spoken query. In some embodiments, one or more keywords include only significant words or portions of speech. For example, in some embodiments, a query application may identify words in an utterance and select some of those words as keywords. For example, a query application may identify words and select from those words non-prepositional words. In a further example, the query application may identify only words that are at least three letters long as keywords. In a further example, the query application may identify keywords as phrases containing two or more words (e.g., to be more descriptive and provide more context), which are useful for potential searches of related content. Can be useful for narrowing down fields. In some embodiments, the query application uses any suitable criteria for identifying keywords from audio input, such as words, phrases, names, places, channels, media asset titles, or other keywords. Identify keywords such as The query application may process words using any suitable word detection technique, speech detection technique, pattern recognition technique, signal processing technique, or any combination thereof. For example, a query application may compare a series of signal templates to a portion of the audio signal to find out whether a match exists (eg, whether a particular word is contained in the audio signal). In a further example, a query application may apply learning techniques to better recognize words in spoken queries. For example, a query application may collect feedback from users on multiple requested content items in the context of multiple queries, and thus use historical data as a training set to make recommendations and retrieve content. . In some embodiments, the query application may store snippets of recorded audio (ie, short-duration clips) during detected utterances and process the snippets. In some embodiments, the query application stores relatively large segments of speech (eg, greater than 10 seconds) as audio files and processes the files. In some embodiments, the query application may detect words by processing utterances and using continuous computation. For example, a wavelet transform can be performed on the speech in real-time, providing continuous computation of the speech pattern (which can be compared, for example, to a reference to identify words) even with some time delay. . In some embodiments, a query application may detect words and the user who pronounced the words (eg, speech recognition) in accordance with this disclosure.

In some embodiments, at step 604, the query application adds the detected word to the list of words detected in the query. In some embodiments, the query application may store these detected words in memory. For example, a query application may store in memory a set of ASCII characters (i.e., an 8-bit code), a pattern (e.g., indicating the speech signal criteria used to match words), an identifier (e.g., a code for a word). ), strings, any other data type, or any combination thereof. In some embodiments, the media guidance application may add words to memory as they are detected. For example, the media guidance application may append newly detected words to a string of previously detected words, add newly detected words to a cell array of previously detected words (e.g., cell array size ), creating a new variant corresponding to the newly detected word, creating a new file corresponding to the newly created word, or one or more of the words detected in step 604 can be performed to store

At step 606 , the query application determines pronunciation information for one or more of the keywords of step 604 . In some embodiments, the phonetic information includes phonemic representations of one or more keywords (eg, using the International Phonetic Alphabet). In some embodiments, the phonetic information includes one or more alternate spellings of one or more keywords to incorporate the pronunciation. In some embodiments, at step 606, the control circuit generates metadata associated with the text query including the phoneme representation.

At step 608 , the query application generates a text query based on the one or more keywords of step 604 and the phonetic information of step 606 . A query application may generate a text query by placing one or more keywords in a preferred order (eg, in the order in which they were spoken). In some embodiments, the query application may omit one or more words of the spoken query (eg, short words, prepositions, or any other word determined to be relatively unimportant). A text query may be generated as a file (eg, a text file) and stored in a suitable storage device (eg, storage device 408).

At step 610, a query application identifies entities among a plurality of entities in a database based on the text query and stored metadata about the entities. In some embodiments, the metadata includes phonetic tags. In some embodiments, the query application may identify entities by identifying metadata tags of content items that correspond to the entities. For example, a content item may include a movie with tags for actors in the movie. If the text query includes actors, the query application may determine matches and, based on the matches, identify entities as associated with the content item. To illustrate, a query application may first identify an entity (e.g., search within the entity) and then retrieve content associated with the entity, or a query application may first identify the content (eg, search through the content) to determine whether entities associated with the content match the text query. A database arranged by entity, by content, or both can be searched by a query application.

In some embodiments, the query application identifies entities based on user profile information. For example, a query application may identify entities based on already identified entities from previous voice queries. In a further example, a query application may identify entities based on popularity information associated with the entity (eg, based on searches for multiple users). In some embodiments, the query application identifies entities based on user preferences. For example, if one or more keywords match a preferred entity name or identifier in user profile information, the query application may identify the entity or weight it more heavily.

In some embodiments, the query application, based on identifying multiple entities (e.g., using metadata stored for each entity) and comparing each phonetic tag with the text query: For each respective entity of the plurality of entities, identifying the entity by determining a respective score and selecting the entity by determining the maximum score. The score may be based on the number of matches identified between the keywords of the text query and the metadata associated with the entity or content item.

In some embodiments, the query application identifies two or more entities (eg, associated metadata) among the plurality of entities based on the text query. A query application may identify content items associated with some or all of the entities of the query. In some embodiments, the query application identifies entities by comparing at least a portion of the text query to metadata tags stored for each entity to identify matches.

At step 612, the query application retrieves content items associated with the entity. In some embodiments, the query application identifies content items, downloads content items, streams content items, generates content items for display, or combinations thereof. . For example, the voice query may include "Show me a recent Tom Cruise movie" and the query application provides a link to the movie "Mission Impossible: Fallout" from which the user may select to view the video content. obtain. In some embodiments, the query application may retrieve multiple pieces of content associated with entities that match the text query. For example, a query application may retrieve a list of links, video files, audio files, or other content or identified content items in accordance with this disclosure.

FIG. 7 shows a flowchart of an illustrative process 700 for responding to spoken queries based on alternative representations, according to some embodiments of the present disclosure. For example, the query application may use user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof, etc. may implement process 700 on any suitable hardware. In a further example, the query application may be an instance of application 560 of FIG.

At step 702, a query application receives a spoken query. In some embodiments, an audio interface (eg, audio device 414, user input interface 410, or a combination thereof) may include a microphone or other sensor that receives audio input and generates electronic signals. In some embodiments, audio input is received at an analog sensor that provides an analog signal that is conditioned, sampled and digitized to generate an audio file. The audio file can then be analyzed by a query application at step 704 . In some embodiments, the audio files are stored in memory (eg, storage device 408). In some embodiments, the query application includes a user interface (eg, user input interface 410) that allows users to record, play, modify, crop, visualize, or otherwise manage audio recordings. to enable. For example, in some embodiments the audio interface is configured to receive audio input at all times. As a further example, in some embodiments, the audio interface outputs audio input when a user provides instructions to the user interface (e.g., by selecting a soft button on the touch screen to initiate audio recording). configured to receive. As a further example, in some embodiments the audio interface is configured to receive audio input and initiate recording when speech or other suitable audio signal is detected. The query application may include any suitable conditioning software or hardware for converting audio input into stored audio files. For example, the query application may apply one or more filters (eg, lowpass, highpass, notch filters, or bandpass filters), amplifiers, digitizers, or other adjustments to generate the audio file. In further examples, the query application may be compression, transformation (e.g., spectral transform, wavelet transform), normalization, equalization, truncation (e.g., in the time or spectral domain), any other suitable processing, or any of them. Any suitable processing may be applied to the conditioned signal to generate an audio file, such as a combination of . In some embodiments, at step 702, the control circuit receives an audio file from a separate application, from a separate module of the query application, based on user input, or any combination thereof. For example, step 702 may include receiving the spoken query as an audio file stored in a storage device (eg, storage device 408) for further processing (eg, steps 704-710 of process 700).

At step 704 the query application extracts one or more keywords from the spoken query of step 702 . In some embodiments, one or more keywords may represent a complete spoken query. In some embodiments, one or more keywords include only significant words or portions of speech. For example, in some embodiments, a query application may identify words in an utterance and select some of those words as keywords. For example, a query application may identify words and select from those words non-prepositional words. In a further example, the query application may identify only words that are at least three letters long as keywords. In a further example, the query application may identify keywords as phrases containing two or more words (e.g., to be more descriptive and provide more context), which are useful for potential searches of related content. Can be useful for narrowing down fields. In some embodiments, the query application uses any suitable criteria for identifying keywords from audio input, such as words, phrases, names, places, channels, media asset titles, or other keywords. Identify keywords such as The query application may process words using any suitable word detection technique, speech detection technique, pattern recognition technique, signal processing technique, or any combination thereof. For example, a query application may compare a series of signal templates to a portion of the audio signal to find out whether a match exists (eg, whether a particular word is contained in the audio signal). In a further example, the query application may apply learning techniques to better recognize words in spoken queries. For example, a query application may collect feedback from users on multiple requested content items in the context of multiple queries, and thus use historical data as a training set to make recommendations and retrieve content. . In some embodiments, the query application may store snippets of recorded audio (ie, short-duration clips) during detected utterances and process the snippets. In some embodiments, the query application stores relatively large segments of speech (eg, greater than 10 seconds) as audio files and processes the files. In some embodiments, the query application may detect words by processing utterances and using continuous computation. For example, a wavelet transform can be performed on the speech in real-time, providing continuous computation of the speech pattern (which can be compared, for example, to a reference to identify words) even with some time delay. . In some embodiments, the query application may detect words and the user who pronounced the words (eg, speech recognition) in accordance with this disclosure.

In some embodiments, at step 704, the query application adds the detected word to the list of words detected in the query. In some embodiments, the query application may store these detected words in memory. For example, a query application may store in memory a set of ASCII characters (i.e., an 8-bit code), a pattern (e.g., indicating the speech signal criteria used to match words), an identifier (e.g., a code for a word). ), strings, any other data type, or any combination thereof. In some embodiments, the media guidance application may add words to memory as they are detected. For example, the media guidance application may append newly detected words to a string of previously detected words, add newly detected words to a cell array of previously detected words (e.g., cell array size ), creating a new variant corresponding to the newly detected word, creating a new file corresponding to the newly created word, or one or more of the words detected in step 704 can be performed to store

At step 706 the query application generates a text query based on the one or more keywords of step 704 . A query application may generate a text query by placing one or more keywords in a preferred order (eg, in the order in which they were spoken). In some embodiments, the query application may omit one or more words of the spoken query (eg, short words, prepositions, or any other word determined to be relatively unimportant). A text query may be generated as a file (eg, a text file) and stored in a suitable storage device (eg, storage device 408).

At step 708, the query application identifies entities based on the text query and metadata about the entities. Metadata includes alternative textual representations of entities based on their pronunciation. In some embodiments, the query application may identify entities by identifying metadata tags of content items that correspond to alternate representations of the entity. For example, a content item may include movies with tags for actors in the movies, where the tags include alternate spellings (eg, derived from a system such as system 300 or otherwise included in metadata). . If the text query includes actors, the query application may determine matches and, based on the matches, identify entities as associated with the content item. To illustrate, a query application may first identify an entity (e.g., search within the entity) and then retrieve content associated with the entity, or a query application may first identify the content (eg, search through the content) to determine whether entities associated with the content match the text query. A database arranged by entity, by content, or both can be searched by a query application. A query application may determine a match when one or more words of a text query match an alternative representation of an entity (eg, as stored in metadata associated with the entity).

In some embodiments, the query application identifies entities based on user profile information. For example, a query application may identify entities based on already identified entities from previous voice queries. In a further example, a query application may identify entities based on popularity information associated with the entity (eg, based on searches for multiple users). In some embodiments, the query application identifies entities based on user preferences. For example, if one or more keywords match a preferred entity name or alternative representation of an identifier in the user profile information, the query application may identify the entity or weight it more heavily.

In some embodiments, a query application generates multiple entities based on identifying multiple entities (eg, with metadata stored for each entity) and comparing the respective metadata with the text query. , and identifying the entity by determining the respective score and selecting the entity by determining the maximum score. The score may be based on the number of matches identified between the keywords of the text query and the metadata associated with the entity or content item.

In some embodiments, the query application identifies two or more entities (eg, associated metadata) among the plurality of entities based on the text query. A query application may identify content items associated with some or all of the entities of the query. In some embodiments, the query application identifies entities by comparing at least a portion of the text query to metadata tags stored for each entity to identify matches.

At step 710, the query application retrieves content items associated with the entity. In some embodiments, the query application identifies content items, downloads content items, streams content items, generates content items for display, or combinations thereof. . For example, the voice query may include "Show me a recent Tom Cruise movie" and the query application provides a link to the movie "Mission Impossible: Fallout" from which the user may select to view the video content. obtain. In some embodiments, the query application may retrieve multiple pieces of content associated with entities that match the text query. For example, a query application may retrieve a list of links, video files, audio files, or other content or identified content items in accordance with this disclosure.

FIG. 8 shows a flowchart of an illustrative process 800 for generating metadata about entities based on pronunciation, according to some embodiments of the present disclosure. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof. Process 800 may be implemented on any suitable hardware. In a further example, the application may be an instance of application 580 of FIG. In a further example, system 300 of FIG. 3 may implement illustrative process 800 .

At step 802, an application identifies an entity for which information is stored among a plurality of entities. In some embodiments, the application selects entities based on a predetermined order. For example, an application may select entities in alphabetical order and perform part of process 800 . In some embodiments, an application identifies an entity when metadata about the entity is created. For example, an application may identify an entity as it is added to a database (eg, a database of entities). In some embodiments, the application identifies entities when a search operation misidentifies an entity and thus an alternative representation may be desired to prevent further misidentification. In some embodiments, the application identifies entities based on user input. For example, a user may indicate to an application entities based on incorrect search results, unreachable entities, or errors observed in search results (eg, in a preferred user interface). In some embodiments, the application need not identify entities in response to errors or predetermined order in the search results. For example, the application may randomly select an entity in the entity database and proceed to step 804 . In some embodiments, an application may identify entities based on their popularity in search queries. For example, greater search effectiveness may be achieved by determining alternative representations for more general entities so that more search queries are answered correctly. In a further example, an application may identify less common or even more ambiguous entities and prevent unreachability of those entities because very few search queries may define these entities. Applications may apply any suitable criteria to determine which entities to identify. In some embodiments, the application may identify more than one entity in step 802, and steps 804-810 may therefore be performed for each identified entity. In some embodiments, an application may identify content items instead of or in addition to entities. For example, an application may identify an entity such as a movie, then identify all other entities of interest associated with that entity and undergo steps 804-810.

At step 804, the application generates an audio file based on the first text string and at least one speech criterion. A first text string describes the entity identified in step 802 . For example, as illustrated in FIG. 3, an application may include a text-to-speech engine 310, which may be configured to generate audio files. An application may generate audio output from a speaker or other suitable sound producing device that may be detected by a microphone or other suitable detection device. An application may apply one or more settings or speech criteria in generating and outputting audio files. For example, aspects of the generated "sound" may be adjusted or otherwise selected based on any suitable criteria. In some embodiments, at least one speech criterion includes pronunciation settings (eg, how one or more syllables, groups of letters, or words are pronounced or phonemes to be used). In some embodiments, at least one speech criterion includes language settings (eg, defining language, accent, regional accent, or other language information).

In an illustrative example involving multiple speech criteria, the application generates respective audio files based on the first text string and respective speech criteria, and generates respective second audio files based on the respective audio files. may generate a text string, compare each second text string to the first text string, and store each second text string if not identical to the first text string ( in the metadata associated with the entity).

In the illustrative example, the application converts a first text string into a first audio signal, generates speech on a speaker based on the audio signal, detects the speech using a microphone, and generates a second audio signal. audio signals, process the audio signals, and generate audio files. In some embodiments, the application generates speech at the speaker based on at least one speech setting of the text-to-speech module.

At step 806, the application generates a second text string based on the audio file. The second text string should match the first text string and describe the entity identified in step 802, apart from possible differences from text-to-speech or speech-to-text conversion. For example, as illustrated in FIG. 3, an application may include a speech-to-text engine 320, which receives audio input or its generated files and translates the audio into a transcription record (e.g., text string). can be configured to convert. Applications may receive audio input at a microphone or other suitable sound detection device. An application may apply one or more settings in receiving, conditioning, and converting audio files to text. For example, aspects of adjusting and transforming the detected "voice" may be adjusted or otherwise selected based on any suitable criteria.

In the illustrative example, the application generates a playback of an audio file on a speaker, uses a microphone to detect the playback, generates an audio signal, identifies one or more words, and classifies the audio signal. 2 to a text string. In some embodiments, the application converts the audio signal into a second text string based on at least one text setting of the Speech to Text module.

At step 808, the application compares the second text string to the first text string. In some embodiments, the application compares each character of the first and second text strings to determine a match. In some embodiments, the application determines the degree to which the first text string and the second text string match (e.g., the percentage of text strings that match, the number of differences that exist, whether they match, or number of keywords that do not match). The application may use any suitable technique to determine whether the first and second text strings are the same, similar, or different, and the degree to which they are similar or different.

At step 810, the application stores the second text string if it is not identical to the first text string. In some embodiments, the application stores the second text string in metadata associated with the entity. In some embodiments, step 810 includes the application updating existing metadata based on one or more text queries. For example, as queries are answered and search results are evaluated, the application may update metadata to reflect new learnings. If the second text string is determined to be identical to the first text string, no new information is obtained by storing the second text string. However, an indication of the comparison of step 808 may be stored in metadata to increase confidence in the reachability of the entity via voice queries. For example, if the second text string is identical to the first text string, it may serve to validate existing metadata for speech-based queries.

FIG. 9 depicts a flowchart of an illustrative process 900 for retrieving content associated with an entity of a voice query, according to some embodiments of the present disclosure. For example, the query application may use user device 400 of FIG. 4, user equipment system 401 of FIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, any other suitable device, or any combination thereof, etc. process 900 may be implemented on any suitable hardware. In a further example, the query application may be an instance of application 560 of FIG.

At step 902, a query application receives an audio signal at an audio interface. The system may include a microphone or other audio detection device and may record audio files based on audio input to the device.

At step 904, the query application analyzes the audio signal of step 902 to identify speech. The query application may be any suitable decimation, conditioning (e.g., amplification, filtering), processing (e.g., in the time or spectral domain), pattern recognition, algorithm, transformation, any other suitable action, or any of them. A combination may be applied. In some embodiments, the query application identifies words, sounds, phrases, or combinations thereof using any suitable technique.

At step 906, the query application determines whether a voice query has been received. In some embodiments, the query application determines that a voice query has been received based on parameters of the audio signal. For example, periods without speech before and after the query may delimit the range of voice queries in the recording. In some embodiments, the query application identifies keywords in the order in which they were spoken, applies sentences or query templates to the keywords, and extracts the text query. For example, the placement of nouns, proper nouns, verbs, adjectives, adverbs, and other parts of speech can provide indications of the beginning and end of a spoken query. The query application may apply any suitable criteria when analyzing the audio signal to extract the text. At step 908 the query application generates a text query based on the results of steps 904 and 906 . In some embodiments, at step 908, the query application may store the text query in a suitable storage device (eg, storage device 408). In step 906, if the query application determines that a voice query has not been received or otherwise a text query cannot be generated based on the parsed audio of step 904, the query application , may return to step 902 and proceed to detect audio until a voice query is received.

At step 910, a query application accesses a database of entity information. The query application uses the text query of step 908 to search through the information in the database. A query application may apply any suitable search algorithm to identify information, entities, or content in the database.

At step 912 the query application determines whether the entity in the database of step 910 matches the text query of step 908 . A query application can identify and evaluate multiple entities to find matches. In some embodiments, the text query includes two or more entities, and the query application searches through the content to determine content items that have entities associated in the metadata (e.g., the text query and content item metadata tags). In some situations, the query application may be unable to identify a match and in response may choose to continue the search, search in another database, modify the text query ( For example, returning to step 908 (not shown); returning to step 904 to modify the settings used in step 904 (not shown); returning an indication that no search results were found; Any other suitable response may be made, or any combination thereof. In some embodiments, the query application may identify multiple entities, content, or both that match the text query. Step 914 includes the query application identifying content associated with the text query of step 908 . In some embodiments, steps 914 and 910 may be reversed and the query application may search through content based on text queries. In some embodiments, the entity may include a content identifier, so steps 910 and 914 may be combined.

At step 916 the query application retrieves the content associated with the text query of step 908 . At step 916, for example, the query application identifies content items, downloads content items, streams content items, content items or lists of content items for display (e.g., or directs to content items). list of links), or a combination thereof.

The above-described embodiments of the present disclosure are presented for purposes of illustration, not limitation, and the present disclosure is limited only by the claims that follow. Moreover, features and limitations described in any one embodiment may apply to any other embodiment herein, and flowcharts or examples relating to one embodiment may be used in any other implementation in a suitable manner. Note that forms may be combined, performed in a different order, or performed in parallel. Additionally, the systems and methods described herein can be implemented in real time. It should also be noted that the systems and/or methods described above may be applied to or used in accordance with other systems and/or methods.
This specification discloses embodiments including, but not limited to:
(Item 1) A method of responding to a voice query, the method comprising:
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query using control circuitry;
determining phonetic information for one or more keywords using a control circuit;
generating a text query based on one or more keywords and phonetic information using control circuitry;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
retrieving a content item associated with the entity;
A method, including
2. The method of claim 1, wherein the phonetic information comprises one phoneme of the one or more keywords.
3. The method of claim 1, wherein identifying the entity is further based on user profile information.
4. The method of claim 3, wherein identifying entities is based on previously identified entities from previous voice queries.
5. The method of claim 1, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 6) Identifying an entity is
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tag to the text query;
selecting an entity by determining a maximum score;
The method of item 1, comprising
(Item 7) the entity is a first entity, further comprising identifying a second entity in the plurality of entities based on the text query and second metadata about the second entity; The method of item 1, wherein the item is associated with a first entity and a second entity.
(Item 8) The method of item 1, wherein identifying an entity among the plurality of entities of the database includes comparing at least a portion of the text query to tags of stored metadata to identify a match. Method.
9. The method of claim 1, wherein a first of the one or more keywords is associated with two or more pronunciations of the first keyword.
10. The method of claim 1, wherein the phonetic information comprises a phoneme representation of a first one of the one or more keywords.
(Item 11) A system for responding to voice queries, the system comprising:
an audio interface for receiving voice queries;
a control circuit coupled to the audio interface;
The control circuit
extracting one or more keywords from the spoken query;
deterministically extracting phonetic information for one or more keywords;
generating and extracting a text query based on one or more keywords and pronunciation information;
identifying and extracting an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
A system configured to: retrieve content items associated with an entity.
12. The system of claim 11, wherein the phonetic information comprises one phoneme of the one or more keywords.
13. The system of claim 11, wherein the control circuitry is further configured to identify entities based on user profile information.
14. The system of claim 13, wherein the control circuitry is further configured to identify entities based on previously identified entities from previous voice queries.
15. The system of claim 11, wherein the control circuitry is further configured to identify the entity based on popularity information associated with the entity.
(Item 16) The control circuit is
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tag to the text query;
12. The system of item 11, further configured to identify the entity by selecting the entity by determining the maximum score.
(Item 17) the entity is a first entity, and the control circuit identifies a second entity in the plurality of entities based on the text query and second metadata about the second entity; 12. The system of item 11, further configured, wherein the content item is associated with the first entity and the second entity.
(Item 18) The control circuitry is further configured to identify an entity among the plurality of entities of the database by comparing at least a portion of the text query to stored metadata tags and identifying matches. of item 11.
19. The system of claim 11, wherein a first one of the one or more keywords is associated with two or more pronunciations of the first keyword.
20. The system of claim 11, wherein the phonetic information comprises a phoneme representation of a first one of the one or more keywords.
(Item 21) A non-transitory computer-readable medium having encoded instructions, the instructions, when executed by a control circuit,
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query;
determining phonetic information for one or more keywords;
generating a text query based on one or more keywords and phonetic information;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
A non-transitory computer-readable medium that causes control circuitry to read content items associated with an entity.
22. The non-transitory computer-readable medium of claim 21, wherein the phonetic information comprises one phoneme of the one or more keywords.
23. The non-transitory computer readable of claim 21 further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify the entity based on the user profile information. medium.
(Item 24) Further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify entities based on previously identified entities from previous voice queries, item 23. 3. The non-transitory computer-readable medium of .
25. The method of claim 21 further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify the entity based on popularity information associated with the entity. transient computer readable medium.
(Item 26) further comprising encoded instructions, the instructions, when executed by the control circuit,
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tag to the text query;
22. The non-transitory computer readable medium of item 21, causing the control circuit to identify the entity by selecting the entity by determining the maximum score.
(Item 27) The entity is the first entity and further comprises encoded instructions which, when executed by the control circuit, provide the control circuit with the text query and second metadata about the second entity. 22. The non-transitory computer-readable according to item 21, wherein a second entity among the plurality of entities is identified based on the content item is associated with the first entity and the second entity medium.
(Item 28) Further comprising encoded instructions which, when executed by the control circuitry, cause the control circuitry to compare at least a portion of the text query to tags of stored metadata to identify matches. 22. The non-transitory computer-readable medium of item 21 that identifies an entity in a plurality of entities of a database by.
(Item 29) The non-transitory computer-readable medium of item 21, wherein a first keyword of the one or more keywords is associated with two or more pronunciations of the first keyword.
30. The non-transitory computer-readable medium of claim 21, wherein the phonetic information comprises a phoneme representation of the first of the one or more keywords.
(Item 31) A system for responding to voice queries, the system comprising:
means for receiving voice queries;
means for extracting one or more keywords from a spoken query;
means for determining pronunciation information for one or more keywords;
means for generating a text query based on one or more keywords and pronunciation information;
means for identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and means for retrieving content items associated with the entity.
32. The system of claim 31, wherein the phonetic information comprises one phoneme of the one or more keywords.
33. The system of claim 31, wherein means for identifying an entity comprises means for identifying an entity based on user profile information.
34. The system of claim 33, wherein means for identifying entities comprises means for identifying entities based on previously identified entities from previous voice queries.
35. The system of claim 31, wherein means for identifying an entity comprises means for identifying an entity based on popularity information associated with the entity.
(Item 36) The means for identifying an entity is
means for identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
means for determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tag to the text query;
and means for selecting an entity by determining a maximum score.
(Item 37) the entity is a first entity and further comprising means for identifying a second entity among the plurality of entities based on the text query and second metadata about the second entity; 32. The system of item 31, wherein the item is associated with a first entity and a second entity.
(Item 38) The means for identifying an entity among the plurality of entities of the database comprises means for comparing at least a portion of the text query to stored metadata tags to identify a match. System as described.
39. The system of claim 31, wherein a first of the one or more keywords is associated with two or more pronunciations of the first keyword.
40. The system of claim 31, wherein the phonetic information comprises a phoneme representation of a first one of the one or more keywords.
(Item 41) A method of responding to a voice query, the method comprising:
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query using control circuitry;
determining phonetic information for one or more keywords using a control circuit;
generating a text query based on one or more keywords and phonetic information using control circuitry;
identifying an entity among a plurality of entities in a database based on a text query and stored metadata about the entity, the metadata comprising a pronunciation tag;
and retrieving a content item associated with the entity.
42. The method of claim 41, wherein the phonetic information comprises one phoneme of the one or more keywords.
(Item 43) The method of any of items 41-42, wherein identifying the entity is further based on user profile information.
(Item 44) The method of any of items 41-43, wherein identifying the entity is based on previously identified entities from previous speech queries.
(Item 45) The method of any of items 41-44, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 46) Identifying an entity is
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective pronunciation tag to the text query;
46. A method according to any of items 41-45, comprising selecting an entity by determining a maximum score.
(Item 47) the entity is the first entity and further comprising identifying a second entity in the plurality of entities based on the text query and second metadata about the second entity; 47. The method of any of items 41-46, wherein the items are associated with a first entity and a second entity.
(Item 48) Identifying an entity among the plurality of entities of the database includes comparing at least a portion of the text query to tags of stored metadata to identify a match. Any method described.
49. The method of any of items 41-48, wherein a first of the one or more keywords is associated with two or more pronunciations of the first keyword.
50. The method of any of items 41-49, wherein the phonetic information comprises a phoneme representation of a first one of the one or more keywords.
(Item 51) A method of responding to a voice query, the method comprising:
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query using control circuitry;
generating a text query based on one or more keywords using control circuitry;
identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity based on the pronunciation of an identifier associated with the entity; ,
and retrieving a content item associated with the entity.
52. The method of claim 51, wherein the one or more alternative textual representations comprise phonemic representations of the entity.
53. The method of claim 51, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of entities.
54. The method of claim 51, wherein the one or more alternative textual representations of the entity comprise text strings generated based on previous speech-to-text conversions.
(Item 55) The one or more alternative text representations comprises a plurality of alternative text representations, each alternative text representation of the plurality of alternate text representations comprising:
converting the first textual representation into an audio file;
52. Method according to item 51, produced by converting an audio file into a second textual representation, the second textual representation being not identical to the first textual representation.
(Item 56) The method of item 51, wherein identifying the entity is further based on user profile information.
(Item 57) The method of item 51, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 58) Identifying the entity is
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
52. The method of item 51, comprising selecting an entity by determining a maximum score.
(Item 59) further comprising generating a plurality of text queries, the plurality of text queries comprising the text queries, each text query of the plurality of text queries corresponding to a respective setting of the control circuit's utterance->text module; 52. The method of item 51, wherein the method is generated based on
(Item 60)
identifying each entity based on each text query of the plurality of text queries;
determining a respective score for each entity based on comparison of each text query to metadata associated with each entity;
60. The method of item 59, further comprising identifying the entity by selecting the maximum score of each score.
(Item 61) A system for responding to voice queries, the system comprising:
an audio interface for receiving voice queries;
a control circuit;
The control circuit
extracting one or more keywords from the spoken query;
generating a text query based on one or more keywords;
identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity based on the pronunciation of an identifier associated with the entity; ,
A system configured to: retrieve content items associated with an entity.
62. The system of claim 61, wherein the one or more alternative textual representations comprise phonemic representations of entities.
63. The system of claim 61, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of entities.
(Item 64) The system of Item 61, wherein the one or more alternative textual representations of the entity comprise text strings generated based on previous speech-to-text conversions.
(Item 65) The one or more alternative text representations comprises a plurality of alternative text representations, the control circuit comprising:
converting the first textual representation into an audio file;
converting the audio file to a second textual representation, the second textual representation not identical to the first textual representation, thereby converting each alternate textual representation of the plurality of alternate textual representations to 62. The system of item 61, configured to generate.
66. The system of claim 61, wherein the control circuitry is further configured to identify entities based on user profile information.
67. The system of claim 61, wherein the control circuitry is further configured to identify the entity based on popularity information associated with the entity.
(Item 68) The control circuit
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
62. The system of item 61, further configured to identify an entity by determining a maximum score, selecting the entity, and identifying the entity.
(Item 69) The control circuit is further configured to generate a plurality of text queries, the plurality of text queries comprising the text queries, the control circuit comprising an utterance to text module, 62. The system according to item 61, wherein each text query is generated based on a respective set of speech to text modules.
(Item 70) The control circuit
identifying each entity based on each text query of the plurality of text queries;
determining a respective score for each entity based on comparison of each text query to metadata associated with each entity;
70. The system of item 69, further configured to: identify the entity by selecting the maximum score of the respective scores.
(Item 71) A non-transitory computer-readable medium having encoded instructions, the instructions, when executed by a control circuit,
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query;
generating a text query based on one or more keywords;
identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity based on the pronunciation of an identifier associated with the entity; ,
A non-transitory computer-readable medium that causes control circuitry to read content items associated with an entity.
(Item 72) The non-transitory computer-readable medium of Item 71, wherein the one or more alternative textual representations comprise phonemic representations of the entity.
(Item 73) The non-transitory computer-readable medium of Item 71, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of the entity.
(Item 74) The non-transitory computer-readable medium of Item 71, wherein the one or more alternative textual representations of the entity comprise text strings generated based on previous speech-to-text conversions.
(Item 75) The one or more alternative textual representations comprises a plurality of alternative textual representations and further comprises encoded instructions which, when executed by the control circuitry, cause the control circuitry to:
converting the first textual representation into an audio file;
converting the audio file to a second textual representation, the second textual representation not identical to the first textual representation, thereby converting each alternate textual representation of the plurality of alternate textual representations to 72. The non-transitory computer readable medium of item 71 that is generated.
76. The non-transitory computer-readable of claim 71 further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify the entity based on the user profile information. possible medium.
77. The method of claim 71 further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to identify the entity based on popularity information associated with the entity. transient computer readable medium.
(Item 78) Further comprising encoded instructions, the instructions, when executed by the control circuit,
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
selecting an entity by determining a maximum score;
72. The non-transitory computer readable medium of item 71 that causes the control circuit to identify the entity by.
(Item 79) Further comprising encoded instructions, the instructions, when executed by the control circuit, causing the control circuit to generate a plurality of text queries, the plurality of text queries comprising the text queries, the plurality of text queries. 72. The non-transitory computer-readable medium of item 71, wherein each text query of the control circuit is generated based on a respective setting of the speech-to-text module of the control circuit.
(Item 80) further comprising encoded instructions, the instructions, when executed by the control circuit,
identifying each entity based on each text query of the plurality of text queries;
determining a respective score for each entity based on comparison of each text query to metadata associated with each entity;
80. The non-transitory computer-readable medium of item 79, causing the control circuit to identify the entity by selecting the maximum score of the respective scores.
(Item 81) A system for responding to voice queries, the system comprising:
means for receiving voice queries at an audio interface;
means for extracting one or more keywords from a spoken query;
means for generating a text query based on one or more keywords;
means for identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity based on pronunciation of an identifier associated with the entity; ,
and means for retrieving content items associated with the entity.
(Item 82) The system of item 81, wherein the one or more alternative textual representations comprise phonemic representations of the entity.
83. The system of claim 81, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of entities.
(Item 84) The system of Item 81, wherein the one or more alternative textual representations of the entity comprise text strings generated based on previous speech-to-text conversions.
(Item 85) The one or more alternative text representations comprises a plurality of alternative text representations, each alternative text representation of the plurality of alternate text representations comprising:
means for converting the first textual representation into an audio file;
82. System according to item 81, produced by means for converting an audio file into a second textual representation, wherein the second textual representation is not identical to the first textual representation.
86. The system of claim 81, wherein means for identifying an entity further comprises means for identifying an entity based on user profile information.
87. The system of claim 81, wherein means for identifying an entity further comprises means for identifying the entity based on popularity information associated with the entity.
(Item 88) The means for identifying an entity is
means for identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
means for determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
82. The system of item 81, comprising means for selecting entities by determining maximum scores.
(Item 89) further comprising means for generating a plurality of text queries, the plurality of text queries comprising the text queries, each text query of the plurality of text queries corresponding to the utterance of the control circuit→the respective setting of the text module; 82. The system of item 81, wherein the system is generated based on:
(Item 90)
means for identifying respective entities based on respective text queries of the plurality of text queries;
means for determining a respective score for each entity based on comparison of each text query to metadata associated with each entity;
and means for identifying entities by selecting the maximum score of each score.
(Item 91) A method of responding to a voice query, the method comprising:
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query using control circuitry;
generating a text query based on one or more keywords using control circuitry;
identifying an entity based on a text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity based on the pronunciation of an identifier associated with the entity; ,
and retrieving a content item associated with the entity.
92. The method of claim 91, wherein the one or more alternative textual representations comprise phonemic representations of the entity.
(Item 93) The method of any of items 91-92, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of entities.
(Item 94) The method of any of items 91-93, wherein the one or more alternative textual representations of the entity comprise text strings generated based on previous speech-to-text conversions.
(Item 95) The one or more alternative text representations comprises a plurality of alternative text representations, each alternative text representation of the plurality of alternate text representations comprising:
converting the first textual representation into an audio file;
95. Any of paragraphs 91-94, produced by converting an audio file into a second textual representation, wherein the second textual representation is not identical to the first textual representation. Method.
(Item 96) The method of any of items 91-95, wherein identifying the entity is further based on user profile information.
(Item 97) The method of any of items 91-96, wherein identifying the entity is further based on popularity information associated with the entity.
(Item 98) Identifying the entity is
identifying a plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
selecting an entity by determining a maximum score.
(Item 99) further comprising generating a plurality of text queries, the plurality of text queries comprising the text queries, each text query of the plurality of text queries corresponding to a respective setting of the control circuit's utterance->text module; 99. A method according to any of items 91-98, wherein the method is generated based on
(Item 100)
identifying each entity based on each text query of the plurality of text queries;
determining a respective score for each entity based on comparison of each text query to metadata associated with each entity;
100. The method of item 99, further comprising identifying the entity by selecting the maximum score of each score.
(Item 101) A method of generating entity metadata for a voice query, the method comprising:
identifying an entity for which information is stored among a plurality of entities;
generating an audio file using a text-to-speech module based on a first text string and at least one speech criterion, wherein the first text string describes an entity; ,
generating a second text string based on the audio file using the Speech→Text module;
comparing the second text string to the first text string;
If not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 102) The method of item 101, wherein the at least one speech criterion comprises a pronunciation setting.
(Item 103) The method of item 101, wherein the at least one speech criterion comprises a language setting.
(Item 104) The at least one speech criterion comprises a plurality of speech criteria, the method comprising:
generating respective audio files based on the first text string and respective speech criteria using a Text to Speech module;
generating respective second text strings based on respective audio files using the Speech to Text module;
comparing each second text string to the first text string;
102. The method of item 101, further comprising, if not identical to the first text string, storing each second text string in metadata associated with the entity.
(Item 105) The method of item 101, further comprising updating metadata based on one or more text queries.
(Item 106) The method of item 101, further comprising storing the phoneme representation of the first text string in metadata associated with the entity.
(Item 107) Generating an audio file based on the first text string includes:
converting the first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
detecting speech using a microphone to generate a second audio signal;
102. The method of item 101, comprising processing an audio signal to generate an audio file.
(Item 108) The method of item 107, wherein generating the speech at the speaker is further based on at least one speech setting of the text-to-speech module.
(Item 109) Generating a second text string based on the audio file includes:
producing playback of the audio file on the speaker;
detecting playback and generating an audio signal using a microphone;
102. The method of item 101, comprising converting the audio signal into a second text string by identifying one or more words.
(Item 110) The method of item 109, wherein converting the audio signal to the second text string is based on at least one text setting of the Speech to Text module.
(Item 111) A system for generating entity metadata for a spoken query, the system comprising a control circuit;
The control circuit
identifying an entity for which information is stored among a plurality of entities;
generating an audio file, using an audio interface coupled to the control circuit, based on a first text string and at least one speech criterion, the first text string describing an entity; to do and
generating a second text string based on the audio file using the audio interface;
comparing the second text string to the first text string;
If not identical to the first text string, store the second text string in metadata associated with the entity.
(Item 112) The system of item 111, wherein the at least one speech criterion comprises a pronunciation setting.
(Item 113) The system of item 111, wherein the at least one speech criterion comprises a language setting.
(Item 114) The at least one speech reference comprises a plurality of speech references, the control circuit comprising:
using audio equipment to generate respective audio files based on the first text string and respective speech criteria;
generating respective second text strings based on respective audio files using audio equipment;
comparing each second text string to the first text string;
If not identical to the first text string, storing each second text string in metadata associated with the entity.
(Item 115) The system of item 111, wherein the control circuitry is further configured to update the metadata based on one or more text queries.
(Item 116) The system of item 111, wherein the control circuitry is further configured to store the phoneme representation of the first text string in metadata associated with the entity.
(Item 117) The audio device includes a speaker and a microphone, and the control circuit includes:
converting the first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
detecting speech using a microphone to generate a second audio signal;
112. The system of item 111, further configured to generate an audio file based on the first text string by processing the audio signal and generating the audio file.
118. The system of claim 117, wherein the control circuit is further configured to generate speech at the speaker based on the at least one speech setting.
(Item 119) The audio equipment includes a speaker and a microphone, and the control circuit includes:
producing playback of the audio file on the speaker;
detecting the playback at the microphone and generating an audio signal;
further configured to generate a second text string based on the audio file by converting the audio signal into a second text string by identifying one or more words; 111. The system of item 111.
(Item 120) The system of item 119, wherein the control circuit is further configured to convert the audio signal to the second text string based on at least one text setting of the speech-to-text module.
(Item 121) A non-transitory computer-readable medium having encoded instructions that, when executed by a control circuit,
identifying an entity for which information is stored among a plurality of entities;
generating an audio file based on a first text string and at least one speech criterion, wherein the first text string describes an entity;
generating a second text string based on the audio file;
comparing the second text string to the first text string;
and storing the second text string in metadata associated with the entity if it is not identical to the first text string.
(Item 122) The non-transitory computer-readable medium of Item 121, wherein the at least one speech criterion comprises pronunciation settings.
(Item 123) The non-transitory computer-readable medium of item 121, wherein the at least one speech criterion comprises a language setting.
(Item 124) The at least one speech criterion comprises a plurality of speech criteria and further comprises encoded instructions, the instructions, when executed by the control circuit,
generating respective audio files based on the first text string and respective speech criteria;
generating respective second text strings based on respective audio files;
comparing each second text string to the first text string;
122. The non-transitory computer reading of clause 121, causing the control circuit to store each second text string in metadata associated with the entity if not identical to the first text string. possible medium.
(Item 125) The non-uniformity of item 121 further comprising encoded instructions that, when executed by the control circuitry, cause the control circuitry to update the metadata based on the one or more text queries. transient computer readable medium.
(Item 126) Item 121 further comprising encoded instructions that, when executed by the control circuit, cause the control circuit to store the phoneme representation of the first text string in metadata associated with the entity. 3. The non-transitory computer-readable medium of .
(Item 127) Further comprising encoded instructions, the instructions, when executed by the control circuit,
converting the first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
detecting speech using a microphone to generate a second audio signal;
122. The non-transitory computer-readable medium of item 121 that causes the control circuit to process the audio signal and generate the audio file.
(Item 128) Further comprising encoded instructions which, when executed by the control circuit, cause the control circuit to generate speech at the speaker based on at least one speech setting of the text-to-speech module. 3. The non-transitory computer-readable medium of .
(Item 129) Further comprising encoded instructions, the instructions, when executed by the control circuit,
producing playback of the audio file on the speaker;
detecting playback and generating an audio signal using a microphone;
122. The non-transitory computer-readable medium of item 121, causing the control circuit to convert the audio signal into the second text string by identifying one or more words.
(Item 130) Further comprising encoded instructions which, when executed by the control circuit, cause the control circuit to convert the audio signal into a second text string based on at least one text setting of the Speech->Text module. 130. The non-transitory computer-readable medium of item 129 that converts to
(Item 131) A system for generating entity metadata for a voice query, the system comprising:
means for identifying an entity of a plurality of entities for which information is stored;
means for generating an audio file based on a first text string and at least one speech criterion, wherein the first text string describes an entity;
means for generating a second text string based on the audio file;
means for comparing the second text string to the first text string;
means for storing the second text string in metadata associated with the entity if it is not identical to the first text string.
(Item 132) The system of item 131, wherein the at least one speech criterion comprises a pronunciation setting.
(Item 133) The system of item 131, wherein the at least one speech criterion comprises a language setting.
(Item 134) The at least one speech criterion comprises a plurality of speech criteria, the system comprising:
means for generating respective audio files based on the first text string and respective speech criteria;
means for generating respective second text strings based on respective audio files;
means for comparing each second text string to the first text string;
132. The system of clause 131, further comprising means for storing each second text string in metadata associated with the entity if not identical to the first text string.
(Item 135) The system of item 131, further comprising means for updating metadata based on one or more text queries.
(Item 136) The system of item 131, further comprising means for storing the phoneme representation of the first text string in metadata associated with the entity.
(Item 137) The means for generating an audio file based on the first text string comprises:
means for converting the first text string into a first audio signal;
means for generating speech at a speaker based on the audio signal;
means for detecting speech and generating a second audio signal using a microphone;
and means for processing audio signals and generating audio files.
138. The system of claim 137, wherein the means for generating speech at the speaker further comprises means for generating speech at the speaker based on at least one speech setting of the text-to-speech module.
(Item 139) The means for generating a second text string based on the audio file comprises:
means for generating playback of an audio file on a speaker;
means for detecting playback and generating an audio signal using a microphone;
means for converting the audio signal into a second text string by identifying one or more words.
(Item 140) The means for converting the audio signal to the second text string comprises means for converting the audio signal to the second text string based on at least one text setting of the speech to text module. , item 139.
(Item 141) A method of generating entity metadata for a voice query, the method comprising:
identifying an entity for which information is stored among a plurality of entities;
generating an audio file using a text-to-speech module based on a first text string and at least one speech criterion, wherein the first text string describes an entity; ,
generating a second text string based on the audio file using the Speech→Text module;
comparing the second text string to the first text string;
If not identical to the first text string, storing the second text string in metadata associated with the entity.
(Item 142) The method of item 141, wherein the at least one speech criterion comprises a pronunciation setting.
(Item 143) The method of any of items 141-142, wherein the at least one speech criterion comprises a language setting.
(Item 144) The at least one speech criterion comprises a plurality of speech criteria, the method comprising:
generating respective audio files based on the first text string and respective speech criteria using a Text to Speech module;
generating respective second text strings based on respective audio files using the Speech to Text module;
comparing each second text string to the first text string;
144. The method of any of items 141-143, further comprising, if not identical to the first text string, storing each second text string in metadata associated with the entity.
(Item 145) The method of any of items 141-144, further comprising updating metadata based on one or more text queries.
(Item 146) The method of any of items 141-145, further comprising storing the phoneme representation of the first text string in metadata associated with the entity.
(Item 147) Generating an audio file based on the first text string includes:
converting the first text string into a first audio signal;
generating speech at a speaker based on the audio signal;
detecting speech using a microphone to generate a second audio signal;
147. A method according to any of items 141-146, comprising processing an audio signal and generating an audio file.
(Item 148) The method of item 147, wherein generating the speech at the speaker is further based on at least one speech setting of the text-to-speech module.
(Item 149) Generating a second text string based on the audio file includes:
producing playback of the audio file on the speaker;
detecting playback and generating an audio signal using a microphone;
converting the audio signal into a second text string by identifying one or more words.
(Item 150) The method of item 149, wherein converting the audio signal to the second text string is based on at least one text setting of the Speech to Text module.

Claims (13)

A method of responding to voice queries, the method comprising:
receiving a voice query at an audio interface;
extracting one or more keywords from the spoken query using control circuitry;
generating a text query based on the one or more keywords using the control circuit;
identifying an entity, wherein identifying the entity is based on the text query and metadata about the entity, the metadata comprising one or more alternative textual representations of the entity; said alternative text representation is based on the pronunciation of an identifier associated with said entity;
and retrieving a content item associated with said entity. 2. The method of claim 1, wherein the one or more alternative textual representations comprise phonemic representations of the entity. 3. The method of any of claims 1 and 2, wherein the one or more alternative textual representations comprise phonetic-based alternative spellings of the entity. The method of any of claims 1-3, wherein said one or more alternative textual representations of said entity comprise text strings generated based on previous speech-to-text conversions. The one or more alternative text representations comprise a plurality of alternative text representations, each alternative text representation of the plurality of alternate text representations comprising:
converting the first textual representation into an audio file;
converting the audio file into a second textual representation; and
A method according to any of claims 1-4, wherein said second textual representation is not identical to said first textual representation. The method of any of claims 1-5, wherein identifying the entity is further based on user profile information. The method of any of claims 1-6, wherein identifying the entity is further based on popularity information associated with the entity. Identifying the entity includes:
identifying the plurality of entities, wherein respective metadata is stored for each entity of the plurality of entities;
determining a respective score for each respective entity of the plurality of entities based on comparing the respective one or more alternative textual representations to the textual query;
and selecting the entity by determining a maximum score. further comprising generating a plurality of text queries, the plurality of text queries comprising the text query, each text query of the plurality of text queries corresponding to a respective setting of the utterance-to-text module of the control circuit; A method according to any one of claims 1 to 8, which is generated based on identifying respective entities based on respective text queries of the plurality of text queries;
determining a respective score for each entity based on comparing the respective text query to metadata associated with the respective entity;
10. The method of claim 9, further comprising: identifying said entity by selecting the maximum score of said respective scores. A system for responding to voice queries, said system comprising:
memory;
and means for implementing the method steps of any of claims 1-10. A non-transitory computer readable medium having encoded instructions which, when executed by a control circuit, causes the control circuit to perform the method steps of any of claims 1-10. A non-transitory computer-readable medium that enables execution. A system for responding to voice queries, said system comprising means for implementing the method steps of any of claims 1-10.

Images