JP2017513047A - Pronunciation prediction in speech recognition. - Google Patents

Abstract

An automatic speech recognition (ASR) device may be configured to predict the pronunciation of a text identifier (eg, song title, etc.) based on a prediction of one or more source languages of the text identifier. One or more source languages may be determined based on the text identifier. The pronunciation may include a pronunciation of one language, a pronunciation of the second language, and a mixed pronunciation that combines a plurality of languages. Pronunciations may be added to the lexicon and matched against content items (eg, songs) and / or text identifiers. The ASR device may receive an oral utterance from a user requesting an ASR device to access the content item. The ASR device determines whether the verbal utterance matches one of the pronunciations of the content item on the lexicon. The ASR device then accesses the content when the verbal utterance matches one of the pronunciations of the previous potential text identifier.

Description

CROSS REFERENCE TO RELATED APPLICATION DATA This application claims priority to US patent application Ser. No. 14 / 196,055 filed Mar. 4, 2014, which is incorporated herein by reference in its entirety.

  Human computer interaction has progressed to a stage where humans can control computing devices by speaking and input to these devices. The computing device utilizes techniques to identify words spoken by human users based on various qualities of received speech input. Such a technique is referred to as speech recognition or automatic speech recognition (ASR). Speech recognition combined with language processing techniques can allow the user to control the computing device and execute tasks based on instructions uttered by the user. Speech recognition may also convert user speech into text data, which may then be provided to various text-based programs and applications.

  Computers, portable terminals, computer systems using telephone lines, kiosks, and various other devices for improving human computer interaction may utilize speech recognition.

  For a more complete understanding of the present disclosure, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:

6 illustrates a speech recognition technique for predicting predicted pronunciation of a foreign word based on the original language of the word according to one aspect of the present disclosure. 1 illustrates a computer network for use of distributed speech recognition according to one aspect of the present disclosure. FIG. 3 is a block diagram conceptually illustrating an apparatus for speech recognition according to one aspect of the present disclosure. Fig. 4 illustrates an audio waveform processed according to one aspect of the present disclosure. Fig. 4 illustrates a speech recognition lattice according to an aspect of the present disclosure. 6 illustrates a speech recognition method for predicting predicted pronunciation of a foreign word based on the original language of the word, according to one aspect of the present disclosure. 6 illustrates a speech recognition method for processing verbal utterances including a text identifier according to one aspect of the present disclosure.

  In an interaction with a device capable of performing automatic speech recognition (ASR), the user may utter instructions to access the content item. These content items may be stored locally on the device or stored remotely, but are accessible by the device. For example, the user may utter a command to the computing device to “play” a particular music item. Verbal commands may be referred to as utterances. Music items may be identified by text identifiers. The text identifier may be text that identifies an item of content such as a song or a video. Exemplary text identifiers include artist names, band names, album names, song names, or some other label that identifies the music being played.

  The ASR system has a lexicon of stored text identifiers (ie, artist name, band name, album name, and song name) that are matched to the corresponding expected pronunciation when the expected pronunciation is based on a text identifier. May be. The lexicon may be stored locally or remotely. When the ASR system receives an utterance, it may match the utterance with the stored expected pronunciation and match the utterance with one or more content items for retrieval. For example, if the user says “Play some AC / DC song”, the system may match the audio corresponding to “AC / DC” with the corresponding expected pronunciation and then the band name. Once the band is identified, the device may then play a song associated with the band.

  Each typical ASR system is associated with a specific language. For example, an English ASR system may be configured to solve English, a German ASR system may be configured to solve German, and so on. Some text identifiers may come from foreign languages that are not the main language of the ASR system. This can lead to confusion when the user attempts to pronounce the text identifier using the linguistic features of the foreign language text identifier. For example, a user who makes an utterance requesting music using the German pronunciation of a German song name or German band name may confuse the English-based ASR system. Similarly, users who utilize English pronunciation of German song titles may also confuse the ASR system because the ASR system may expect different pronunciations based on the song title text.

  A method for determining the expected pronunciation of a text identifier based on the prediction of the original language of the text identifier is presented. The original language may be determined based on the text identifier. In some aspects of the present disclosure, the expected pronunciation of a text identifier may also be based on the pronunciation history of a particular user or user category. Predicted pronunciation includes a combination of predicted pronunciations based on the original language, for example, a specific phoneme of a text identifier that is expected to have one original language and an expected pronunciation that has another phoneme of the text identifier expected to have a different original language May be included. Further, multiple predicted pronunciations may be determined for each text identifier when each predicted pronunciation may be related to the likelihood of occurrence. The likelihood may be based on text identifiers, user behavior, other user behavior, or other factors.

  Predicted pronunciations with different text identifiers may be added to the lexicon to accommodate different pronunciations from different users. Expected pronunciations may be linked to content items such as songs stored on a music catalog. When the computing device receives an oral utterance that includes a text identifier, the computing device determines whether the verbal utterance includes a text identifier by comparing the utterance with a lexicon with a corrected predicted pronunciation. If the verbal utterance matches the expected pronunciation, the computing device operates on the content, eg, by playing the requested song, as indicated in the utterance command portion.

  FIG. 1 illustrates a speech recognition technique for predicting expected pronunciation of a text identifier based on the original language of the text identifier, according to one aspect of the present disclosure. FIG. 1 includes an ASR device 100 having a predicted pronunciation prediction module 128 and an ASR module 314 and a user 120 located proximal to the ASR device 100. Predictive pronunciation prediction module 128 may be configured to access a text identifier (such as a song title) as indicated at block 102 and determine the original language of the text identifier as indicated at block 104. Prediction module 128 may then determine one or more predicted pronunciations of the text identifier based on the original language, as shown in block 106. Expected pronunciations may be matched to content items (eg, songs) for retrieval by the system. The prediction module 128 may perform these actions in advance when configuring or training the operation of the ASR system prior to receiving the utterance.

  Upon receipt of the verbal utterance of the device shown in block 108, the utterance is transferred to the ASR module 314. The ASR module may then match the utterance with the expected pronunciation as shown at block 110. That expected pronunciation may then be matched to a content item such as a song mentioned in the utterance, as shown in block 112. The device may then access the content item (eg, play a song), as shown at block 114.

  Although FIG. 1 shows a particular task being performed by a particular module, the task may be performed by various modules, as configured by a particular ASR system.

  Further, the techniques described herein may be performed on a local device, such as ASR device 100, a network device, or some combination of different devices. For example, the local device and the remote device may exchange the local device's text identifier with the remote device to actually perform the determination of the original language and the expected pronunciation (s). Further, while the local device may receive voice data including verbal speech, the local device may send the voice data to the remote device for processing. The remote device may then perform voice ASR processing. The ASR result may then be sent to the local device for matching with the content item of utterance and access to the content item, or the result sent to the local device for playback to the remote device and the user ( For example, those tasks may be executed by streaming music. Alternatively, the local device and the remote device may work together in other ways.

  The plurality of ASR devices may be connected via a network. As shown in FIG. 2, a plurality of devices may be connected via a network 202. Network 202 may include a local or private network, or may include a wide area network such as the Internet. The device may be connected to the network 202 through either a wired or wireless connection. For example, the wireless device 204 may be connected to the network 202 through a wireless service provider. Other devices such as computer 212 may connect to network 202 through a wired connection. For example, other devices such as a refrigerator 218 located in a home or commercial facility may connect to the network 202 through a wired or wireless connection. Other devices such as laptop 208 or tablet computer 210 may be able to connect to network 202 using a variety of connection methods, including via a WiFi service provider, such as a WiFi connection. The network device may input verbal audio through a number of audio input devices, including via a headset 206 or 214 or the like. The voice input device may be connected to the network device through either a wired or wireless connection. The network device may also include an embedded voice input device such as a laptop 208, a wireless device 204 or an internal microphone (not shown) within the tablet computer 210.

  In certain ASR system configurations, one device may capture audio signals and another device may perform ASR processing. For example, audio input to headset 214 may be captured by computer 212 and sent to computer 220 or server 216 via network 202 for processing. Alternatively, computer 212 may partially process the audio signal before transmitting it over network 202. Because ASR processing may use a lot of computational resources, both the storage and processing power when the processing capability of the device that captures voice is lower than that of the remote device and a higher quality ASR result is desired Such a divided configuration may be used. Audio capture may occur near the captured audio signal that is sent to the user and other devices for processing. For example, one or more microphone arrays may be located at a different location than the ASR device, and captured audio may be transmitted from the array to the ASR device (or device) for processing.

  FIG. 3 shows an automatic speech recognition (ASR) device 302 for performing speech recognition. Aspects of the present disclosure include computer readable and computer executable instructions that may reside on the ASR device 302. Although FIG. 3 shows some components that may be included within ASR device 302, other components not shown may also be included. Also, some of the illustrated components may not be present in all devices that can utilize aspects of the present disclosure. Further, some components shown in ASR device 302 as a single component may also appear multiple times in a single device. For example, the ASR device 302 may include multiple input devices 306, output devices 307, or multiple control / processing devices 308.

  Multiple ASR devices may be utilized within a single speech recognition system. In such a multi-device system, the ASR device may include different components for performing different aspects of the speech recognition process. The plurality of devices may include overlapping components. The ASR device as shown in FIG. 3 is exemplary and may be a stand-alone device, or some or all of it may be included as a component of a larger device or system.

  The teachings of the present disclosure include, for example, general purpose computing systems, server client computing systems, mainframe computing systems, computing systems utilizing telephone lines, laptop computers, mobile phones, personal digital assistants (PDAs), tablet computers, and other mobile It may be applied in a number of different devices and computer systems, including devices and the like. The ASR device 302 also provides voice recognition functions such as, for example, an automated teller machine (ATM), kiosk, home appliance (refrigerator, oven, etc.), vehicle (car, bus, motorcycle, etc.), and / or exercise equipment. It may be a component of another device or system.

  The ASR device 302 may include an audio capture device 304 that captures verbal utterances for processing, as shown in FIG. The audio capture device 304 may include a microphone or other suitable component for capturing audio. The audio capture device 304 may be integrated into the ASR device 302 or separated from the ASR device 302. The ASR device 302 may also include an address / data bus 324 for carrying data between the components of the ASR device 302. Each component within ASR device 302 may also be connected directly to other components in addition to (or instead of) connecting to other components across bus 324. Although specific components are shown in FIG. 3 as being directly connected, these connections are merely exemplary and other components may be directly connected to each other (the ASR module 314 may be connected to the controller / process). Device 308, etc.).

  The ASR device 302 may include a controller / processor 308, which may be a central processing unit (CPU) for processing data and computer readable instructions and a memory 310 for storing data and instructions. Memory 310 may include volatile random access memory (RAM), non-volatile read only memory (ROM), and / or other types of memory. The ASR device 302 may also include a data storage component 312 for storing data and instructions. Data storage component 312 may include one or more storage types, such as magnetic storage, optical storage, solid state storage, and the like. ASR device 302 may also be connected to removable or external memory and / or storage (removable memory card, memory key drive, network storage, etc.) through input device 306 or output device 307. Computer instructions for processing by the controller / processor 308 that operates the ASR device 302 and its various components are executed by the controller / processor 308, in the memory 310, storage 312, external devices, or below. May be stored in a memory / storage included in the ASR module 314 described in FIG. Alternatively, some or all of the executable instructions may be embedded in hardware or firmware in addition to or instead of software. The teachings of the present disclosure may be implemented in various combinations of software, firmware, and / or hardware, for example.

  The ASR device 302 includes an input device (s) 306 and an output device (s) 307. Various input / output devices may be included in the device. An exemplary input device 306 includes an audio capture device 304, such as a microphone (shown as a separate component), a touch input device, a keyboard, a mouse, a stylus, or other input device. Exemplary output devices 307 include visual displays, tactile displays, audio speakers, headphones, printers or other output devices. Input device 306 and / or output device 307 may also include an interface for external peripheral device connection, such as Universal Serial Bus (USB), FireWire®, Thunderbolt®, or other connection protocols. . The input device 306 and / or the output device 307 may also include a network connection such as an Ethernet port, a modem net, and the like. The input device 306 and / or the output device 307 may also be a wireless communication device such as radio frequency (RF), infrared, Bluetooth, Bluetooth, wireless local area network (WLAN) (such as WiFi), or long term evolution (LTE). A wireless network device such as a wireless device capable of communication using a wireless communication network such as a network, a WiMAX network, or a 3G network may be included. ASR device 302 may connect through input device 306 and / or output device 307 to a network that may include a distributed computing environment, such as the Internet or a private network.

  The device may also include an ASR module 314 for verbal audio data processing to text. The ASR module 314 copies the voice data into text data representing a voice word included in the voice data. The text data may then be utilized by other components for various purposes such as executing system instructions, entering data, etc. Voice data including verbal utterances may be processed in real time or may be stored and processed later. The verbal utterance on the speech data is input to the ASR module 314, which then interprets the utterance based on the similarity between the utterance and the model known to the ASR module 314. For example, the ASR module 314 may compare input speech data with a model for speech (eg, speech units or phonemes) and speech sequences and identify words that match the speech sequence uttered in the speech of speech data. Good. Different ways in which verbal utterances can be interpreted may be assigned probabilities or recognition scores representing the likelihood that each particular set of words matches the set of words uttered in the utterance. The recognition score is, for example, the similarity of speech in speech to a model for language speech (eg, an acoustic model) and a specific word that matches the speech (eg, using a language model or grammar) at a particular location. It may be based on several factors including the possibility of being included in the sentence. Based on the factors considered and the assigned recognition score, the ASR module 314 may output the most probable word recognized in the speech data. The ASR module 314 may also output a plurality of alternative recognized words in the form of a lattice or N-best list (described in more detail below).

  While the recognition score may represent the probability that a portion of the speech data corresponds to a particular phoneme or word, the recognition score may also indicate the ASR processing quality of the speech data with a score relative to the ASR processing of other speech data. May be incorporated. The recognition score may be expressed as a numerical value from 1 to 100 as a probability from 0 to 1, a log probability or other indicator. The recognition score may indicate a relative reliability that a part of the voice data corresponds to a specific phoneme, word, or the like.

  The ASR module 314 includes the bus 324 of the ASR device 302, the input device (s) 306 and / or the output device (s) 307, the audio capture device 304, the encoder / decoder 322, the controller / processing. It may be connected to the device 308 and / or other components. Voice data sent to the ASR module 314 may be captured by the remote entity and may come from the voice capture device 304 or received by the input device 306, such as voice data sent to the ASR device 302 via the network. May be. The audio data may be in the form of a digital representation of an audio waveform of an oral utterance. Other aspects of sampling rate, filtering, and analog-to-digital conversion processing can affect the overall quality of the audio data. Various settings of the audio capture device 304 and the input device 306 may be configured to adjust the audio data based on conventional tradeoffs between quality and data size or other considerations.

  The ASR module 314 includes an acoustic front end (AFE) 316, a speech recognition engine 318, and a speech storage 320. The AFE 316 converts the voice data into data for processing by the voice recognition engine 318. The speech recognition engine 318 compares the speech recognition data with the acoustic, language, and other data models and information stored in the speech storage 320 for recognition of speech contained in the original speech data. AFE 316 and speech recognition engine 318 may include their own controller (s) / processor (s) and memory, or they may be, for example, controllers / processors of ASR device 302. 308 and memory 310 may be used. Similarly, instructions for operation of AFE 316 and speech recognition engine 318 may be within ASR module 314 within memory 310 and / or storage 312 of ASR device 302 or within an external device.

  The received audio data may be sent to the AFE 316 for processing. The AFE 316 may reduce noise in the audio data, identify portions of the audio data that include audio for processing, and divide and process the identified audio components. The AFE 316 may divide the digital audio data into frames or audio segments where each frame represents a time interval of, for example, 10 milliseconds (ms). During a frame, the AFE 316 determines a set of values called a feature vector that represents the feature / quality of the speech portion within the frame. The feature vector may include a variable number of values, such as 40, for example. The feature vector may represent different qualities of the audio data in the frame. FIG. 4 shows a digital audio data waveform 402 having a plurality of points 406 of the first word 404 as the first word 404 is processed. Those point voice qualities may be stored in a feature vector. The feature vectors may flow or be combined into a matrix representing the time of verbal utterance. These feature vector matrices may then be transferred to the speech recognition engine 318 for later processing. Several approaches may be utilized by AFE 316 for processing audio data. Such techniques include the use of mel frequency cepstrum coefficients (MFCC), perceptual linear prediction (PLP) techniques, neural network feature vector techniques, linear discriminant analysis, semi-joint covariance matrices, or other techniques known to those skilled in the art. May be included.

  The processed feature vectors are then output from ASR module 314 and may be sent to output device 307 for the purpose of communicating to other devices for further processing. Feature vectors may be encoded and / or compressed prior to transmission by encoder / decoder 322. The encoder / decoder 322 may be customized for encoding and decoding ASR data such as digital audio data, feature vectors. The encoder / decoder 322 may also be e.g. The non-ASR data of the ASR device 302 may be encoded using a general encoding method such as zip. The functions of encoder / decoder 322 may be in separate components as shown in FIG. 3, or may be performed by, for example, controller / processor 308, ASR module 314, or other components. .

  The speech recognition engine 318 may process the output from the AFE 316 with reference to information stored in the speech storage 320. Alternatively, front-end post-processed data (such as feature vectors) may be received by ASR module 314 from another source other than internal AFE 316. For example, another entity may process the voice data into a feature vector and communicate that information to the ASR device 302 through the input device (s) 306. The feature vector may be encoded to reach ASR device 302, where it may be decoded (eg, by encoder / decoder 322) prior to processing by speech recognition engine 318.

  The speech storage 320 includes various information for speech recognition, such as data that matches phoneme pronunciations with specific words. This data may be referred to as an acoustic model. The voice storage may also include a dictionary of words or lexicons. Voice storage may also include lexicons that match text identifiers with the expected pronunciation of those identifiers. The text identifier may identify digital content such as music on a catalog, content on an address book, and / or other content stored on an ASR device (or elsewhere). The text identifier may also have a name derived from the language (s) that may be different from the ASR system and / or the user's default language (ie, ingredients, dishes, etc.), restaurants, Non-digital items such as events or other items may be identified. Voice storage may also include data indicating words that are easy to use together in a particular context. This data may be referred to as a language or grammar model. Speech storage 320 may also include a training corpus that may include recorded speech and / or corresponding transcriptions that may be utilized to train and improve the models utilized by ASR module 314 in speech recognition. Good. The training corpus may be used to pre-train speech recognition models including acoustic models and language models. The model may then be utilized during ASR processing.

  The training corpus may include a number of sample utterances with associated feature vectors and associated accurate text that may be utilized, for example, to create acoustic and language models. Sample utterances may be utilized to create a mathematical model that corresponds to the expected speech for a particular speech unit. These speech units may include phonemes, syllables, parts of syllables, words, and the like. The speech unit may also include contextual phonemes such as triphones, quinphones. Contextual phonemes regularly used in speech may be associated with its own model. Less common contextual phonemes may be clustered to have a group model. By clustering phonemes in this way, fewer models may be included in the training corpus, thus facilitating ASR processing. The training corpus may include multiple versions of the same utterance from different speakers to provide a comparison of different utterances of the ASR module 314. The training corpus may also include correctly recognized utterances and incorrectly recognized utterances. These incorrectly recognized utterances may include grammatical errors, misrecognition errors, noise, or other errors that provide the ASR module 314 with, for example, error types and corresponding correction examples. The training corpus includes foreign words and the ASR system may be trained to recognize such words. The training corpus may also be adapted to incorporate particular user trends to improve system performance as described below.

  Other information may also be stored in the voice storage 320 for use in voice recognition. The content of the audio storage 320 may be prepared for general ASR usage or may be customized to include audio and words that are accessible in a particular application. For example, for ASR processing in an ATM (Automated Teller Machine), the voice storage 320 may include unique customized data for bank transactions. In some cases, the voice storage 320 may be customized for individual users based on the user's personalized voice input. To improve performance, the ASR module 314 modifies / updates the content of the voice storage 320 based on the feedback of the results of the ASR process, improving speech recognition so that the ASR module 314 exceeds the capabilities provided in the training corpus. May be possible.

  Speech recognition engine 318 attempts to match the received feature vector with a word or subword unit as known in speech storage 320. A subword unit may be a phoneme, a contextual phoneme, a syllable, a part of a syllable, a contextual syllable, or any other such part of a word. The speech recognition engine 318 calculates a recognition score for the feature vector based on the acoustic information and the language information. Acoustic information is used to calculate an acoustic score that represents the likelihood that the intended speech represented by the feature vectors will match a subword unit. To adjust the acoustic score by considering which voices and / or words are used in context, thereby improving the likelihood that the ASR module will output grammatically meaningful speech results Language information is used.

  Speech recognition engine 318 may utilize several techniques for matching feature vectors with phonemes or other phonetic units such as diphones, triphones, and the like. One common technique uses a hidden Markov model (HMM). An HMM is used to determine the probability that a feature vector may match a phoneme. Using the HMM, several states are shown, each of which represents a potential phoneme (or other speech unit such as a triphone) and each state is associated with a model such as a mixed Gaussian model. . Transitions between states may also have an associated probability that represents the likelihood that the current state can be reached from the previous state. Received speech may be represented as a path between HMM states, and multiple paths may represent multiple possible text matches for the same speech. Each phoneme may be represented by a plurality of potential states corresponding to different known pronunciations of the phonemes and their parts (such as the beginning, middle, and end of verbal verbal speech). The initial determination of potential phoneme probabilities may relate to one state. When a new feature vector is processed by the speech recognition engine 318, the state may change or remain the same based on the processing of the new feature vector. A Viterbi algorithm may be utilized to find the most probable sequence of states based on the processed feature vectors.

  Probabilities and states may be calculated using several techniques. For example, the probabilities for each state may be calculated using a Gaussian model based on the feature vector and the content of the audio storage 320, a mixed Gaussian model, or other techniques. Techniques such as maximum likelihood estimation (MLE) may be utilized to estimate the probability of phoneme states.

  In addition to computing a potential state for one phoneme as a potential match with a feature vector, the speech recognition engine 318 also determines a potential state for another phoneme as a potential with a feature vector. It may be calculated as a match. Thus, a plurality of states and state transition probabilities may be calculated.

  The likely state calculated by the speech recognition engine 318 and the likely state transition may be configured in the path. Each path represents a phoneme evolution that potentially matches the speech data represented by the feature vector. One path may overlap with one or more other paths depending on the recognition score calculated for each phoneme. A particular probability is associated with each transition from state to state. A cumulative path score may also be calculated for each path. When combining scores as part of ASR processing, the scores may be multiplied (or otherwise combined) to arrive at the desired combined score, or the probabilities are converted to log areas and processed May be added to assist.

  The speech recognition engine 318 may combine potential paths into lattices that represent speech recognition results. A sample lattice is shown in FIG. Lattice 502 shows multiple potential paths for speech recognition results. Paths between large nodes represent potential words (eg, “hello”, “yello”, etc.), while paths between smaller nodes represent potential phonemes (eg, “H”, “E”, “L”, “ O ”and“ Y ”,“ E ”,“ L ”,“ O ”). For illustrative purposes, individual phonemes are shown only for the first two words of the lattice. The two paths between node 504 and node 506 represent the selection of two potential words, “hello how” or “yello now”. Each path point between nodes (such as potential words) is associated with a recognition score. A recognition score may also be assigned to each path across the lattice. The highest recognition score path, language model score, and / or other factors when the recognition score is a combination of acoustic model scores may be returned by the speech recognition engine 318 as an ASR result for the associated feature vector. .

  Following ASR processing, the ASR result is sent by ASR module 314 to another component of ASR device 302, such as controller / processor 308, for further processing (such as execution of instructions contained in the interpreted text), or It may be transmitted to the output device 307 for transmission to an external device.

  The speech recognition engine 318 may also calculate a path branch score based on a language model or grammar. The language model uses score determination as to which words are easy to use together to form meaningful words and sentences. The application of the language model may improve the possibility that the ASR module 314 correctly interprets the speech included in the speech data. For example, “H E L O” (interpreted as the word “hello”), “H A L O” (interpreted as the word “halo”), based on the language context of each word in the verbal utterance, and To adjust the recognition score of “Y E L O” (interpreted as the word “yello”), the potential of “H E L O”, “H A L O”, and “Y E L O” Acoustic model processing that returns phoneme paths may be adjusted by the language model. The language model may be determined from a training corpus stored in the voice storage 320 and may be customized for a particular application. The language model may be implemented using techniques such as the N-gram model, where the probability of perceiving a particular next word depends on the context history of the previous n-1 words. The N-gram model also has bigrams (n = 2) and trigrams whose probability of perceiving the next word depends on the previous word (in the case of the bigram model) or the previous two words (in the case of the trigram model). It may be configured as a model (where n = 3). The acoustic model may also apply N-gram technology.

  As part of the language model (or at other stages of ASR processing), the speech recognition engine 318 responds to verbal utterances either with a low recognition score according to the language model or for other reasons to conserve computational resources Low recognition score states or paths that are unlikely to do may be removed and discarded. Further, during ASR processing, the speech recognition engine 318 may iteratively perform additional processing passes on the already processed speech portion. In order to refine and improve the results, the later pass may incorporate the results of the previous pass. As the speech recognition engine 318 determines potential words from the input speech, the lattice may become very large so that many potential speeches and words are considered as potential matches with the input speech. Potential matches may be shown as a network of word results. The network of speech recognition results is a connected sequence of possible sequences of speech units that may be recognized and arcs and nodes representing the potential of each sequence. A network of word results is a network of speech recognition results at the word level. Voice recognition networks at other levels are also possible. The resulting network may be generated by any type of speech recognition decoder (or engine). For example, the resulting network may be generated based on a decoder by a finite state transducer (FST). The resulting network may be utilized to create a final set of speech recognition results, such as a lattice or N-best list of best score results. Neural networks may also be utilized to perform ASR processing.

  The speech recognition engine 318 may return an N-best list of routes corresponding to the top N routes along with their respective recognition scores, as determined by the speech recognition engine 318. An application that receives the N-best list (such as a program or component, either internal or external to the ASR device 302) then performs further actions or analysis on the list taking into account the list and associated recognition score. May be. For example, the N-best list may be utilized in error correction and training of various options and processing conditions of the ASR module 314. The speech recognition engine 318 may compare the actual accurate utterance with the best results with other results on the N-best list to determine why the incorrect recognition received a particular recognition score. The voice recognition engine 318 may correct the technique (and update the information in the voice storage 320) to reduce the recognition score of the incorrect technique in subsequent processing attempts.

  An ASR device may be utilized to process voice commands for content items. The content item itself may be stored locally on the ASR device (such as a music collection on a mobile phone) or stored remotely (such as a movie that may be streamed from a remote server). These content items may include, for example, music, electronic books (e-books), movies, contact information, documents, short message service communications, email and / or other audio, video or text information. ASR device users may request access to such content items for a variety of purposes, including playback, editing, forwarding, and the like. For example, the user may request that the mobile phone be able to play music in response to a verbal request from the user. A catalog of content items may be linked to a dictionary of words or a lexicon to fulfill a request from a user. A lexicon may include a text identifier that may be a text identifier linked to an individual content item. For example, the text identifier may include an artist name, album name, song / movie / ebook title, and the like. Each text identifier may correspond to one or more items of content on the catalog (such as band names linked to multiple songs), and each content item may have one or more text identifiers (song names, bands). Name, album name, etc.). Text identifiers may also refer to items other than digital content.

  As described above, the lexicon may also include one or more expected pronunciations for each text identifier, thereby allowing the user to access related content items through voice commands. For example, a user may attempt to play a song stored on a music catalog by speaking an artist name, album or song name. The expected pronunciation may be determined based on the spelling of the word. The process of determining the expected pronunciation of a word based on spelling is defined as grapheme phoneme (G2P) conversion or pronunciation guessing (generally called pronunciation guessing). In some cases, the text identifier may include foreign words. For illustrative purposes, the foreign language (or foreign language) mentioned in this application will be derived from a foreign language relative to the default language of the ASR system. Although the techniques described herein may be applied to ASR systems based on different languages, the default language of the ASR system is shown as English for this purpose.

  To assist ASR processing of text identifiers that incorporate words or linguistic features of different languages, this disclosure allows the ASR system to predict one or more pronunciations of a text identifier based on the language origin of the text identifier. Provide a configured system. In one aspect of the present disclosure, the ASR system determines the original language of the text identifier based on the text identifier. The ASR system then determines the expected pronunciation of the text identifier based on the text and the identified original language. The ASR system may determine multiple expected pronunciations of a particular text identifier that each has a potential to relate to. Expected pronunciations (and / or their associated possibilities) may also be adjusted based on the pronunciation tendency of the user or group of users. The expected pronunciation may be added to the lexicon and linked to its respective content item for final search by the ASR system.

  To determine the original language, the ASR system may utilize a classifier that predicts the origin of the language based on the spelling / text identifier. The classifier may be a statistical model, such as a statistical model based on characters. Since text identifiers (eg, band names) may be short for long forms of text such as documents, paragraphs, etc., the classifier for source language prediction is a paragraph that may be utilized by other language prediction systems. Emphasis may be placed on the basic linguistic unit of text that is shorter than detection based on multiple texts in a row. For example, the classifier may be trained to identify the possibility of a sequence of characters in one or more languages (eg, language A, B, or C). In some aspects, the possibilities for each language may be learned individually. The classifier may also implement an n-gram based character model for words in different languages. The n-gram may be based on a sequence of items such as syllables, letters, words or base pairs according to different configurations of the ASR system.

  A score may be assigned that represents the likelihood that the spelling of the word matches a particular language. For example, a score may be assigned to more than one language from which a text identifier (or portion thereof) is likely to come from. In some aspects, the score may be a probabilistic weight assigned to each of the different languages to improve the identification of the original language. One or more languages with the highest score for the foreign language may be identified as the original language. If the text is “Gotye”, for example, 70% of the probabilistic weights may be assigned to French and 30% to German. Based on this determination, expected pronunciations of both French and German words and corresponding probabilistic weights may be added to the lexicon. This implementation allows the selection of the most probable source language of the text. In one aspect, some of the text identifiers may have different source language scores. For example, the first word of the name “Ludwig van Beethoven” may have a high German score, while the central word may have a high Dutch score, etc. Some of the words may also have different language scores. Such different scores may be used to create the different expected pronunciations described below.

  In some aspects, a classifier may be implemented that is based on a machine learning classifier in which language features are expanded. A feature may include a combination of specific characters at the beginning, middle or end of a text identifier word string. Based on these features, scores may be assigned to different languages that are easy to incorporate features. For example, the classifier identifies a feature such as the presence of VA-N in the middle of a word string indicating the Dutch original language. The classifier assigns points or weights to each potential source language based on the likelihood that the text identifier is from each of those languages. Other classifier models include support vector machines / models or maximum entropy models, character level language models, and conditional random field models. These models may combine features and scores for different languages to score the most probable source language.

  In some aspects of the disclosure, the original language of the foreign language may be determined based on the original language of other text identifiers associated with the content item. For example, if one or more song titles or song lyrics of a particular artist are in German, the likelihood that the artist name comes from German may increase. In this case, the song title may be used as evidence for determining the original language of the artist name. In addition, other text identifiers may include metadata associated with the identified content. For example, an item of digital content may relate to metadata that may be used to identify or identify the original language of a text identifier. Other relationships between text identifiers may be explored to adjust the judgment of the original language.

  When one or more source languages are associated with a text identifier (or part thereof), the system is based on the source language (s) and text of the text identifier, and the expected pronunciation (s) of the text identifier. May be judged.

  In some aspects of the disclosure, a transformation model, such as a grapheme phoneme (G2P) transformation or pronunciation guessing model, may be developed for each potential source language. The transformation model derives the pronunciation of the foreign language text from the spelling of the foreign language text. Each language includes different language units such as phonemes. Cross-lingual mapping technology may be used to determine the expected pronunciation of the foreign language. A phoneme of a first language (eg, German) may be mapped to a phoneme of a second language (eg, English) that is most similar to the phoneme of the first language. However, some German pronunciations / phonemes may not be similar to or correspond to standard English phonemes. For example, the German pronunciation of the first letter “r” in Kraftwerk does not correspond to an English phoneme. The pronunciation of the letter “r” in German is actually “palatosis / r /” which is intermediate between the pronunciation of the letter “h” and the pronunciation of the letter “r”. In such a case, the German phoneme may be mapped to the closest English phoneme.

  In one aspect of the present disclosure, linguistic techniques are utilized to determine the nearest pronunciation of a foreign language. For example, in order to determine the nearest pronunciation of a foreign language, linguistic articulation features such as “back tongue”, “lips” or articulation may be implemented. The articulatory site may be a site in the oral cavity where the articulatory organ (eg, tongue, teeth, soft palate, etc.) restricts, forms or closes air flow during speech. Examples include both lip sounds (between lips), lip sounds (between lips and teeth), gum sounds (immediately behind the teeth), and uvula sounds (near the uvula). “Back tongue” may be defined as the degree to which a voice (usually a vowel) is tuned toward the throat. The back tongue vowels may include “au” for “caight”, “o” for “rote”, and “u” for “lute”. “Clips” or “lips” may be defined as a degree. Voice (often vowels, but not always) is tuned with rounded lips. The round lip vowel includes “o” of “rote” and “u” of “lute”. Linguistic techniques may be applied using a first language recognition device, such as, for example, an English phoneme recognition device, to recognize some embodiments of foreign languages having the target phonemes. The recognizer then determines the potential pronunciation of the foreign language.

  Several linguistic techniques (eg, expectation maximization algorithms, statistical models, hidden Markov models (HMMs) are used to analyze the association of multiple words and their corresponding pronunciations to determine the expected pronunciation of a new word. )) May be used. For example, lexicons including German and corresponding German pronunciations may be analyzed to determine associations between character sequences, phoneme sequences and the speech of each word. For example, the expectation maximization algorithm may learn that the letter PH in English may be pronounced as F with some exceptions. The expected value maximization algorithm may also learn when E is pronounced as “eh”, etc. with respect to “ee”. The model may be developed based on the analysis of the expectation maximization algorithm and used to predict new phoneme sequences and then the expected pronunciation of new words. Linguistic techniques may be used with other techniques to determine the expected pronunciation of a foreign language.

  Linguistic techniques also allow prediction of multiple alternative pronunciations for text identifiers based on the original language (s). For example, a plurality of pronunciations of each text identifier may be represented by a graph. Different parts of the graph may represent possible pronunciations for different parts of the text identifier. A portion of the graph, such as a graph edge, may be an assigned score or weight that indicates the likelihood of a path on the graph. Different graphs may be developed to represent different languages (eg, English and German). For example, separate graphs may be developed for English and German pronunciation. However, individual graphs may be combined together to predict mixed pronunciation of foreign languages in some embodiments. The combination graph allows the switching of the two languages as the pronunciation of the text identifier progresses, but this is the part of the text identifier that favors one language and the other of the text identifier that favors another language. This is desirable in situations where the part may be pronounced.

  For example, the German band “Kraftwerk” may be pronounced in German (eg, K HH AA F T V EH R K). However, some users may be unfamiliar with German pronunciation and may pronounce the band name “Kraftwerk” in English (eg, KRAEFTWURK). In addition, for some users, the selection of band name pronunciation may not be consistent. As a result, the text identifier (such as the band name “Kraftwerk”) is matched against multiple expected pronunciations, each predicted pronunciation itself may be based on a plurality of different languages including the original language (s) of the text identifier. May be.

  Some users may have a first original language but reside in a country where the users communicate in different languages (or operate the ASR device). These users may pronounce a foreign language using a combination of pronunciations from a plurality of languages including the user's original language. The user may pronounce part of the foreign language in the first language and the other part in one or more different languages. For example, the user may pronounce the first part of the band name, Kraftwerk in English (eg, K R AE FT) and the second part in German (eg, V EHRK).

  Pronunciation of English, KR AE FT W UR K, pronunciation of German, pronunciation of K HH AA FT V EH R K and combinations of K R AE F T V EH R K when added to lexicon In addition, the band name may be collated. Multiple expected pronunciations and band names may be linked to songs by bands stored on the ASR device or elsewhere.

  The predicted pronunciation of the foreign language may also be based on a specific user's pronunciation history. For example, the ASR system may be trained to recognize a particular user's pronunciation pattern or habit. If a word has a weight of 80% for French and 20% for English based on the spelling of the word, the classifier or speech recognition model may adjust the weight assigned to the language based on the particular user's habit. . Pronunciation patterns may also be based on the language rank preferred by a particular user. For example, the weight assigned to a language may be adjusted based on the language (s) preferred by the user. For example, the name Ludwig van Beethoven may have different versions of pronunciation due to its German and Dutch origin. In this case, weights may be assigned to German (eg 60%) and Dutch (eg 40%). In pronouncing foreign words such as Ludwig van Beethoven's name, the assigned weights may be adjusted based on whether a particular user prefers English, German or Dutch. The resulting pronunciation may be a mixture or combination of German, Dutch and English.

The user's pronunciation pattern may be determined based on the pronunciation history of the same or different words by the user. The ASR device may anticipate future pronunciation of the same or different words by the user based on the pronunciation pattern or history. The ASR device may also learn whether the user is familiar with pronunciation in one or more languages based on the user's pronunciation pattern. For example, based on the user's history of band name and Kraftwerk pronunciation,

Alternatively, the ASR device may expect the pronunciation of other German users, such as “Gustav Mahler”. The ASR device may also assign weights to different languages for a particular user based on the user's pronunciation pattern. For example, the ASR device may assign a greater weight to the pronunciation (for example, one language or a combination of languages) preferred by the user when a foreign word is pronounced. Similarly, a graphical representation of a preferred language or preferred route for a particular user may be assigned a higher score or weight. By assigning higher scores, these paths in the graph are more likely to represent the predicted pronunciation of the foreign language by the user. Thus, the expected pronunciation may be related to the expected pronunciation graph, the N-best list of expected pronunciations, or some other configuration of the expected pronunciation.

  In addition, multiple users with similar behavior may be clustered together for purposes of predictive pronunciation weighting or judgment. Features of automatic speech recognition techniques for clustered users are selected based on the behavior of the clustered users. For example, a user's cluster may have similar musical preferences (eg, music from India) and thus may have a music catalog that is dominated by Indian music. As a result, pronunciations from new users in the cluster may be treated the same as other users in the cluster, or follow a similar path along the graph (representing a possible pronunciation of a foreign language). May be. Weights may be assigned to corresponding features (eg, pronunciation, preferred language, etc.) of speech recognition technology associated with the user's cluster. Thus, the graph (representing a possible pronunciation of a foreign language) may be trimmed based on user behavior patterns or user clusters with similar behavior patterns.

  FIG. 6 shows a flow diagram of a method for predicting predicted text pronunciation of a foreign language based on the original language in speech recognition, according to one aspect of the present disclosure. The method may be implemented in the predicted pronunciation prediction module 128, the ASR device 100, and / or a remote speech processing device (eg, the ASR device 302). At block 602, content that is made available to the user may be incorporated into a catalog that is available to the ASR device 100. In block 604, one or more text identifiers may be linked to the content item as shown in block 604. At block 606, the ASR system may determine one or more source languages based on the text identifier (s). Each of the source language (s) may be associated with a score and / or specific portion of the text identifier (s). At block 608, the ASR system may determine one or more expected pronunciations of the text identifier based at least in part on the determined source language (s). Each of the expected pronunciation (s) based on the original language (s) may be associated with a score and / or a particular portion of the text identifier (s). At block 610, the ASR system may determine the expected pronunciation (s) of the text identifier based at least in part on the user information and / or user history. The user history may include a native language or a language frequently used by the user. The user history may also include how the user has previously pronounced similar words. User information may also include the determined language (s) of the device or the user's environment. Correlated location data having the known language (s) of the geographic region by determining the identified language (s) in other speech sensed by the device or through other means The language used at the location of the device that may be determined by the environment language may include the language of the environment. The language of the environment may also include the default language of the ASR system. Each of the predicted pronunciation (s) based on the user's language (s) may be associated with a score and / or a particular portion of the text identifier (s).

  At block 612, the ASR system combines the predicted pronunciation and based on the combination of the text identifier's original language (s) and the determined user's language (s) based on the text identifiers. One or more expected pronunciations may be determined. Each of the expected pronunciation (s) based on a combination of the user's language (s) may be associated with a score and / or a particular portion of the text identifier (s). In block 614, each or their priority of the expected pronunciation (s) and / or weights may be adjusted based on the user's history, such as the user's typical pronunciation or user category. At block 616, the predicted pronunciation (s) may be associated with the text identifier (s) and / or content item on the lexicon.

  The above-described determination of expected pronunciation may be made during training or configuration of the ASR system, or through addition to local storage when the ASR device becomes available for new content, or ASR It may be performed by making the device accessible but stored remotely. The determination of the predicted pronunciation may be performed by a local ASR device, a remote ASR device, or a combination thereof.

  As shown in FIG. 7, the ASR system may process utterances upon receipt of verbal utterances. At block 702, an utterance including a verbal text identifier is received. At block 704, the ASR system may match the verbal text identifier with the expected pronunciation (s) for the text identifier. Matching may include returning an N-best list of potential matches, or simply returning the highest score match. At block 706, the content item associated with the highest score match text identifier is determined. At block 708, the content item is accessed and any instructions related to speech (such as playing music) may be executed by the ASR system or by another device.

  The above-described aspects of the present disclosure are intended to be exemplary. They are selected to illustrate the principles and applications of the present disclosure and are not intended to be exhaustive or to limit the present disclosure. Many modifications and variations of the disclosed aspects will be apparent to those skilled in the art. For example, the ASR technique described herein based on linguistic information stored in voice storage may be applied to many different languages.

  Aspects of the present disclosure may be implemented as a computer-implemented method, system, or product such as a memory device or non-transitory computer-readable storage medium. The computer readable storage medium may be readable by a computer and may include instructions that cause a computer or other device to perform the processes described in this disclosure. The computer readable storage medium may be implemented by volatile computer memory, non-volatile computer memory, hard drives, solid state memory, flash drives, removable disks, and / or other media.

  Aspects of the present disclosure may be implemented in different types of software, firmware, and / or hardware. Further, the teachings of the present disclosure may be performed by, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other components.

  Aspects of the present disclosure may be performed on a single device or may be performed on multiple devices. For example, program modules that include one or more components described herein may be located in different devices, each performing one or more aspects of the disclosure. As used in this disclosure, the term “a” or “one” may include one or more items, unless stated otherwise. Further, the phrase “based on” is intended to mean “based at least in part on” unless stated otherwise.

  Clause

Article 1
A computer-implemented method for processing verbal speech,
Determining at least one source language of the song title based at least in part on the spelling of the song title;
Determining a plurality of potential pronunciations of the song title based at least in part on the at least one source language and the language spoken by the user, each of the plurality of potential pronunciations being associated with a score. The step of determining;
Storing an association between each of the plurality of potential pronunciations and the song name;
Receiving an oral utterance including a request to play a song;
Matching a portion of the verbal utterance with one of the plurality of potential pronunciations based at least in part on a score of the plurality of potential pronunciations;
Identifying the song based at least in part on one of the plurality of potential pronunciations;
Playing the song on a computing device;
Said method.

Article 2
The method of claim 1, wherein determining the plurality of potential pronunciations is further based at least in part on a user's pronunciation history of words for which at least one source language is common to the song title.

Article 3
The method of claim 1, further comprising determining at least one potential pronunciation by associating with one part of the song title in a first source language and a second part of the song title in a second source language. The method described.

Article 4
The method of claim 1, wherein determining the at least one source language of the song title is based at least in part on the source language of another song reproducible by the computing device.

Article 5
A computing system,
At least one processing device;
A memory device comprising instructions operable to be executed by the at least one processing unit for performing a set of actions, wherein the instructions are at least one processor,
Determining the potential source language of the text identifier such that the potential source language is based at least in part on the text identifier;
Determining the potential pronunciation of the text identifier such that the potential pronunciation is based at least in part on the potential source language and a potential verbal language;
The memory device configured to store an association between the potential pronunciation and the text identifier;
Including said computing system.

Article 6
The instruction is received by the at least one processing unit,
Determining the second potential source language for the text identifier, wherein the second potential source language is based at least in part on the text identifier;
Determining a second potential pronunciation of the text identifier, wherein the second potential pronunciation is based at least in part on the second potential source language;
6. The computing system of clause 5, further configured to store an association between the text identifier with the second potential pronunciation.

Article 7
7. The computing system of clause 6, wherein the potential source language, the second potential source language, the potential pronunciation and the second potential pronunciation are each associated with a respective score.

Article 8
The at least one processing device is further configured to determine a second potential source language of the text identifier;
The potential source language is associated with a first portion of the text identifier;
The second potential source language is associated with a second portion of the text identifier;
The potential pronunciation is further based at least in part on the second potential source language;
The calculation system according to clause 5.

Article 9
6. The computing system of clause 5, wherein the at least one processing device is further configured to determine the potential pronunciation based at least in part on a user's pronunciation history.

Article 10
The calculation system according to clause 9, wherein the pronunciation history of the user includes a language spoken by the user.

Article 11
The clause 5, wherein the at least one processing device is further configured to determine the potential source language based at least in part on an original language of a second text identifier associated with the text identifier. Calculation system.

Article 12
The instructions are executed by at least one processor;
Receive audio data including utterances,
Identifying the potential pronunciation in the utterance;
Identifying the text identifier based on the stored association;
6. The computing system of clause 5, further configured to retrieve at least a portion of a content item associated with the text identifier.

Article 13
6. The computing system of clause 5, wherein the text identifier includes an artist, album, band, movie, book, song and / or food name accessed by the computing device.

Article 14
6. The computing system of clause 5, wherein the potential verbal language comprises a language associated with a position of the system device.

Article 15
The at least one processing unit utilizes at least one of a finite state transducer (FST) model, a maximum entropy model, a character level language model, and / or a conditional random field model to generate the potential pronunciation of the text identifier. 6. The computing system of clause 5, further configured to determine.

Article 16
Program code for determining a potential source language for a text identifier, wherein the potential source language is based at least in part on a text identifier;
Program code for determining a potential pronunciation of the text identifier, wherein the potential pronunciation is based at least in part on the potential source language and a potential verbal language;
Program code for storing an association between the potential pronunciation and the text identifier;
A non-transitory computer readable storage medium storing processing unit executable instructions for controlling a computing device.

Article 17
Program code for determining a second potential source language for the text identifier, wherein the program code is based at least in part on the text identifier;
Program code for determining a second potential pronunciation of the text identifier, wherein the second potential pronunciation is based at least in part on the second potential source language; ,
Program code for storing an association between the second potential pronunciation and the text identifier;
The non-transitory computer readable storage medium of clause 16, further comprising:

Article 18
The non-transitory computer readable storage medium of clause 17, wherein the potential source language, the second potential source language, the potential pronunciation and the second potential pronunciation are each associated with a respective score. .

Article 19
A non-transitory computer readable storage medium further comprising program code for determining a second potential source language of the text identifier,
The potential source language is associated with a first portion of the text identifier;
The second potential source language is associated with a second portion of the text identifier;
The potential pronunciation is further based at least in part on the second potential source language;
The non-transitory computer readable storage medium of clause 16.

Article 20
The non-transitory computer readable storage medium of clause 16, further comprising program code for determining the potential pronunciation based at least in part on a user's pronunciation history.

Article 21
21. A non-transitory computer readable storage medium according to clause 20, wherein the pronunciation history of a user includes a language spoken by the user.

Article 22
17. The non-transitory computer readable code of clause 16, further comprising program code for determining the potential source language based at least in part on the source language of a second text identifier associated with the text identifier. Storage medium.

Article 23
Program code for receiving audio data including speech, and
Program code for identifying the potential pronunciation in the utterance;
Program code for identifying the text identifier based on the stored association;
Program code for retrieving at least a portion of a content item associated with the text identifier;
The non-transitory computer readable storage medium of clause 16, further comprising:

Article 24
The non-transitory computer readable storage medium of clause 16, wherein the text identifier accessed by the computing device comprises the name of an artist, album, band, movie, book, song and / or food.

Article 25
The non-transitory computer readable storage medium of clause 16, wherein the potential verbal language is associated with a location of a device of the system.

Article 26
The program code for determining the potential pronunciation of the text identifier is based at least in part on a finite state transducer (FST) model, a maximum entropy model, a character level language model, and / or a conditional random field model; 21. A non-transitory computer readable storage medium according to clause 16.

Claims (15)

A computer-implemented method for processing verbal speech,
Determining at least one source language of the song title based at least in part on the spelling of the song title;
Determining a plurality of potential pronunciations of the song title based at least in part on the at least one source language and the language spoken by the user, each of the plurality of potential pronunciations being associated with a score. The step of determining;
Storing an association between each of the plurality of potential pronunciations and the song name;
Receiving an oral utterance including a request to play a song;
Matching a portion of the verbal utterance with one of the plurality of potential pronunciations based at least in part on a score of the plurality of potential pronunciations;
Identifying the song based at least in part on one of the plurality of potential pronunciations;
Playing the song on a computing device;
Said method.   The method of claim 1, wherein determining the plurality of potential pronunciations is further based at least in part on a user's pronunciation history of words for which at least one source language is common to the song title.   The method of claim 1, further comprising determining at least one potential pronunciation by associating with one part of the song title in a first source language and a second part of the song title in a second source language. The method described.   The method of claim 1, wherein determining the at least one source language of the song title is based at least in part on the source language of another song reproducible by the computing device. A computing system,
At least one processing device;
A memory device comprising instructions operable to be executed by the at least one processing unit for performing a set of actions, wherein the instructions are at least one processor,
Determining the potential source language of the text identifier such that the potential source language is based at least in part on the text identifier;
Determining the potential pronunciation of the text identifier such that the potential pronunciation is based at least in part on the potential source language and a potential verbal language;
The memory device configured to store an association between the potential pronunciation and the text identifier;
Including said computing system. The instruction is received by the at least one processing unit,
Determining the second potential source language for the text identifier, wherein the second potential source language is based at least in part on the text identifier;
Determining a second potential pronunciation of the text identifier, wherein the second potential pronunciation is based at least in part on the second potential source language;
Further configured to store an association between the text identifier with the second potential pronunciation;
The calculation system according to claim 5.   The computing system of claim 6, wherein the potential source language, the second potential source language, a potential pronunciation and a second potential pronunciation are associated with respective scores. The at least one processing device is further configured to determine a second potential source language of the text identifier;
The potential source language is associated with a first portion of the text identifier;
The second potential source language is associated with a second portion of the text identifier;
The potential pronunciation is further based at least in part on the second potential source language;
The calculation system according to claim 5.   The computing system of claim 5, wherein the at least one processing device is further configured to determine the potential pronunciation based at least in part on a user's pronunciation history.   The calculation system according to claim 9, wherein the pronunciation history of the user includes a language spoken by the user.   6. The at least one processing device is further configured to determine the potential source language based at least in part on an original language of a second text identifier associated with the text identifier. Calculation system. The instructions are executed by at least one processor;
Receive audio data including utterances,
Identifying the potential pronunciation in the utterance;
Identifying the text identifier based on the stored association;
To search for at least some of the content items associated with the text identifier;
6. The computing system according to claim 5, further configured.   6. The computing system of claim 5, wherein the text identifier includes an artist, album, band, movie, book, song, and / or food name accessed by the computing device.   The computing system of claim 5, wherein the potential verbal language comprises a language associated with a position of a device of the system.   The at least one processing unit utilizes at least one of a finite state transducer (FST) model, a maximum entropy model, a character level language model, and / or a conditional random field model to generate the potential pronunciation of the text identifier. The computing system of claim 5, further configured to determine.