DE102015212650A1 - Method and system for the computer-aided processing of a voice input - Google Patents

Abstract

The invention relates to a method for the computer-aided processing of a voice input for a mobile unit (1) and an external server (10), between which at least temporarily a data connection exists. In this case, at least two sub-processes (Pn, P1, P2) are executed, which are executable both by the mobile unit (1) and by the external server (10). The invention further relates to a system for computer-aided processing of a voice input with a mobile unit (1) and an external server (10). In this case, the speech input can be detected by a detection unit of the mobile unit (1). The mobile unit (1) and the external server (10) each comprise a data interface (6a, 6b) and in each case a computing unit (5a, 5b). In this case, at least temporarily a data-technical connection between the mobile unit (1) and the external server (10) can be produced by the data interface (6a) of the mobile unit (1) and the data interface (6b) of the external server (10). At least two sub-processes (Pn, P1, P2) can be executed both by the arithmetic unit (5a) of the mobile unit (1) and by the arithmetic unit (5b) of the external server (10).

Description

The invention relates to a method for the computer-aided processing of a voice input for a mobile unit and an external server, between which at least temporarily a data connection exists. At least two sub-processes are executed, which can be executed both by the mobile unit and by the external server. The invention further relates to a system for computer-aided processing of a voice input with a mobile unit and an external server.

Speech recognition systems find a wide range of applications for operating systems. This applies equally to the operation of devices and the completion of telephone tasks using voice computers, such as telephone banking. Also, the operation of facilities of a vehicle via voice inputs has become established as an essential feature of current user interfaces in the vehicle. For example, to input complex facts corresponding information is queried sequentially by a user through a speech dialogue system.

It is of crucial importance that the recognition and interpretation of a user's speech input is as error-free and fast as possible. Although various methods have already been proposed for this, the large computing capacity required for the implementation frequently makes deployment difficult, for example in the field of mobile devices and in particular in vehicles. One approach to solving this problem is to outsource the computational tasks to a server accessible via the Internet. However, this presupposes that the voice recognition system can exchange data with the server at any time, which is not always guaranteed, especially in areas with incomplete coverage by mobile radio networks.

That in the DE 10 2012 022 630 A1 described method for communication of a driver with a driver assistance system provides that the driver directed natural language requests to the system, which are processed and answered natural language.

For this purpose, based on the voice input of the driver, a keyword recognition is performed, with keywords are also found in an external source, such as an Internet connection, in a database.

When in the WO 2014/060054 A1 Speech recognition described above is generated from speech inputs of a user in a vehicle by a (local) onboard speech recognizer and an offboard external speech recognizer recognition results. The best language text can now be selected, for example based on the confidence values for the individual recognition results.

The present invention has for its object to provide a method and a system of the above type, in which a voice input is processed reliably and with optimum utilization of all available resources.

According to the invention this object is achieved by a method having the features of claim 1 and a system having the features of claim 9. Advantageous embodiments and further developments emerge from the dependent claims.

In the method according to the invention of the type mentioned above, the voice input is detected by the mobile unit. It generates (a) offline input data and online input data for the first sub-process and (b) transfers the online input data to the external server if the data connection exists. In a further step (c), the mobile unit performs the current sub-process, whereby offline output data is generated on the basis of the offline input data. Further, if the online input data has been transmitted to or generated by the external server, (d) the current thread is executed by the external server, generating online output data from the on-line input data. If the data link exists, the online output data is transmitted to the mobile unit. Finally, (e) new offline input data and new online input data are generated for the next subprocess. Thereby, steps (c) to (e) are executed for each of the sub-processes until the last sub-process has been executed. In each case, the next sub-process is executed and the sequence of steps (c) to (e) is repeated until the end of the last sub-process.

The mobile unit is arranged in particular in a vehicle or integrated in this.

The inventive method is used to process a voice input with at least two sub-processes. This is typical, for example, for voice control methods in which a machine-readable text is generated on the basis of audio data of spoken speech by a speech recognizer, by means of which commands are subsequently recognized and executed. In further steps, the data can be further processed or analyzed, for example to improve the reliability of the voice control.

In the following, "sub-processes" are understood to mean modular functions that are executed by computers. In particular, input data is used to generate output data. The individual sub-processes can be executed after or next to each other. The input data for executing a sub-process may have been generated by any suitable computer and the output data may be further processed by any suitable computer.

In particular, the at least two sub-processes are executed one after the other during the processing of the speech input, whereby a sub-process generates output data which either directly serve as input data for the subsequent sub-process or from which the input data for the subsequent sub-process are generated. The sub-processes may comprise, for example, several consecutive steps of processing the speech input.

The data connection between the mobile unit and the external server can in particular be wireless, for example, by a local network or a larger network, such as the Internet. Furthermore, the connection can be made via a telecommunications network, such as a telephone network, or a wireless local area network (WLAN). Furthermore, the data connection can be made by connecting a data cable. The connection can also be made through another device that can itself connect to the external server. For example, a data connection may exist between the mobile unit and a mobile telephone connected to the Internet, for example by means of a data cable or a radio connection, for example via Bluetooth. In particular, the connection to the external server can be made via the Internet.

The voice input is detected by the mobile unit, for example by a built-in microphone. However, the voice input may also be detected indirectly, for example by retrieving a voice input stored at another device.

When capturing the speech input, in particular audio data are recorded and stored. Furthermore, a first processing of the data of the speech input can take place, for example a compression according to methods known per se.

In order to be able to carry out the first sub-process, input data is generated for the mobile unit and the external server. In the following, the terms offline input data and offline output data refer to the mobile unit and the terms online input data and online output data to the external server. To carry out the first subprocess, the generated online input data is transmitted to the external server. If there is currently no data connection between the mobile unit and the external server, then the transfer can not be performed. In this case, the first sub-process is initially performed only by the mobile unit. If the data connection is established at a later time, the online input data can be transmitted to the external server and the first sub-process can be performed by the external server. As a result of the first sub-process, offline or online output data are generated.

To carry out the next subprocess, new offline and online input data must now be generated. This can be done in particular on the basis of the output data generated in the first sub-process. If there is no data connection between the mobile unit and the external server, then only the own output data is available to the two components. In this case, the new online input data is generated by the external server and the new offline input data is generated by the mobile unit. On the other hand, if the data connection exists and the online output data has been transmitted to the mobile unit, the mobile unit can generate the new online and offline input data based on the output data generated in the previously performed subprocess. In this case, identical online and offline input data can be generated.

The next subprocess is now performed, using the new online and offline input data. If the online input data were generated by the mobile unit, they must first be transferred to the external server. This can be repeated for any number of subprocesses until the last subprocess has been executed.

In this way, the sub-processes can be performed in parallel by the mobile unit and the external server. If the data connection exists at the respective time, a merger and adjustment of the respective output data can take place between the execution of the individual sub-processes.

In particular, it can thus be ensured that both components use the same input data and that the resulting output data are thus comparable.

Thus, advantageously, the resources of the mobile unit and the external server can be used simultaneously, in particular in the case of an unstable data connection, the entire parallel sequence of the plurality of sub-processes can be coordinated between the individual sub-processes.

At least two sub-processes are carried out in parallel by the mobile unit and the external server and coordinated in the meantime, as far as the data connection permits a comparison between the individual sub-processes. As far as the data connection permits, the output data of the two components are brought together between each subprocess and, in particular, the same offline and online input data are generated for the following subprocess. If, however, there is no data connection between the mobile unit and the external server for a period of time, the processing of the data for the voice input for individual sub-processes can nevertheless be continued without coordination, namely by the mobile unit or the external server. This allows a high flexibility of the whole process by utilizing the modular processing of voice input.

In one embodiment, one of the sub-processes includes speech recognition, wherein the offline input data and the online input data are identical and generated based on the speech input. Further, the offline output data and the online output data each include machine-readable text data.

As a result, advantageously, a voice recognition module can be performed in parallel by the mobile unit and the external server. In particular, an automatic speech recognition (ASR) is performed. This can be done by methods known per se. It is assumed in particular by the speech input in the form of audio data. A machine-readable text is generated, wherein in particular a language model is used for recognition. The language model comprises about a vocabulary and expression, in particular, individual words, phrases and linguistic constructions are represented, but also statistical relationships of the individual words and phrases are represented to each other. The language model may include further supplementary data. The computational effort required to carry out the speech recognition depends, in particular, on the size of the language model.

Furthermore, the quality of speech recognition depends on the language model. Since the computing capacity of the mobile unit is typically small compared to the external server, mobile units often can only perform basic speech recognition using a simpler speech model, while the external server can typically provide higher quality speech recognition. The quality of the speech recognition performed can be evaluated according to known methods, for example, a confidence value can be determined.

In one development, one of the sub-processes includes the generation of a semantic analysis. The offline input data and the online input data each comprise machine-readable text data, and the offline output data and the online output data each include analysis data.

This advantageously makes it possible to carry out a semantic analysis involving the computing capacities of the mobile unit and the external server. In particular, natural language understanding (NLU) can be carried out by methods known per se. For example, keywords can be recognized. In particular, statements are generated from the machine-readable text data, which in the following can be transmitted and executed as instructions to the system.

In another embodiment, one of the sub-processes includes performing a search. The offline input data and the online input data respectively include analysis data, and the offline output data and the online output data each include search result data.

As a result, advantageously, a search functionality can be implemented, for example within a voice control. The search can be carried out according to methods known per se, wherein in particular a search query is generated on the basis of the analysis data and search results are generated as output data. For example, a database of records can be searched by a search query. The search result data in this case may include a list of records relevant to the search query.

In one embodiment, if in step (d) the on-line output data has been transmitted to the mobile unit, the new online input data generated in step (e) and new offline input data for the next respective sub-process will be based on a comparison of the online output data and the offline output data is generated.

As a result, advantageously the presence of offline and online output data generated in parallel can be used to generate the new offline and online input data. In particular, it is determined which output data is most suitable, with a comparison of the output data being performed. In the simplest case, the selected output data can be used directly as new input data. Furthermore, the new input data can also be generated based on both previously generated output data. In particular, they are there generated online and offline input data essentially identical.

In a development, if in step (d) the online output data has not been transmitted to the mobile unit, in step (e) the new online input data is generated from the online output data and the new offline input data is generated from the offline output data generated.

As a result, advantageously, the next sub-process can be carried out, even if the data connection does not allow the new input data to be generated centrally. This means, in particular, that the online and offline input data used in the next step are not identical to one another. Instead, two or more subprocesses are performed in parallel by the mobile unit and the external server without any intermediate adjustment. This is particularly important if the data connection does not exist continuously.

In a further embodiment, if in step (d) the online output data has been transmitted to the mobile unit, after another sub-process has already been started, it is determined based on a comparison of the online output data with the associated offline output data, whether the later sub-processes to be carried out again with newly generated online input data and new offline input data.

As a result, in particular a delay in the parallel execution of the sub-processes can be taken into account. The time delay may arise, for example, because the data connection does not exist permanently or the data transmission takes place with only a small bandwidth. This may result in the online output data generated by the external server reaching the mobile unit only after it has been assumed that no data transmission will take place. In this case, the mobile unit continues execution with the next thread, with the on-line output data initially disregarded as they were not available at the time of execution. Conversely, the time delay may be due to a slower execution of a sub-process by the mobile unit.

In this case, the external server can start performing the next thread without considering the offline output data. Thus, one component as a whole can perform the sequence of sub-processes faster than the other. If output data of one of the components for a sub-process are available late, a delayed analysis can be carried out as to whether the newly added output data is better suited for carrying out the next sub-process. In this case, it can be determined that the following sub-process is carried out again with input data, the newly added output data being taken into account in the generation of the input data. This allows in particular the treatment of cases in which the processing and data transmission does not allow fully parallel and synchronous execution of the sub-processes.

In training, after the last thread has been executed, final output data is generated from the offline output data and the online output data.

As a result, an optimized result of the execution of all sub-processes can advantageously be generated. For example, the optimized result may be output or used to perform other functions, such as the voice control of a device.

In the system according to the invention of the type mentioned above, the speech input can be detected by a detection unit of the mobile unit. Furthermore, the mobile unit and the external server each comprise a data interface and a respective computing unit. In this case, at least temporarily a data connection between the mobile unit and the external server can be produced by the data interface of the mobile unit and the data interface of the external server. Furthermore, at least two sub-processes can be executed both by the arithmetic unit of the mobile unit and by the arithmetic unit of the external server.

The system according to the invention is in particular designed to implement the method according to the invention described above. The system thus has the same advantages as the method according to the invention.

The computing units of the mobile unit and of the external server are in particular designed to coordinate the execution of the sub-processes by the mobile unit and by the external server. They are designed so that (a) offline input data and online input data can be generated for the first subprocess and (b) the online input data can be transmitted to the external server if the data connection exists. In a further step (c), the current subprocess can be executed by the mobile unit, wherein offline output data can be generated on the basis of the offline input data. Further, if the online input data has been transmitted to or generated by the external server, (d) the current thread is executable by the external server, using the Input data online output data can be generated. If the data connection exists, the online output data is transferable to the mobile unit. Finally, (e) new offline input data and new online input data can be generated for the next sub-process. The steps (c) to (e) are executable for each of the sub-processes until the last sub-process has been executed. In each case, the next sub-process can be executed and the sequence of steps (c) to (e) can be repeated until the end of the last sub-process.

The invention will now be explained by means of embodiments with reference to the drawings.

1 shows an embodiment of the system according to the invention,

2 shows an embodiment of the method according to the invention with a permanent data connection,

3 shows the embodiment of the method according to the invention with an interrupted in the meantime data connection and

4 shows the embodiment of the method according to the invention with a time-delayed implementation of sub-processes.

In reference to 1 an embodiment of the system according to the invention will be explained.

The system according to the invention comprises a mobile unit 1 and an external server 10 , The mobile unit 1 can be arranged in a vehicle or integrated into this. The mobile unit 1 and the external server 10 each comprise a data interface 6a and 6b and in each case one arithmetic unit 5a and 5b , The mobile unit 1 further comprises a detection unit 2 with a microphone 2a ,

By the registration unit 2 Voice inputs of a user can be detected, wherein the user in the illustrated embodiment signals the beginning of a voice input by pressing a push-to-talk switch (PTT). The voice input spoken by the user is then captured and recorded. A caching is performed so that the voice input data is available for further use. In particular, audio data is recorded. The beginning of speech input can also be recognized in other ways, for example after activating a function that allows voice control.

The registration unit 2 is with the arithmetic unit 5a the mobile unit 1 coupled and the stored data of voice input can be transmitted. The arithmetic unit 5a the mobile unit 1 is also with the data interface 6a the mobile unit 1 coupled. Furthermore, the arithmetic unit 5b the external server 10 with the data interface 6b the external server 10 coupled. Between the data interfaces 6a and 6b the mobile unit 1 and the external server 10 a data connection can be established. This is done in the illustrated embodiment via a mobile network through which the mobile unit 1 Has access to the Internet and the data connection to the external server 10 can produce. As a result, as long as the data connection exists, data can be exchanged and transmitted.

The arithmetic unit 5a the mobile unit 1 includes a speech recognizer 3a , a semantic expert 4a and a search engine 7a , Analogously, the computing unit also includes 5b the external server 10 a speech recognizer 3b , a semantic expert 4b and a search engine 7b ,

The speech recognizers 3a and 3b the mobile unit 1 and the external server 10 generate machine-readable text data based on data from a voice input. Based on such text data, the semantic experts generate 4a and 4b the mobile unit 1 and the server 10 each analysis data. The search engines 7a and 7b the mobile unit 1 and the external server 10 perform a search on the basis of such analysis data and / or a search query and thereby generate search result data. For example, in the exemplary embodiment shown, the database of a database is searched, whereby individual entries are output which are relevant for a search request.

In particular, the data of the speech input provided by the detection unit 2 be captured directly as input data for the speech recognizers 3a and 3b used by the speech recognizers 3a and 3b generated text data can be used directly as input data for the semantic connoisseurs 4a and 4b used by semantic experts 4a and 4b generated analytic data can be obtained directly from the search engines 7a and 7b be interpreted as a search query. It does not matter what component of the system, whether the mobile unit 1 or the external server 10 , the data was generated, which serve as the next step as input. The system is thus modular, with the individual modules being interchangeable.

In reference to 2 is the embodiment of the system according to the invention and a Embodiment of the method according to the invention explained with a permanent data connection.

The embodiment is of the in 1 shown executed system. In a first step 201 a voice input from a user is recorded. The recorded audio data of the voice inputs are in a further step 202 used to generate first input data. Here, first offline input data I ^off ₁ and first online input data I ^on _{1 are} generated, which are identical in the case shown. The first online input data I ^on ₁ are sent via the data connection to the server 10 transfer.

Subsequently, the first sub-process P1 is performed. This is done in parallel by the mobile unit 1 and through the external server 10 , while carrying out 203a of the first sub-process P1 by the mobile unit 1 first offline output data O ^off _{1 are} generated. In parallel, be in the process of implementation 203b of the first sub-process P1 by the external server 10 first online output data O ^on ₁ generated. The latter are from the external server 10 to the mobile unit 1 transferred and in one step 204 are generated on the basis of the first offline output data O ^off ₁ and the first online output data O ^on ₁ second input data for performing the second sub-process P2. In this case, second offline input data I ^off ₂ and second online input data I ^on _{2 are} generated.

The production 204 is done by comparing the first offline output data O ^off ₁ and the first online output data O ^on ₁ . In the example shown, speech recognition is performed in the first sub-process P1. It is based on the first step 201 detected speech input recognized a machine-readable text.

In parallel offline and online speech recognition steps 203a and 203b In addition to a recognized text in each case, confidence values are determined which specify the quality of the speech recognition. In the following step 204 can be determined based on these confidence levels of better quality recognized text. As an alternative or in addition to the comparison of the output data, general selection rules can be defined. For example, it can always be assumed that the one from the external server 10 generated output data should be selected.

The recognized text is used as second input data for the second sub-process P2, in this case a semantic analysis. The second offline input data I ^off ₂ and the second online input data I ^on ₂ are therefore essentially identical. The second online input data I ^on ₂ are sent to the external server 10 transfer. The second sub-process P2 is in the process steps 205a and 205b from the mobile unit 1 or from the external server 10 carried out. In this case, the second offline output data O ^off ₂ or the second online output data O ^on _{2 are} generated. The latter are connected via the data connection to the mobile unit 1 transfer. The second output data O ^off ₂ and O ^on ₂ are in the final step 206 compared with each other and it is determined which of the analysis data should serve as final output data. In the illustrated case, a plausibility check is carried out on the basis of recognized keywords.

In a further exemplary embodiment, a series of further sub-processes Pn is then carried out analogously to the first sub-process P1 and the second sub-process P2. In particular, a search request is processed that was recognized on the basis of the semantic analysis in the second sub-process P2. The processing is analogous to the steps described above, ie it will be handled by the mobile unit 1 Based on the offline input data I ^off _n offline output data O ^off _n generated and it will be from the external server 10 generated on the basis of the online input data I ^on _n online output data O ^on _n . These eventually become a mobile unit again 1 and it is determined by a comparison which output data should be reused or discarded.

In reference to 3 the embodiment of the system according to the invention and the embodiment of the method according to the invention is explained with an intermittently interrupted data connection.

The method is described by the above with reference to 1 explained system executed. This is essentially analogous to that described above with reference to 2 illustrated embodiment. In contrast, in this case, however, the data connection is interrupted in the meantime, which in 3 is indicated by a multiple line. It is therefore not possible in the implementation of the first sub-process P1 by the external server 10 generated first online output data O ^on ₁ to the mobile unit 1 transferred to. Conversely, the second online input data I ^on ₂ for carrying out the second partial process P2 can not be provided by the mobile unit 1 to the external server 10 be transmitted.

Analogous to that related to 2 explained embodiment are in a first step 301 Data of a voice input of a user recorded. This data will be in a second step 302 used to generate first offline input data I ^off ₁ and first online input data I ^on ₁ . The first online input data I ^on ₁ will be sent to the external server 10 transfer. Subsequently, the first sub-process P1 of the mobile unit 1 in one step 303a performed, wherein first offline output data O ^off _{1 are} generated, while in parallel the execution by the external server 10 in one step 303b takes place, wherein first online output data O ^on _{1 are} generated. After completion of the first subprocess P1 are in an offline step 304a and an online step 304b generates the second offline input data I ^off ₂ and the second online input data I ^on ₂ . These are generally not equal to each other.

Subsequently, the second sub-process P2 is performed, wherein in one step 305a through the mobile unit 1 second offline output data O ^off _{2 are} generated and in one step 305b through the external server 10 second online output data O ^on _{2 are} generated. Since the data connection has been restored in the case shown, the generated second online output data O ^on _{2 are} transmitted to the mobile unit and in a further step 306 is determined from the second offline output data O ^off ₂ and the second online output data O ^on ₂ , which should be the final output data.

In the case of the embodiment shown, the system recognizes that the data link does not exist and the generation 304a the second offline input data I ^off ₂ takes place immediately after the generation 303a the first offline output data O ^off ₁ . Furthermore, the generation of the second online input data I ^on _{2 takes place} in step 304b immediately after the production 303b the first online output data O ^on ₁ .

In another embodiment, the steps include 304a and 304b the generation of the second input data, a waiting condition, being waited for a certain time whether both first output data is present or whether they are not transmitted. After the specific time has elapsed, the generation of the input data takes place on the basis of the present output data of the previously executed subprocess Pn.

In reference to 4 the embodiment of the system according to the invention and the embodiment of the method according to the invention with a time-delayed implementation of sub-processes will be explained.

The procedure is carried out with the in 1 illustrated system. Unlike the related to 2 and 3 In the cases shown, the individual steps are carried out on the external server 10 and the mobile unit 1 not in sync.

First, in a first step 401 recorded a voice input of a user and it will be in a second step 402 first offline input data I ^off ₁ and first online input data I ^on ₁ generated. These are essentially the same in the case shown. The first online input data I ^on ₁ becomes the external server 10 transfer. The implementation of the first sub-process P1 is performed by the external server 10 in an online step 403b and by the mobile unit in an offline step 403a , It takes the execution of the online step 403b longer than performing the offline step 403a , That is, first offline output data O ^off _{1 is} generated, while the first online output data O ^on ₁ is not yet present. The system therefore sets the processing of the speech input by the mobile unit 1 in the next step 404 on the basis of the first offline output data O ^off ₁ second offline input data I ^off _{2 are} generated and based on this, the second sub-process P2 in a further step 405a is carried out.

In the case of the embodiment shown, at this time, the execution of the first sub-process P1 is performed by the external server 10 completed and first online output data O ^on _{1 is} generated and sent to the mobile unit 1 transfer. In a parallel step 404 ' it is checked whether the first offline output data O ^off ₁ with respect to the added first online output data O ^on _{1 have} a sufficient quality. If in this step 404 ' that is determined by the mobile unit 1 O ^off _{1 are} good enough (this case is in the 4 not shown), the execution of the sub-processes Pn is continued as it was already started and there are second online input data I ^on ₂ , which is essentially the one previously in step 404 generated second offline input data I ^off ₂ , to the external server 10 transferred and used to carry out the second sub-process P2.

On the other hand, it is found that the first online output data O ^on ₁ , by the external server 10 are produced, are better suited to the continuation, so will the implementation 405a of the second sub-process P2 by the mobile unit 1 canceled ( 407 ) and again second offline input data I ^off ₂ and second online input data I ^on _{2 are} generated. The latter are sent to the external server 10 then transfer where in an online step 405b ' second online output data O ^on ₂ generated and sent to the mobile unit 1 be transmitted. In parallel, by the mobile unit 1 in an offline step 405a ' Based on the second offline input data I ^off ₂ second offline output data O ^off ₂ generated. In a final step 406 It is determined which of the second output data will be used as the final output data.

The illustrated cases of the embodiment of the method according to the invention can also occur in combination. In particular, any number of further sub-processes can be carried out analogously to the illustrated sub-processes Pn. There are time delays, such as an unstable data connection and / or the different performance of the mobile unit 1 and the external server 10 considered.

In a further embodiment, the processing of the voice input also includes sub-processes that are only from the mobile unit 1 or from the external server 1 can be executed. This includes conducting a search based on a database of the external server 10 to which the mobile unit 1 can not directly access, or access to a device of the mobile unit, such as a navigation system, a calendar program, an address book or a multimedia device to which the external server 10 can not access directly. This is not a subprocess Pn in the sense of modular functionality described above, which is used by both the mobile unit 1 and the external server 10 can be executed. Nevertheless, such special functionalities of the system components can also be carried out on the basis of output data which were generated according to the invention and / or they can generate input data for executing further subprocesses Pn.

LIST OF REFERENCE NUMBERS

1

 Mobile unit

2

 acquisition unit

2a

 microphone

3a

 Speech recognizer of the mobile unit

3b

 Speech recognizer of the external server

4a

 Semantic identifier of the mobile unit

4b

 Semantic identifier of the external server

5a

 Arithmetic unit of the mobile unit

5b

 Arithmetic unit of the external server

6a

 Data interface of the mobile unit

6b

 Data interface of the external server

7a

 Search engine of the mobile unit

7b

 Search engine of the external server

10

 External server

P1, P2, Pn

 subprocesses

I ^on ₁ , I ^on ₂ , I ^on _n

 Online input data

I ^off ₁ , I ^off ₂ , I ^off _n

 Offline Input data

O ^on ₁ , O ^on ₂ , O ^on _n

 Online output data

O ^off ₁ , O ^off ₂ , O ^off _n

 Offline output data

201, 301, 401

 Recording a voice input

202, 302, 402

 Generation of the first input data

203a, 303a, 403a

 Implementation of the first sub-process (mobile unit)

203b, 303b, 403b

 Execution of the first subprocess (external server)

204, 404, 404 '

 Generation of the first input data

205a, 305a, 405a, 405a '

 Implementation of the first sub-process (mobile unit)

205b, 305b, 405b '

 Execution of the first subprocess (external server)

206, 306, 406

 Generation final output data

304a

 Generation of the second input data (mobile unit)

304b

 Generation of the second input data (external server)

407

 cancellation

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102012022630 A1 [0004]
• WO 2014/060054 A1 [0006]

Claims (9)

Method for the computer-aided processing of a voice input for a mobile unit ( 1 ) and an external server ( 10 ), between which at least temporarily a data connection exists, wherein at least two sub-processes (Pn, P1, P2) are carried out, both by the mobile unit ( 1 ) as well as from the external server ( 10 ), in which the voice input is detected by the mobile unit and the following steps are performed: (a) for the first sub-process, offline input data (I ^off ₁ , I ^off _n ) and online input data (I ^on ₁ , I ^on _n ), (b) if the data link exists, the online input data (I ^on ₁ , I ^on _n ) is transmitted to the external server, (c) from the mobile unit ( 1 ), the current sub-process (Pn, P1, P2) is executed, using the offline input data (I ^off ₁ , I ^off _n ) offline output data (O ^off ₁ , O ^off _n ) generated, (d) if the online Input data (I ^on ₁ , I ^on _n ) have been transmitted to or generated by the external server, the external server ( 10 ) the current sub-process (Pn, P1, P2) is executed, wherein on the basis of the online input data (I ^on ₁ , I ^on _n ) online output data (O ^on ₁ , O ^on _n ) are generated, and, if the data connection exists , the online output data (O ^on ₁ , O ^on _n ) is transmitted to the mobile unit (1), and (e) new offline input data (I ^off ₂ , I ^off _n ) and new online input data (I ^on ₂ , I ^on _n ) are generated for the next sub-process (Pn, P1, P2); wherein the steps (c) to (e) are carried out for each of the sub-processes (Pn, P1, P2), wherein in each case the next sub-process (Pn, P1, P2) is carried out until the last sub-process (Pn, P1, P2) has been executed. A method according to claim 1, characterized in that one of the sub-processes (Pn, P1, P2) comprises a speech recognition, wherein the offline input data (I ^off _n ) and the online input data (I ^on _n ) are identical and generated based on the voice input and the offline output data (O ^off _n ) and the online output data (O ^on _n ) each comprise machine-readable text data. Method according to one of the preceding claims, characterized in that one of the sub-processes (Pn, P1, P2) comprises the generation of a semantic analysis, the offline input data (I ^off _n ) and the online input data (I ^on _n ) each being machine-readable Comprise text data and the offline output data (O ^off _n ) and the online output data (O ^on _n ) each comprise analysis data. Method according to one of the preceding claims, characterized in that one of the sub-processes (Pn, P1, P2) comprises carrying out a search, wherein the offline input data (I ^off _n ) and the online input data (I ^on _n ) each comprise analysis data and the offline output data (O ^off _n ) and the online output data (O ^on _n ) each include search result data. Method according to one of the preceding claims, characterized in that, if in step (d) the online output data (O ^on ₁ , O ^on _n ) has been transmitted to the mobile unit (1), the new one generated in step (e) is online -Inputdaten (I ^on _2, I ^on _n) and new offline input data (I ^off _2, I ^off _n) for the respective next partial process (Pn, P1, P2) based on a comparison of the online-output data (O ^on _1, O ^on _n ) and the offline output data (O ^off ₁ , O ^off _n ) are generated. Method according to one of the preceding claims, characterized in that, if in step (d) the online output data (O ^on ₁ , O ^on _n ) were not transmitted to the mobile unit (1), in step (e) the new ones online Input data (I ^on ₂ , I ^on _n ) are generated from the online output data (O ^on ₁ , O ^on _n ) and the new offline input data (I ^off ₂ , I ^off _n ) from the offline output data (O ^off ₁ , O ^off _n ) are generated. Method according to one of the preceding claims, characterized in that, if in step (d) the online output data (O ^on ₁ , O ^on _n ) were transmitted to the mobile unit (1), after already a further sub-process (Pn, P1 , P2) was started, based on a comparison of the online output data (O ^on ₁ , O ^on _n ) with the associated offline output data (O ^off ₁ , O ^off _n ) determines whether the later sub-processes (Pn, P1, P2 ) with newly generated new online input data (I ^on ₂ , I ^on _n ) and new offline input data (I ^off ₂ , I ^off _n ) should be carried out again. Method according to one of the preceding claims, characterized in that, after the last sub-process (Pn, P1, P2) has been executed, based on the offline output data (O ^off _n ) and the online output data (O ^on _n ) produces final output data become. System for the computer-aided processing of a voice input with a mobile unit ( 1 ) and an external server ( 10 ), wherein a detection unit of the mobile unit ( 1 ) the voice input is detectable and the mobile unit ( 1 ) and the external server ( 10 ) each have a data interface ( 6a . 6b ) and one arithmetic unit ( 5a . 5b ), whereby through the data interface ( 6a ) of the mobile unit ( 1 ) and the data interface ( 6b ) of the external server ( 10 ) at least temporarily a data connection between the mobile unit ( 1 ) and the external server ( 10 ) and at least two sub-processes (Pn, P1, P2) can be produced both by the arithmetic unit ( 5a ) of the mobile unit ( 1 ) when also from the arithmetic unit ( 5b ) of the external server ( 10 ) are executable.

Images