DE102022208762A1 - Text extraction from audio data in a vehicle - Google Patents

Abstract

The invention relates to a system (1) for outputting data in text form from audio data in a vehicle (3), comprising a receiver unit (5) for receiving audio data and for acoustically playing back the audio data, and a temporary storage unit (7) for temporarily storing the Audio data, an input unit (9) for recording input from a user in the vehicle (3), the temporary storage unit (7) being designed to store a data stream of the audio data for a specific period of time and to delete it after the specific period of time has elapsed, characterized by an artificial intelligence module (11), wherein the artificial intelligence module (11) is designed to analyze the audio data stored in the temporary storage unit (7) using artificial intelligence functions when a predetermined command is received at the input unit (9). and extract relevant information and, at the user's command, to make the extracted information available to the user in text form or to save it.

Description

The invention relates to a system for outputting data in text form from audio data in a vehicle, and to a vehicle with such a system.

A driver of a vehicle such as a passenger car can acoustically perceive interesting information from the radio, telephone, or other sources of the vehicle's infotainment system, but the content of which he cannot easily note down or save while the vehicle is driving in order to use the information again later. Taking notes by hand on paper, for example, distracts from traffic and leads to a loss in the safety of driving the vehicle.

to temporarily store audio information in a vehicle and discard it by default after a certain period of time, unless a request from a user such as the driver of the vehicle requires the audio information of interest to be converted into text for further use, is known in the art.

The US 2016/0019892 A1 relates to a device for automating Internet searches using audio information received from a vehicle radio, the device comprising: a radio adapted to receive digital audio data with voice; a circular buffer coupled to the radio and configured to continuously store digital data output from the radio; after the circular buffer is filled, previously stored digital data is written to, the circular buffer being sized to store digital data representing between about fifteen seconds of audio and up to about two minutes of audio data; a speech-to-text converter coupled to the ring buffer and configured to convert digital data in the ring buffer into one or more text strings that can be displayed on a display device; a touch-sensitive display device coupled to the speech-to-text converter and configured to display a text string generated by the speech-to-text converter; the touch-sensitive device is also configured to enable tactile selection of the displayed text string and to receive tactile input that, when provided to a processor, causes the processor to stop recording digital data in the ring buffer and the processor converts the digital data in the ring buffer into text that can be displayed on said touch-sensitive display device; a processor coupled to the touch-sensitive display device is configured to perform an action responsive to the selection of the displayed text.

The object of the invention is to improve the conversion of data into text form from audio data in a vehicle, in particular to make it more relevant and efficient.

The invention results from the features of the independent claims. Advantageous further developments and refinements are the subject of the dependent claims.

A first aspect of the invention relates to a system for outputting data in text form from audio data in a vehicle, comprising a receiver unit for receiving audio data and for acoustically playing back the audio data, a temporary storage unit for temporarily storing the audio data, an input unit for recording inputs of a User in the vehicle, wherein the temporary storage unit is designed to store a data stream of audio data for a specific period of time and to delete it after the specific period of time, characterized by an artificial intelligence module, wherein the artificial intelligence module is designed to receive of a predetermined command on the input unit, to analyze the audio data stored in the temporary storage unit using artificial intelligence functions and to extract relevant information and, at the user's command, to make the extracted information available to the user in text form or to save it, and in particular only that to make the extracted information available to the user in text form or to save it.

The receiver unit is advantageously arranged in the vehicle and designed to receive audio data such as from public radio stations or telephone conversations and to reproduce this audio data acoustically through appropriate conversion by loudspeakers. The temporary storage unit preferably temporarily stores this audio data with its data streams in a ring memory in such a way that after a certain time the oldest data in the ring memory is continuously overwritten with the most recent data. When the user receives a command that the audio data contains information of interest to him, he gives a corresponding command to the input unit, which is recognized by the system. The audio data is thus read from the ring buffer before it is overwritten. This The audio data read out is analyzed by the artificial intelligence module for relevant content.

For this purpose, the input unit is advantageously designed as a touch-sensitive screen or as a unit that detects voice control commands. Any input units are possible that enable the purpose of recording the user's specified command.

The artificial intelligence module has correspondingly high computing power and is preferably arranged in the interior of the vehicle, but can also be implemented by a central computing unit in the cloud. The artificial intelligence module is therefore able to semantically analyze the audio data with the language it contains, i.e. H. to check the content of the language. The analyzed content is checked for relevance, in particular using heuristic methods from local or central databases that keep a history of an individual user or of a large number of users generally stored. The language in the audio data is also preferably semantically checked to see whether it contains facts about contact details such as telephone numbers, email addresses, names, etc., or facts about events such as meetings, appointments, events, etc., or other facts, that could be relevant to the user in some way. Appropriate language models are used to advantageously recognize generalities and filter them out as irrelevant. Complex and pre-trained artificial neural networks are advantageously used here. Other statistical approaches such as Bayesian approaches can support the approach to checking for relevance of language in the audio data.

This relevant information is then transferred by the artificial intelligence module into an output data stream in order to store the relevant information or output it directly to the user - both are done in particular in text form, which has the advantage that content can be easily copied and sent to the user, for example via SMS or E -Mail can be transmitted to a device, and that the information in text form can be easily searched and specific information of interest to the user can be easily found again.

In contrast to the prior art described above US 2016/0019892 A1 According to the invention, an artificial intelligence module is used to search for relevant information in the audio data and, depending on the meaning of the audio data, a corresponding extraction of the audio data is carried out, which in turn is improved in quality and effectiveness through appropriate processing (summary, verification, etc.). can. The function of the artificial intelligence module advantageously ensures that relevant information is extracted. This reduces the amount of data required to store relevant information for later use. The artificial intelligence module is also advantageously able to improve the information recorded, in particular to verify it, for example by verifying the information from the audio data using corresponding search engine queries via an Internet search.

According to an advantageous embodiment, the artificial intelligence module is designed to supplement and/or verify, at the user's command, the information extracted as relevant by an Internet query and/or a query from a local database, and the supplemented and/or to make verified information available to the user in text form.

Keywords in the semantic sense in the audio data can therefore be supplemented with extended information from the Internet or local databases and form coherent information in the overall picture. The linguistic information from the audio data is thus enriched with information from other sources and allows better information and information storage for the future for the user if he later wants to access the relevant data in a telephone conversation or from a radio signal.

According to a further advantageous embodiment, the artificial intelligence module is designed to store the information in text form on a permanent storage unit so that the information is available to the user at any later time.

According to a further advantageous embodiment, the artificial intelligence module is designed to store the information in text form on a permanent storage unit in compressed form, but in searchable text form.

According to a further advantageous embodiment, the temporary storage unit is designed to store audio data from different sources, with the artificial intelligence module being designed to semantically link the audio data from different sources to each other for the extraction of relevant information and thereby supplement the audio data from the source to which the user's command relates with the audio data from the other sources and/or use the audio data from the other sources to determine the relevance of the audio data from the requested source.

If the user uses a command to refer to a recent telephone conversation or announcement from public radio, in this embodiment the relevance of the audio data can be determined by the artificial intelligence module as to what other audio data is available from other sources. Once the relevance has been determined, the data deemed relevant can be advantageously supplemented by that from other sources.

According to a further advantageous embodiment, the temporary storage unit is designed to store audio data from different sources, the artificial intelligence module being designed to check the relevance of information from the individual sources and, if there is overlapping relevance, to remove the information above a predetermined limit value to link the audio data from the various sources, regardless of which source the user's command refers to.

According to this embodiment, the information from the various sources is checked for relevance in combination and overlaps are compared with a predetermined limit value. If this limit is exceeded, the content in the audio data from the various sources is assumed to be connected and the information is thus linked in order to further increase the relevance of the extracted data.

According to a further advantageous embodiment, the various sources include at least one of the following: vehicle radio, telephone, data stream from a microphone installed in the interior of the vehicle for recording voice inputs such as commands from the user;

Another aspect of the invention relates to a vehicle with a system as described above and below.

Advantages and preferred developments of the proposed vehicle result from an analogous and meaningful transmission of the statements made above in connection with the proposed system.

Further advantages, features and details emerge from the following description, in which at least one exemplary embodiment is described in detail - if necessary with reference to the drawing. Identical, similar and/or functionally identical parts are provided with the same reference numerals.

• 1: Ein Fahrzeug mit einem System zum Ausgeben von Daten in Textform aus Audiodaten des Fahrzeugs gemäß einem Ausführungsbeispiel der Erfindung.

It shows:

• 1 : A vehicle with a system for outputting data in text form from audio data of the vehicle according to an embodiment of the invention.

The representations in the figure are schematic and not to scale.

1 shows a system 1 for outputting data in text form from audio data in a vehicle 3, having a receiver unit 5 for receiving audio data and for acoustically playing back the audio data. In the present case, this concerns public radio from a well-known radio station, whose audio data is output by a loudspeaker unit of the vehicle 3. A temporary storage unit 7 in the vehicle 3 continuously records the audio data from the radio in a ring buffer and temporarily stores this audio data. In this example, the driver of vehicle 3 hears on the currently playing radio station that a donation account is being set up for people in a disaster area, and the associated account number is given by the radio presenter. This piques the interest of the driver of vehicle 3, who would like to donate to the people in need. In order to get the account number in text form, he speaks a predetermined command “save data” to an input unit 9 of the vehicle 3 that is receptive to voice commands. This command is sent to an artificial intelligence module 11, which, based on the specified command on the input unit 9, calls up the audio data stored in the temporary storage unit 7 and analyzes it using artificial intelligence functions and extracts relevant information such as the account number. At the same time as the radio announcement, however, the driver of the vehicle 3 received a telephone call via the hands-free system of the vehicle 3, which is also temporarily stored in its own ring buffer by the storage unit 7, with the participant outside the vehicle 3 commenting on the current crisis during the telephone call and provided the driver with an alternative donation account. Because of the correspondence in content, the artificial intelligence module 11 links both numbers of the donation accounts into a relevant data set of information and, after verification of both account numbers via a fully automatic Internet search, offers both account numbers to the driver of the vehicle together with additional information such as bank sort codes,

Recipient organization and various information about the donation organizations via email to the driver.

Although the invention has been illustrated and explained in detail by preferred embodiments, the invention is not limited by the examples disclosed and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a large number of possible variations exist. It is also to be understood that exemplary embodiments are truly examples only and should not be construed in any way as limiting the scope, application, or configuration of the invention. Rather, the preceding description and the description of the figures enable the person skilled in the art to concretely implement the exemplary embodiments, whereby the person skilled in the art can make a variety of changes with knowledge of the disclosed inventive concept, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Reference symbol list

1

system

3

vehicle

5

Receiver unit

7

Storage unit

9

Input unit

11

Artificial intelligence module

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2016/0019892 A1 [0004, 0012]

Claims (8)

System (1) for outputting data in text form from audio data in a vehicle (3), comprising a receiver unit (5) for receiving audio data and for acoustically playing back the audio data, a temporary storage unit (7) for temporarily storing the audio data, an input unit (9) for recording input from a user in the vehicle (3), the temporary storage unit (7) being designed to store a data stream of audio data for a specific period of time and to delete it after the specific period of time has elapsed, characterized by a module of the artificial Intelligence (11), wherein the artificial intelligence module (11) is designed to analyze the audio data stored in the temporary storage unit (7) using artificial intelligence functions when a predetermined command is received at the input unit (9) and to provide relevant information extract and, at the user's command, make the extracted information available to the user in text form or save it. System (1) after Claim 1 , wherein the artificial intelligence module (11) is designed to supplement and/or verify the information extracted as relevant by an Internet query and/or a query from a local database following the user's command, and the supplemented and/or or verified information to the user in text form. System (1) according to one of the preceding claims, wherein the artificial intelligence module (11) is designed to store the information in text form on a permanent storage unit so that the information is available to the user at any later time. System (1) after Claim 3 , wherein the artificial intelligence module (11) is designed to store the information compressed in text form on a permanent storage unit, but in searchable text form. System (1) according to one of the Claims 1 until 4 , wherein the temporary storage unit (7) is designed to store audio data from different sources, wherein the artificial intelligence module (11) is designed to semantically link the audio data from different sources to extract relevant information and thereby extract the audio data the source to which the user's command refers with the audio data from the other sources and/or using the audio data from the other sources to determine the relevance of the audio data from the requested source. System (1) according to one of the Claims 1 until 4 , wherein the temporary storage unit (7) is designed to store audio data from different sources, the artificial intelligence module (11) being designed to check the relevance of information from the individual sources and, if there is overlapping relevance, above a predetermined limit to link the information from the audio data from different sources, regardless of which source the user's command refers to. System (1) according to one of the Claims 5 until 6 , wherein the various sources include at least one of the following: vehicle radio, telephone, data stream from a microphone installed in the interior of the vehicle (3) for recording voice inputs such as commands from the user; Vehicle (3) with a system (1) according to one of the preceding claims.

Images