DE102020214556A1 - Communication system for a vehicle for dealing with an occupant's sleep disorder - Google Patents

Abstract

Method for operating a communication system (20) for a vehicle (1) with at least one input device (22) for entering at least one instruction relating to at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system (20) is suitable and intended to cause execution of the instruction according to the input by providing an execution signal. According to the invention, a state of consciousness, in particular a sleeping state, of the occupant (10) is recorded and/or determined when the instruction is entered and the communication system (20) makes an assessment based on the state of consciousness, in particular the sleeping state, as to whether it is a through the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder, is an input.

Description

The present invention relates to a method for operating a communication system for a (motor) vehicle and also to a communication system for a (motor) vehicle. The communication system has at least one input device, such as a man-machine interface, for inputting at least one instruction relating to at least one vehicle function of the vehicle by an occupant of the vehicle. In particular, the present invention relates to taking into account an input by an occupant during a sleep state, in particular an input caused by a sleeping disorder of the occupant. In particular, the present invention relates to a procedure for sleepwalking, movement during sleep or talking during sleep of a vehicle occupant.

Vehicles are currently usually equipped with a communication system which is suitable and intended to cause the instruction to be executed in accordance with the input by providing an execution signal. As a result, the execution of the instruction can be implemented and, for example, a driving function can be changed, activated or a function can be set.

Methods are known from the prior art in which a sleeping state of a vehicle occupant is identified, for example to warn a vehicle driver if fatigue occurs or to increase the driving comfort of the vehicle occupant.

In the WO 2018/226220 A1 describes a system in which a sleep state of a vehicle occupant is identified. For this purpose, a vehicle movement is compared with an occupant movement. Based on the identified sleep state and vehicle location, activation of a predetermined vehicle system can be performed. Furthermore, speech recognition systems are provided, also in combination with systems for detecting facial expressions or lip movements, in order to identify the awake state of the occupant based on this.

From the WO 2019/122414 A1 describes a vehicle control system with physiological movement recognition, such as a gesture or breathing, for example for input control for an entertainment system, a navigation system or a system for autonomously driving a vehicle. In this way, a sleep state can be derived, for example. Based on the physiological movement signal, the vehicle control system can instruct an acoustic output via a loudspeaker, a setting of a glass transparency, an adjustment of a light condition, the movement of the vehicle.

In the DE 10 2016 225 222 A1 describes a method for influencing the course of an interaction between vehicle equipment, e.g. for the output of media content or Internet access, and a vehicle occupant depending on a parameter which is responsible for a physiological or psychological condition, including tiredness, fitness, sadness, joy, the occupants is characteristic. If the user is tired, the brightness of the vehicle interior can be reduced.

In the US 2020/0139992 A1 describes a control method for a vehicle in which a reactivity and wakefulness state of a driver is detected. These states are used for switching between a mode in which the vehicle is controlled autonomously and a manual driving mode of the vehicle.

From the DE 10 2016 213 916 A1 an infotainment system for a motor vehicle is known. This can include VR glasses, in which sleep detection can be integrated. This makes it possible, for example, to take into account that the user wants to listen to music with his eyes closed and that the transmission of a film is stopped when a predetermined period of time is exceeded. For sleep detection, the electrical signals from brain waves, heartbeat, muscle tension as well as movements and breathing sounds such as snoring or other acoustic expressions during sleep can be taken into account. The driver can be woken up taking into account a sleep phase that has been determined.

Particularly in connection with fully automated vehicles (“Level 5”), more and more user scenarios are being considered, in which vehicle occupants can relax or even fall asleep in the vehicle cabin. In particular, usage scenarios are even being designed in which vehicle occupants are brought comfortably from a starting point (e.g. the front door) to a destination (e.g. holiday destination) with a fully automatically (autonomously) driving vehicle, for example overnight, and comfortably during the autonomous drive be able to sleep. In the DE 10 2019 204 201.5 the adaptation of the driving style of an automatically driving vehicle to the sleeping conditions of vehicle occupants is proposed. Further describes DE 10 2020 202 388.3 Procedures for waking up a vehicle occupant in an autonomously driving vehicle while driving. In the DE 10 2017 220 120 is a neurofeedback system described, which is intended to facilitate relaxation in the vehicle cabin to the point of falling asleep and can thereby recognize tiredness and sleep states.

During sleep time, however, many people can also have states in which they "talk in their sleep" (somniloquia), unconsciously carry out active movements or even - in the state of "sleepwalking" (somnambulism) - carry out actions from everyday life in an unconscious state. Such states of vehicle occupants can also occur in the usage scenarios of an autonomously driving vehicle described above, in which people are transported in the vehicle cabin during sleep time.

In the case of the sleep disorder of sleepwalking (or movements during sleep), the affected person partially awakens from deep sleep. Sufferers in this state have little consciousness, but perform actions that are normally only performed in the waking state. These are often simple, repetitive movements such as sitting up in bed, chewing, or tugging at clothing. In addition, however, complex actions such as walking around, cleaning or cooking are also possible. In particular, those affected have their eyes open, but their facial expressions are fixed. In addition, those affected hardly react to environmental stimuli and are difficult to wake up. Sometimes it occurs that those affected speak with murmurs. Those affected cannot remember the episode after waking up. Most sleepwalking occurs in the first third of the night, which is also when most deep sleep occurs.

About 10 - 30% of children and adolescents with at least one phase of sleepwalking and 15% are regularly affected by the sleep disorder of sleepwalking. Boys and girls are equally affected. Sleepwalking usually begins between the ages of four and eight. The sleepwalking disorder usually resolves itself in adolescence. Sleepwalking can occur in connection with other disorders that occur during sleep, such as pavor nocturnus (night terrors - intensely excited startle during the deep sleep phase without actually and consciously waking up), restless legs Syndrome, trouble breathing during sleep and/or bedwetting.

It is estimated that seven percent of adults experience at least occasional sleepwalking and one to two percent experience chronic sleepwalking. Compared to children, chronic sleepwalking adults tend to have a longer-lasting disorder and occur more frequently. In addition, a clinic for sleep medicine reports (retrieved from https://www.ksm.ch/de/sleep-diseases/parasomnie/symptome-und-causes-in-detail) that sleepwalking adults are more prone to aggressive actions and acts of violence (up to to the injury of a bed partner), the results of which they then have to take note of stunned after waking up.

Although most sleepwalking, especially in children and adolescents, is harmless and goes away on its own before puberty, four precautionary measures are recommended by a Swiss clinic for sleep medicine, for example. Objects with which an affected person could injure themselves or others should be locked and therefore out of reach for the person affected. Windows should also be tightly closed and not too easy to open. There should also be a bell on the bedroom door, for example, which reports when you leave the room. Finally, a new environment should be informed about the existence of the sleep disorder of sleepwalking and precautionary measures should also be observed there, because the risk of accidents is particularly high in strange and unfamiliar surroundings.

Another parasomnia, the so-called somniloquia, refers to speaking during sleep, which ranges from the reproduction of undifferentiated sounds to words and entire sentences. Somniloquia occurs in about 50% of children and about 5% of adults (retrieved from https://derschlaffuchs.de/studie-reden-im-schlaf-syntax-und-semantik-vonschlaf quoted-aeusserungen/; see also Arnulf , Isabelle et al., "What Does the Sleeping Brain Say? Syntax and Semantics of Sleep Talking in Healthy Subjects and in Parasomnia Patients", Sleep, vol.40, no.11, 2017, doi:10.1093/sleep/zsx159). Usually, speech during sleep occurs when alternating between different stages of non-REM sleep, but it can also occur during REM sleep, causing dreamed phrases to be spoken aloud.

Sleep disorders such as sleepwalking or somniloquia can affect transport in autonomous vehicles. In a vehicle cabin, for example, sleep disorders such as "sleepwalking", performing movements while sleeping or "talking while sleeping" can have such an effect that people in these states can unconsciously activate and/or deactivate vehicle functions or, for example, change navigation destinations simply by speaking. This becomes all the more important as more gesture and/or language-based inter actions between the vehicle and the vehicle user are possible.

Methods for operating a communication system and communication systems for vehicles that are currently known from the prior art have the disadvantage that an input from an occupant triggered by a sleep disorder can lead to a dangerous situation for the occupant himself or for other occupants.

The object of the present invention is to overcome the disadvantages known from the prior art and to provide a method for operating a communication system and a communication system which prevents a dangerous situation caused by an occupant's sleep disorder and at the same time has a high level of reliability the operation of a vehicle function guaranteed by an input from the occupant.

The object is achieved according to the invention by the subject matter of the independent claims. Advantageous embodiments and developments of the invention are the subject matter of the dependent claims.

In a method according to the invention for operating a communication system for a (motor) vehicle, in particular for taking into account the sleeping state of an occupant, the communication system has at least one input device for inputting at least one instruction relating to at least one vehicle function of the vehicle by an occupant of the vehicle. In this case, the communication system is suitable and intended to cause the instruction to be executed in accordance with the input by providing an execution signal. The vehicle is preferably a vehicle that can be operated at least partially autonomously.

In the method according to the invention, a state of consciousness, in particular a sleeping state, of the occupant is recorded and/or determined, in particular when the instruction is entered, and the communication system makes an assessment based on the state of consciousness, in particular the sleeping state, as to whether it is a through the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder, is an input. This is in particular the state of consciousness, in particular the sleeping state, of the occupant (essentially) at the time of the input. A neurological disorder can in particular be a disorder caused or triggered by a disorder of the nervous system. For example, a neurological disorder can be a sleep disorder. In addition, it is also possible that the neurological disorder is a (particularly short-term) functional disorder and/or (neurological) disease of the brain, such as epilepsy.

A state of consciousness of the occupant can be understood to mean a state of the occupant (in particular when he is awake) in which the occupant, for example in a state of wakefulness or in a state of wakefulness (vigilance), has perceptions in the sense of a conscious experience (such as, for example, sensory perception, action intention) are possible. A state of consciousness can also be understood to mean that the occupant is not in a state of wakefulness. In addition, a state of consciousness can be understood to mean a state of sleep and/or a trance state.

The present invention is described below in relation to the detection and/or determination of a sleep state (as a state of consciousness) and a sleep disorder as a neurological disorder. However, the features described below in connection with a sleep state and/or a sleep disorder can also be transferred in an equivalent manner to a state of consciousness or a neurological disorder and are also to be regarded as disclosed in this context.

A sleep disorder can involve, in particular, movements during sleep and/or sleepwalking and/or somniloquia (talking during sleep).

Based on this assessment, the communication system can, for example (preferably for the duration of a state of consciousness, in particular a sleep state and/or a specified sleep phase and/or for the duration of an episode of a sleep disorder) block (in particular temporarily) an input option and/or change a initiate input modality. This is preferably a blocking of an input option and/or a change in an input modality (only) for this occupant and in particular not for another occupant. A change in an input modality can be understood, for example, as an additional requirement for a (further) input confirmation by the occupant.

In other words, the communication system checks whether the occupant was in a sleeping state at the time of the input. Based on this, the communication system can assess whether the occupant did not consciously make the input, for example because he was in a sleep state and/or in a specific - subsequently executed - sleep phase and in this sleep state and/or in this sleep phase no conscious action, in particular no conscious input, could do.

A probability value is preferably determined (by the communication system) which is characteristic of a probability of the existence of a sleep state and/or of the existence of a (specified) phase at a specified point in time and/or period of time and particularly preferably at a point in time at which the occupant makes an input is.

This probability value is preferably compared, in particular by the communication system and preferably by at least one processor of the communication system, with a threshold value that is specified in particular and/or can be input into the communication system. The threshold value can be predefined and stored in a storage device. The threshold value can preferably be specified in an occupant-specific manner.

Based on the result of this (or several such comparisons carried out), it is preferably assessed (in particular by the communication system) whether the occupant was in a state of consciousness and/or sleeping state, in particular at the time of the input, in which the occupant was not (sufficiently) consciously decides about his actions.

The assessment preferably takes into account whether at least one or one further feature and/or indication of the presence of a sleep disorder is present, in particular at the time of the input. The respective feature can preferably be combined with the presence of a specific sleep phase (explained in more detail below), such as REM sleep, non-REM sleep, and/or a transition between two sleep phases and/or a change between different phases of the non-REM sleep sleep are taken into account. This makes it possible to take into account features that typically occur in certain sleep phases.

Such a feature can be a usual side effect of a sleep disorder that is detected and/or determined (e.g. by the detection device and/or a further detection device). Such a side effect can be, for example, in the case of the sleep disorder of sleepwalking, as mentioned at the beginning, a further sleep disorder, such as pavor nocturnus, restless legs syndrome, breathing disorders during sleep and bedwetting at night.

A feature to consider in the assessment, particularly in the sleep disorder of somniloquia, may be a detection of undifferentiated sounds and/or nonverbal utterances (murmurs, shouts, whispers, laughter) and/or scattered words (which an inmate may hear during sleep). expels sleep). One study found that 59% of the utterances in an episode of somniloquia are non-verbal (see Arnulf, Isabelle et al., "What the Sleeping Brain Say? Syntax and Semantics of Sleep Talking in Healthy Subjects and in Parasomnia Patients", Sleep, vol. 40, no. 11, 2017, doi:10.1093/sleep/zsx159). A feature can preferably be a ratio of non-verbal utterances to verbal utterances by the occupant, which is particularly characteristic of the presence of an episode of somniloquia.

The feature can preferably be the frequency of specified words. The study mentioned above was able to determine that "No" was the most frequently used word and negative sentences occurred at 21.4% (more frequently in non-REM sleep). The feature can preferably be a characteristic variable for a frequency, a specific sentence structure (questions, subordinate clauses). The feature can preferably be a characteristic variable for a semantic analysis of the occupant's verbal utterances, such as the use of swear words (more frequently in non-REM sleep), verbal abuse, insults, judgments, cursing. The above study found that as many as 9.7% of the sentences in an episode of somniloquia contained profanity.

One or more of the features can be general or occupant-specific features. In the case of a known somniloquia of an occupant, for example, an evaluation of the syntax or semantics of the occupant in episodes that have already occurred can be taken into account for the feature. Such features and/or data can be stored in a user profile.

An indication can be a risk assessment for the occurrence of a sleep disorder. Such an indicator can be the age of the occupant and/or the frequency with which the occupant has already had sleep disorders (in a previous period of time).

In this way, advantageously, inputs made and intended by the occupant can be differentiated from inputs that the occupant did not want, and accidental input of instructions can thereby be avoided.

An instruction relating to a vehicle function is understood in particular as an activation, deactivation and/or operation of a vehicle function, a change in a function setting with regard to a vehicle function and/or a specification and/or change in data input for a vehicle function.

A vehicle function can be a function of a vehicle component of the vehicle. The vehicle component is preferably selected from a group which includes a locking system of a vehicle door and/or a vehicle window, a vehicle closing system, a roof removal mechanism, a sunroof mechanism, an infotainment system, an entertainment system, a navigation system, a vehicle safety system (e.g. a braking system), a windshield wiper mechanism, a System for driving the vehicle and / or a comfort system to increase the driving comfort of an occupant and combinations thereof.

An input device can be a gesture and/or facial expression and/or voice-based input device, an activity tracking device for detecting movement of an occupant (such as a wrist), a haptic operating element (such as a switch, a lever), a touch-sensitive Operating element, such as a touch-sensitive display, a graphics-based input device by means of visual interaction elements, an input device based on a determination of a physiological state and/or a perception-controlled input device.

The input device can preferably be an interaction device. Input from an occupant may be an interaction between the occupant and the input device.

With regard to a preferred activity tracking device, reference is made in particular to the publication DE 10 2015 120 803 A1 , the content of which is hereby incorporated into this application. An activity tracking device is described therein, which detects a movement of the occupant's wrist and is preferably able to exchange this data with the communication system and/or a vehicle data processing system. The communication system and/or the vehicle data processing system preferably generates at least one output signal based on the data from the activity tracking device. For example, a wiper mechanism or a sunroof mechanism may be operated based on signals indicative of a detected long wipe movement.

With regard to a preferred gesture-based input device that is suitable and intended for a gesture-based interaction between a vehicle and a user or occupant, reference is made in particular to the application DE 10 2016 121 032 A1 , the content of which is hereby incorporated into this application. An input device described therein enables the execution of vehicle functions to be initiated with gestures.

With regard to a preferred input device based on a determination of a physiological condition, reference is made in particular to the patent application DE 10 2017 101 343 A1 , the content of which is hereby incorporated into this application. Physiological data of a vehicle occupant can be recorded by means of the input device and this physiological data can be used to determine one or more physiological states of one or more vehicle occupants, in order preferably to use them to control a vehicle.

With regard to a preferred method for monitoring a state of alertness of a person by a sensor device for detecting at least two body expression features of the person, wherein at least one of the body expression features is a non-eye-based body expression feature, reference is made in particular to the published application DE 10 2017 216 328 B3 , the content of which is hereby incorporated into this application.

With regard to a preferred input device based on facial expressions and/or gestures, reference is made in particular to the published application DE 10 2017 216 837 A1 , the content of which is hereby incorporated into this application. Such an input device preferably has a plurality of devices and a human-machine interface (HMI) for mimicking and/or gesturing actuate a function of a device of the vehicle. A camera for capturing a specific occupant of the vehicle and in particular a control unit connected to the camera is preferably provided.

With regard to a preferred gesture-based input device, reference is made in particular to the published application DE 10 2018 208 866 A1 , the content of which is hereby incorporated into this application. It describes a method for recognizing an input from a person using gestures.

With regard to a preferred pointing gesture-based input device, reference is made in particular to the disclosure DE 10 2018 211 624 A1 , the content of which is hereby incorporated into this application. It describes a method and a user interface for capturing an input using pointing gestures.

With regard to a preferred speech-based input device, reference is made in particular to the published application DE 10 2019 119 171 A1 , the content of which is hereby incorporated into this application. It provides voice recognition for vehicle voice commands.

In this case, the input device can be part of the vehicle equipment and in particular can be arranged firmly (not destructibly detachable) on the vehicle. It is also conceivable that the input device can be a mobile end device (e.g. a smartphone, tablet or laptop, e.g. belonging to an occupant), which can be removed from the vehicle and which, in particular, can be connected to the vehicle via a wireless and/or wired data communication connection Communication system can be connected.

In a preferred method, the state of consciousness, in particular the sleep state, and/or the assessment of whether it is an input caused by the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder, is used when executing the instruction, in particular by means of a modification and/or blocking of execution and/or delay of execution.

In other words, the sleep state of the occupant recorded and/or determined (when inputting) can be taken into account in an assessment and/or determination as to (whether and) in what form an instruction relating to the vehicle function is initiated. This is in particular the sleep state of the occupant (essentially) at the time of the input.

This offers the advantage that the risk of an occupant inadvertently entering an instruction and thereby inadvertently initiating a vehicle function is significantly minimized.

In particular, the present invention proposes that the sleep state and/or the presence of a sleep disorder of a vehicle occupant can be recognized and then input options for this vehicle occupant (depending on the sleep state) for example gesture or speech recognition and/or active operation of vehicle functions or input options blocked and/or delayed (particularly temporarily) via switches, levers, displays and the like. It preferably extends over the entire duration of the sleep of the occupant and/or over the entire duration of an episode of a speech disorder.

If, for example, interactions based on gestures, facial expressions, and/or speech are provided between the vehicle and the occupant or user, for example in order to activate functions such as the windshield wipers with a gesture, such interactions could occur, for example, when sleepwalking, performing movements while sleeping or be carried out unintentionally while talking in your sleep.

The proposed method advantageously ensures that vehicle occupants of a vehicle, in particular an autonomous vehicle, are unable to activate and/or deactivate any vehicle functions and/or cannot change any function settings when they are asleep, in particular in states of sleepwalking, performing movements in their sleep or sleep-talking and/or cannot specify or change data entries for functions. Preferably, touch interaction with a touch screen can be restricted.

With regard to deactivating vehicle functions, reference is made to the disclosure DE 10 2009 018074 referenced, in which a system is described in which a current driving situation is assessed and an incoming call is not put through to a driver if the current driving situation does not allow it. The content of this disclosure is hereby included in this application.

A particularly momentary state of consciousness, particularly a sleeping state, of an occupant is preferably detected by a detection device, particularly (essentially) when the occupant makes an entry. It is also conceivable that in addition or as an alternative to a sleeping state of the occupant, a waking state is detected and/or determined and a sleeping state of the occupant is preferably determined from the waking state of the occupant.

The state of consciousness, in particular the sleeping state, of an occupant is preferably recorded at regular time intervals (by the recording device). However, it is also conceivable that a sleep state of the occupant at a later and/or earlier point in time and/or between two measurement times is derived and/or determined based on at least one recorded sleep state and preferably on the basis of a plurality of recorded sleep states of an occupant.

It is conceivable that based on a predefined state of consciousness, in particular sleeping state, a predicted state of consciousness, in particular sleeping state, of the occupant is determined, for example at the time of an input by the occupant (in particular by the detection device and/or the communication system).

The detection device can be a vehicle-internal detection device. However, it is also possible for an external detection device to be involved, in particular a mobile detection device and/or a detection device which can be moved relative to the vehicle and which is or can be connected wirelessly and/or by wire to the communication system for data exchange.

However, it is also conceivable that the detected sleep state of the occupant is a signal transmitted to the vehicle and/or the communication system. This can be a predetermined input from the occupant and/or another occupant. For example, the occupant may input information regarding an intended sleep, for example while driving and/or during a specific section of the route.

In a preferred method, a sleep phase and/or a state of tiredness and/or a state of alertness of the occupant is recorded and/or determined upon input and taken into account when executing the instruction. The detection device (or a different detection device) preferably detects a sleep phase and/or a state of tiredness. A sleep state is preferably present when the occupant is not alert.

A state of consciousness of an occupant is preferably recorded and/or determined on the basis of a brain wave measurement (for example via EEG sensors) or on the basis of a brain current image (EEG). With the help of an electroencephalographic measurement (EEG), for example, an epileptic seizure, caused by the neurological disorder of epilepsy, of an occupant can be detected.

An occupant's sleeping state and/or a transition from a sleeping to a waking state (and/or vice versa) based on a (sensed and/or determined) physiological movement and/or breathing movement and/or a heartbeat and/or breathing rate is preferred and/or a limb movement and/or a gesture movement and/or a body movement and/or brain wave measurement (e.g. via EEG sensors) or brain wave imaging (EEG) and/or heart rhythm (ECG) and/or oxygen content of the blood (pulse oximetry) and/or or body temperature and/or respiratory flow (mouth and/or nose) and/or muscle tension (EMG) and/or leg movement and/or eye movement (EOG) and/or body position and/or body noise and/or an eyelid beat (in particular by the detection device) or combinations thereof are recorded and/or determined.

The detection device preferably has at least one sensor and/or at least one measuring device, in particular for detecting and/or determining a state of consciousness, in particular a sleeping state, and/or a waking state and/or a process of waking up and/or falling asleep of an occupant, which is selected from a group that includes at least one EEG sensor (preferably a large number of EEG sensors), in particular for (direct) measurement of brain waves, a microphone, a camera, a motion sensor, an acceleration sensor, a pressure sensor, an eyelid sensor, a respiratory rate sensor, a temperature sensor, a heart rate sensor , pulse oximetry sensor, and combinations thereof.

Preferably, at least one sensor and preferably a large number of these sensors of the detection device and/or at least part of the detection device can be arranged in a contact area (to the occupant) of the bed area and/or a seat area of the vehicle.

It can be measured and, if necessary, the transition from a sleeping to a waking state can be deduced from combinations of different sensors: brain current image (EEG), heart rhythm (ECG), oxygen content of the blood (pulse oximetry), body temperature, respiratory flow (mouth and/or nose). ), respiratory movement, muscle tension (EMG), leg movement, eye movement (EOG), body position, body sounds.

The EEG process (electroencephalography, recording of voltage fluctuations on the surface of the head) has been around since around 1930 known and is z. B. used in sleep laboratories to recognize the sleep phases or phases of tiredness of people (see e.g. https://de.wikipedia.org/wiki/Elektroencephalografie).

The detection device (particularly at least partially and preferably mobile in its entirety) preferably has sensors for taking an EEG measurement to detect a tiredness and/or sleep state and/or state of consciousness. These sensors are preferably arranged on a pair of glasses for the occupant. The detection device and/or the sensors for carrying out an EEG measurement can be arranged in virtual reality or augmented reality glasses and can preferably be used as such in a vehicle.

The (in particular at least partially and preferably entirely mobile) detection device, which particularly preferably has sensors for performing an EEG measurement, is preferably arranged stationary on the vehicle and in particular on a headrest and/or a head area of a lying area. The detection device is preferably suitable and intended for carrying out an EEG measurement over a distance (for example from headrest to head), so that an occupant's state of alertness and/or tiredness and/or sleep can be detected.

With regard to a preferred detection device and a preferred method for detecting and/or determining and/or monitoring an alertness and/or tiredness and/or sleep state of an occupant (there driver) of a vehicle, reference is made in particular to the published application DE 10 2015 204 247 A1 , the content of which is hereby incorporated into this application. For example, a driver's viewing direction can be detected and it can be checked whether the driver has opened his eyes. This is used, for example, to detect drowsiness in order to prevent a driver from falling asleep for a second.

The detection device preferably has at least one acceleration sensor, in particular an accelerometer, which is preferably positioned in the vicinity of the occupant, for example on a vehicle seat or a vehicle bed, next to or on the occupant. The detection device and/or the acceleration sensor is/are preferably arranged in relation to the occupant in such a way that it can record movements of the occupant. The acceleration sensor can be arranged in a mobile (end) device, such as a smartphone. A sleep phase alarm clock is preferably used as the detection device.

A smartphone can be used as a recording device together with a so-called “digital sleep phase alarm clock” (“Sleep Cycle Clock”) as a smartphone app (see also https://de.wikipedia.org/wiki/Schlafphasewecker). Such applications usually work as follows: The smartphone with the corresponding app (e.g. "Sleep as Android") is placed on the bed next to the desired person. The smartphone then uses its built-in accelerometer to record movements of the sleeping person. As a result, it should and can preferably recognize in which sleep phase the person is and whether one is asleep, i.e. whether one is in a sleep state. You go through about three different types of sleep every night: light sleep, deep sleep and what is known as REM (rapid eye movement) sleep. If you are woken up in the middle of a deep or REM phase, you will find it difficult to get up. In between, however, there are so-called "almost awake moments" in which it is particularly easy to wake up. These moments are preferentially recognized by the app.

With regard to a preferred detection device and a preferred method for detecting and/or determining a sleeping state of an occupant of a vehicle, reference is made in particular to the published application DE 10 2016 213 916 A1 , the content of which is hereby incorporated into this application. This application describes how, in connection with the use of virtual reality glasses in the vehicle cabin, communication and sleeping in fully automated vehicles are supported in a comfortable manner. Sleep states are recognized and infotainment and/or virtual reality and/or augmented reality systems in the vehicle cabin are adapted to the sleep phases of the occupant.

The detection device is preferably suitable and intended for analyzing the sleep phases of a person or an occupant with an analysis of their brain current curves and the like. A suitable wake-up process is preferably set in a phase of light sleep before the last possible wake-up time. Such solutions are known, for example, under the name "SleepSmart" (www.axonlabs.com).

With regard to a preferred detection device and a preferred method for detecting and/or determining a sleep state and/or for determining a sleep state of an occupant of a vehicle, including their movements and heart rate, reference is made in particular to the published application EP 2712300 A1 , the content of which is hereby incorporated into this application.

With regard to a preferred detection device and a preferred method for detecting and/or determining a sleeping state of an occupant of a vehicle, reference is made in particular to the published application DE 10 2008 038022 A1 , the content of which is hereby incorporated into this application. A wake-up device is proposed therein to enable sleeping that is as optimal as possible and gentle waking of a person sleeping in a motor vehicle in particular. The wake-up device should be equipped with a sensor device that detects a movement of the sleeping person and thus preferably a so-called waking phase. Furthermore, the alarm device should be equipped with a lighting device which, when activated, continuously illuminates the area surrounding the sleeping person. Furthermore, the alarm device should be equipped with a time measuring device for setting a wake-up time or a wake-up time period, the sensor device being designed such that it activates the lighting device when it detects that the sleeping person is awake at the wake-up time or within the wake-up time period.

With regard to a preferred detection device and a preferred method for detecting and/or determining a state of tiredness and/or a state of sleep of an occupant of a vehicle, reference is made in particular to the published application DE 10 2017 220 120.7 , the content of which is hereby incorporated into this application. A neurofeedback system is described therein, which is intended to facilitate relaxation up to the point of falling asleep, preferably in the vehicle cabin, and can thereby recognize states of tiredness and/or sleep.

The detection device is preferably suitable and determines (in particular for determining and/or detecting a sleep state and/or a sleep phase) a sleep phase of the occupant, in particular from a predetermined group of sleep phases, and/or to detect. The predefined group of sleep phases is preferably stored in a memory device. The predefined group of sleep phases preferably has at least two different sleep phases, preferably at least three and particularly preferably at least five and in particular precisely five sleep phases.

The occupant may be in a sleeping state if a predefined sleep phase or one (each) of the predefined sleep phases of the occupant is present or can be detected and/or determined.

A state of consciousness, in particular a sleep state, a sleep phase is preferably recorded and/or determined by means of a brain wave measurement and/or an EEG (electroencephalography, recording of the voltage fluctuations on the surface of the head), in particular by deriving the summed cortical potential fluctuations and preferably by an evaluation and/or identified.

It is preferably determined whether the occupant is in a sleep state, and in particular in the so-called non-REM sleep stages (stages I-III) or in the REM sleep state. Different sleep phases are preferably distinguished from one another by means of an EEG.

In particular, healthy sleep includes five sleep phases. The sleep phases or sleep stages can be recognized from the brain activity. From falling asleep to deep sleep, a person's sleep remains essentially the same. This changes abruptly with the onset of so-called REM sleep or dream sleep. The REM phase is named after the rapid eye movements that are characteristic of this phase of sleep. In this phase, brain activity accelerates significantly and an EEG shows many small spikes. This is similar to falling asleep. Dreams occur especially in the REM phase. Sleep researchers assume that in the REM phase, primarily emotional sensory impressions, but also information, are processed.

The five sleep phases are preferred (particularly within a sleep state), (sleep) stage I (light sleep), (sleep) stage II, (sleep) stage III (transition to deep sleep), (sleep) stage IV ( deep sleep) and REM sleep. The detection device preferably determines whether and which of these sleep phases is present in the occupant.

(Sleep) stage I describes the falling asleep phase, a phase shortly after falling asleep. In sleep stage I, the brain prefers to switch from alpha waves to theta waves (4 to 7 Hz) (in the EEG). Muscle tension is reduced and the conscious perception of the environment slowly disappears. This can be the first few minutes of falling asleep and after, or just after falling asleep. In this phase, the organism sinks further and further into calming and relaxation. The pulse rate slows down, breathing becomes deeper and the number of breaths decreases. This sleep stage I is often experienced as a time of becoming heavy or falling, which is perceived as dream-like. The sleep is still there very superficial and even small disturbances can wake up the sleeper so that he is wide awake again immediately.

In sleep stage II, theta waves continue to appear (in an EEG), so-called sleep spindles and K-complexes are particularly important in this phase. This sleep stage gets progressively longer over the course of an eight-hour sleep, accounting for more than 50 percent of total sleep.

In sleep stage III (transition to deep sleep), delta waves (0.1 to < 4 Hz - slow waves with high amplitude) occur (in an EEG) and come to the fore in particular (20 - 50 percent of the measured brain waves) . Muscle tension continues to decrease.

In sleep stage IV (deep sleep), delta waves occur, which now account for more than 50 percent of the measured brain waves. It is the deepest phase of sleep. Sleepers who are woken up during this sleep phase appear correspondingly disoriented and sleepy. During this phase of sleep, sleep disorders such as sleepwalking and talking during sleep occur.

REM sleep (also dream sleep or paradoxical sleep) can be distinguished from the other sleep phases in many respects. The EEG is similar to sleep stage I in that theta waves predominate. Periodic, rapid, directionless movements of the eyeball occur during REM sleep at a frequency of 1 to 4 Hz. Dream reports of awakenings in this phase are significantly more vivid, visual, and emotional than awakenings in other phases. During REM sleep, the skeletal muscles are maximally relaxed, but not the eye muscles. Most of the vegetative functions are activated with an increase in blood pressure, breathing rate and heart rate, as well as increased blood flow to the genitals.

The sequence of the sleep phases is repeated several times a night in healthy people who sleep well. A full cycle takes about an hour and a half - plus or minus 10 minutes. About 50 minutes of this is spent on falling asleep or light sleep. Deep sleep and REM phase make up about 40 minutes of this sleep cycle. At the beginning of the night, deep sleep dominates, towards the end of the sleeping period a sleeper spends more time in the REM phases and prepares for waking up.

The so-called light sleep phase accounts for about half of sleep. It is also known as sleep stage 2. Sometimes sleep stages 1 and 2 are combined into the light sleep phase. In this phase, brain activity is mainly limited to low frequencies. Consciousness is switched off, a sleeper sleeps with relaxed muscles and almost no eye movements.

The deep sleep phase is divided into sleep stages 3 and 4. It is the phase of the deepest physical relaxation. Many studies show that the deep sleep phase is the most valuable phase for physical and mental recovery. In this phase, many people are difficult to wake up. As a result, those who have just been woken up only slowly find their way back into consciousness.

When a person is alert, beta waves can be measured or recorded in an EEG. An occupant's state of alertness can preferably be identified by measuring beta waves (using an EEG).

In a relaxed (or waking) state with eyes closed, alpha waves (8 to 13 Hz) occur. A wakefulness of an occupant is preferably identified via the measurement of alpha waves (by means of an EEG).

An assessment is preferably made as to whether the input is prompted by the sleep state and/or a sleep disorder, depending on which sleep phase and/or which sleep stage is detected and/or is present. an assessment is made as to whether the input is caused by the sleep state and/or a sleep disorder, depending on whether a deep sleep phase and/or a light sleep phase is detected and/or is present. An assessment is preferably made as to whether the input is prompted by the sleep state and/or a sleep disorder, depending on whether REM sleep and/or non-REM sleep is present.

In a further preferred method, at least one vehicle user profile is stored in a memory device of the communication system. Depending on the recorded and/or determined state of consciousness, in particular the sleeping state, of the occupant and/or depending on a predefined vehicle function, it is derived whether the instruction relating to the vehicle function is being carried out and/or whether the vehicle function can be operated and/or activated. An above-mentioned occupant-specific feature and/or indicator is preferably stored in the vehicle user profile.

In vehicle user profiles can be preferably stored as a function of recognized Sleep phases and states of tiredness certain vehicle functions are switched to non-operable and / or activated.

In a further preferred method, depending on the detected and/or determined state of consciousness, in particular the sleeping state, of the occupant, a blocking of the execution of the instruction is released only after the instruction has been additionally confirmed by the occupant. This offers the advantage that although the blocking ensures the security of a conscious input, it is still possible to operate it at any point in time.

The suspension can be a temporary suspension. The blocking can preferably be released after confirmation by the occupant or another occupant.

In a further preferred method, the state of health of the occupant (and preferably also of all other occupants) is monitored, in particular as a function of the state of consciousness, in particular the sleeping state, of the occupant. The state of health of the occupant is preferably monitored while he is asleep.

In the case of vehicle occupants, it should preferably not only be possible to detect a sleep state or even a sleep phase or to detect a sleep phase, but the vehicle occupant should also be monitored with regard to his state of health while he is asleep.

With regard to a preferred monitoring of the state of health of the occupant, reference is made in particular to published application “K 27561” [please replace], the content of which is hereby included in this application. A detection described therein of a health emergency that may occur for a vehicle occupant in the vehicle cabin is preferably used. Advantageously, procedures can thus be implemented in such a way that the vehicle user is endangered as little as possible and/or an emergency program is preferably activated after detection of a health emergency. For example, bio-vital data from vehicle occupants can be recorded and the state of health of vehicle occupants can be monitored, in particular permanently, in the vehicle cabin and, if necessary, an emergency program can be activated.

In a further preferred method, an assistance system supports the occupant in taking precautionary measures against a source of danger present in the vehicle interior with regard to a neurological disorder, in particular a sleep disorder, of the occupant. This offers the advantage that potential sources of danger can be eliminated.

If an occupant already knows in advance (i.e. before using the vehicle and/or possibly with the intention of sleeping there as well) that the occupant is affected by a sleep disorder, such as sleepwalking, supporting precautionary measures can be carried out in a vehicle cabin, to protect yourself or fellow passengers in the vehicle cabin from injury. The risk of accidents for sleepwalkers is known to be high, especially in unfamiliar and unfamiliar surroundings. Items that the sleepwalker could use to injure themselves or others should be locked away so they remain out of reach of the sleepwalker.

In this way, the potential sleepwalker can advantageously be reminded on the one hand and optionally also actively supported with an assistance system to safely stow away objects from the vehicle cabin.

In a further preferred method, at least one object in the vehicle interior is identified by means of an image acquisition device, in particular an interior camera, which represents a potential source of danger with regard to a neurological disorder, in particular a sleep disorder, of the occupant.

The interior camera can (preferably using image processing and object recognition and preferably using algorithms such as AI, machine learning and/or deep learning or the like) help to locate potentially dangerous objects, especially before bedtime, in the vehicle cabin and in particular the vehicle user above them to inform. Suggestions can be made to the vehicle user or the occupant as to how such objects can be safely stowed away (for example in a trunk or in lockable storage spaces which can only be opened when the person is awake). The suggestions for stowage of the objects can be displayed, for example, via animations on a screen in the vehicle and/or communicated acoustically via announcements.

Potentially dangerous vehicle functions during sleepwalking (for example opening the windows or doors of the autonomous vehicle) can be set as not operable by the vehicle user or the occupant themselves—in the case of sleepwalking.

A procedure can preferably be agreed with the vehicle user or the occupant, that the vehicle functions are nevertheless released or activated if the vehicle user or the occupant makes an agreed confirmation, for example mentions a given keyword or answers (given) questions. This offers the advantage that an on-board sleep detection can be bypassed in case it does not work. It is preferably ensured that the vehicle user or the occupant, in particular, can identify a health emergency as well as possible at any time.

In a further preferred method, the communication system provides a notification signal and /or warning signal for, in particular acoustic, output ready.

Such a notification and/or warning method can be activated if, for example, the occupant presents himself to the communication system as a (known) sleepwalker and/or person suffering from another sleep disorder and/or a communication system does so (itself, for example by comparison with a database and/or identification of the occupant).

The notification signal and/or warning signal can be directed and/or output to the occupant and/or to another occupant. In the case of a known sleepwalker, a fellow passenger in the same vehicle cabin (eg spouse) can be warned or awakened at an early stage when the first signs of incipient sleepwalking in the potential sleepwalker are recognizable. The fellow traveler can then take precautions at an early stage or calm down or even wake up the sleepwalker at an early stage. This procedure makes particular sense if the sleepwalker is known to be violently sleepwalking (see, for example, https://www.spiegel.de/gesundheit/psychologie/seltsame-stoerung-wenn-schlafende-zuschlaegern- Werden-a-854389. html).

The present invention is also directed to a communication system for a vehicle, in particular for taking into account a sleeping state of an occupant, comprising at least one input device for inputting at least one instruction relating to at least one vehicle function of the vehicle by an occupant of the vehicle, the communication system being suitable for this and is determined, in response to the input, to cause execution of the instruction related to the vehicle function by providing an execution signal.

According to the invention, the communication system has a detection device for detecting and/or determining a state of consciousness, in particular a state of sleep, of the occupant when the instruction is entered, and the communication system is suitable and intended to make an assessment based on the state of consciousness, in particular the state of sleep, as to whether it is an input caused by the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder.

It is therefore also proposed within the scope of the communication system according to the invention that, depending on a detected or determined state of consciousness, in particular a sleep state, and preferably a sleep phase, an assessment is made as to whether the input made by the occupant is a conscious and/or intended input . This offers the advantage that countermeasures, in particular to prevent the execution, can be initiated before an unintentional input by the occupant is executed.

The communication system is preferably set up, suitable and/or intended to carry out the method described above and all the method steps already described above in connection with the method individually or in combination with one another. Conversely, the method can be equipped with all of the features described in connection with the communication system, either individually or in combination with one another.

The communication system is preferably suitable and intended to take into account the state of consciousness, in particular the sleeping state, of the occupant when executing the instruction, in particular by means of a modification and/or blocking and/or delaying the execution.

The present invention is also aimed at a vehicle, in particular a motor vehicle, comprising a communication system for a vehicle as described above, according to an embodiment. The vehicle interior can be equipped with at least one detection device for detecting the state of consciousness, in particular the sleeping state, of an occupant and/or at least one and preferably a large number of input devices. The vehicle can in particular be a (motorized) road vehicle. Basically, the proposed procedure and the proposed Communication system usable for all automation levels of a vehicle.

A vehicle can be a motor vehicle which, in particular, is a motor vehicle controlled by the driver himself (“driver only”), a semi-autonomous, autonomous (e.g. autonomy level 3 or 4 or 5 (of the SAE J3016 standard)) or self-driving motor vehicle is. Autonomy Level 5 designates fully automated vehicles. The vehicle can also be a driverless transport system. The vehicle can be controlled by a driver or drive autonomously.

The present invention is also aimed at a computer program or computer program product, comprising program means, in particular a program code, which represents or encodes at least the method steps of the method according to the invention and preferably one of the preferred embodiments described and is designed to be executed by a processor device.

The present invention is also aimed at a data memory on which at least one embodiment of the computer program according to the invention or a preferred embodiment of the computer program is stored.

• Darin zeigt:
  • 1 eine schematische Darstellung eines Fahrzeugs mit einem erfindungsgemäßen Kommunikationssystem gemäß einer Ausführungsform.

Further advantages and embodiments result from the attached drawing:

• It shows:
  • 1 a schematic representation of a vehicle with a communication system according to the invention according to one embodiment.

1 shows a schematic representation of a vehicle with a communication system according to the invention according to an embodiment. Reference number 20 schematically designates the communication system for a vehicle 1.

Reference number 10 designates an occupant of vehicle 1. The occupant is in 1 shown embodiment in a lying position and in a sleeping state.

The reference number 22 denotes an input device of the communication system 20. This can be, for example, a mobile terminal device, but it can also be a physical actuating element. The occupant can make an input to the communication system via the input device 22 and thereby transmit and trigger an instruction relating to a vehicle function, for example opening a window.

If the occupant is in a deep sleep phase, in particular the stage IV sleep phase, sleep disorders such as sleepwalking and talking during sleep can occur. If this is the case with the occupant, there is a risk that the occupant (unintentionally) makes an input to the input device 22 during the episode of sleepwalking and thereby (unintentionally) the execution of the execution entered by him at the input device 22 by the communication system 20 is caused.

Reference number 24 designates a detection device, for example a camera, which can determine and/or detect a sleep state of the occupant, and preferably also a sleep phase. Depending on the recorded and/or determined sleep state, the communication system can evaluate or assess the input made by the occupant as an input caused by a sleep disorder (e.g. sleepwalking). Depending on this assessment, the execution of the entered instruction can be initiated, modified, delayed and/or prevented. For example, if the communication system comes to the conclusion that the input was made unconsciously as a result of an episode of insomnia, the communication system can prevent the inputted instruction from being carried out and preferably also block further inputs by the occupant 10 .

The applicant reserves the right to claim all features disclosed in the application documents as essential to the invention, provided they are new compared to the prior art, either individually or in combination. It is also pointed out that the individual figures also describe features which can be advantageous in and of themselves. The person skilled in the art recognizes immediately that a specific feature described in a figure can also be advantageous without adopting further features from this figure. Furthermore, the person skilled in the art recognizes that advantages can also result from a combination of several features shown in individual figures or in different figures.

Reference List

1

vehicle

10

inmate

20

communication system

22

input device

24

detection device

QUOTES INCLUDED IN DESCRIPTION

This list of the documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• WO 2018/226220 A1 [0004]
• WO 2019/122414 A1 [0005]
• DE 102016225222 A1 [0006]
• US 2020/0139992 A1 [0007]
• DE 102016213916 A1 [0008, 0074]
• DE 102019204201 [0009]
• DE 102020202388 [0009]
• DE 102017220120 [0009, 0078]
• DE 102015120803 A1 [0041]
• DE 102016121032 A1 [0042]
• DE 102017101343 A1 [0043]
• DE 102017216328 B3 [0044]
• DE 102017216837 A1 [0045]
• DE 102018208866 A1 [0046]
• DE 102018211624 A1 [0047]
• DE 102019119171 A1 [0048]
• DE 102009018074 [0056]
• DE 102015204247 A1 [0071]
• EP 2712300 A1 [0076]
• DE 102008038022 A1 [0077]

Claims (10)

Method for operating a communication system (20) for a vehicle (1) with at least one input device (22) for entering at least one instruction relating to at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system (20) is suitable and intended for causing the instruction to be executed in accordance with the input by providing an execution signal, characterized in that a state of consciousness, in particular a sleeping state, of the occupant (10) is detected when the instruction is entered and/or is determined and the communication system (20) makes an assessment based on the state of consciousness, in particular the sleep state, as to whether the input is caused by the state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular a sleep disorder. procedure after claim 1 , characterized in that the sleep state is taken into account when executing the instruction, in particular by means of a modification and/or blocking and/or delay in execution. Method according to one of the preceding claims, characterized in that at least one vehicle user profile is stored in a memory device of the communication system, from which, depending on the detected and/or determined sleeping state of the occupant and/or depending on a predefined vehicle function, it is derived whether the instruction in Is carried out in relation to the vehicle function and / or whether the vehicle function can be operated and / or activated. Method according to one of the preceding claims, characterized in that depending on the recorded and/or determined sleeping state of the occupant (10), blocking the execution of the instruction is released only after the instruction has been additionally confirmed by the occupant. Method according to one of the preceding claims, characterized in that the state of health of the occupant is monitored, in particular as a function of the sleeping state of the occupant. Method according to one of the preceding claims, characterized in that an assistance system supports the occupant in taking precautionary measures against a source of danger present in the vehicle interior with regard to a sleeping disorder of the occupant. Method according to one of the preceding claims, characterized in that objects in the vehicle interior are identified by means of an image acquisition device, in particular an interior camera, which represent a potential source of danger in relation to a sleeping disorder of the occupant. Method according to one of the preceding claims, characterized in that the communication system provides a notification signal and/or warning signal for, in particular acoustic, output as a function of the sleeping state of the occupant (10). Communication system (20) for a vehicle (1) comprising at least one input device (22) for inputting at least one instruction relating to at least one vehicle function of the vehicle (1) by an occupant (10) of the vehicle (1), wherein the communication system (20 ) is suitable and intended to cause the instruction in relation to the vehicle function to be executed in response to the input by providing an execution signal, characterized in that the communication system has a detection device (24) for detecting and/or determining a state of consciousness, in particular a sleep state that the occupant has when entering the instruction and the communication system (20) is suitable and intended to make an assessment based on the state of consciousness, in particular the sleep state, as to whether it is a state of consciousness, in particular the sleep state, and/or a neurological disorder, in particular re a sleep disorder that is induced typing. Vehicle (1), in particular motor vehicle, comprising a communication system (20) for a vehicle according to the preceding claim.

Images