Effective Alerts for Autonomous Solutions to Aid Drivers Experiencing Medical Anomalies

There are many factors that may occur while driving that can increase the risk of an accident. Many are preventable, but how can situations that cannot be foreseen be prepared for? This is a reality for people with certain medical conditions. In a study from the National Household Transportation Survey, 1.3% of all reported drivers had a motor vehicle accident caused by their medical condition (Hanna 2009). If there were autonomous technologies in these vehicles that could pull over the vehicle for a driver when a medical emergency occurs there is a potential for less of these accidents. Within this study, the focus is to determine the most effective alarm to alert the driver experiencing a medical emergency that a self-driving system is pulling over the vehicle. The emergency in our study has been defined as a medical anomaly that leaves the driver unable to maintain control of the vehicle in order to pull over safely.

If a method to pull over the vehicle in a manner that is safe and comfortable for the user can be defined, it could create opportunities for many people. It may change the mind of those who have given up driving due to medical conditions. Corey Harper and the other authors of their article have shown how autonomous vehicles could increase transportation for the elderly, those with medical restrictions on driving, and just general non-drivers (Harper et al. 2016). With the response system this study working to define it may be a way to have those who decided to stop driving, like in those Harper’s study, feel more comfortable getting behind the wheel. This could also be useful for those that didn’t know that their medical conditions make it unsafe for them to drive. In the article, “Medical Restrictions to Driving: The Awareness of Patients and Doctors” the authors evaluate that many patients have difficulty estimating their ability to drive (Kelly et al. 1998). In addition, their study showed that not all doctors and appropriately advised these patients who are having trouble deciding if they should drive. This system could be what keeps the unknown patient from having a medical-related car accident.

In this study, an open source driving simulator was used to create a driving environment. With this, several different methods were created to alert the user, and then pull the simulated car over with hopes to identify an effective alert system for these safety procedures. An effective alert during the autonomous methods will ensure that the user can safely identify that the vehicle is being pulled over for them without causing additional stress. Participants in the study drove a simulation using a monitor and a connected steering wheel and pedals. After the simulation, a survey was given to inquire the participant’s opinion for each of the alerts that were shown.

The decision of whether or not to drive has been investigated by several researchers with a focus on those with restrictive medical conditions. Rosemary Kelly, Timothy Warke, and Ian Steel in their article, “Medical Restrictions to Driving: the Awareness of Patients and Doctors” research the awareness of doctors and their elderly patients on their knowledge of their medical restrictions on driving (Kelly 1998). This study gave a questionnaire to 150 patients and 103 of them thought that they were eligible to drive. However, 48 of those 103 patients had a medical restriction on driving. The authors of this study have concluded from the results that patients have a hard time knowing if their medical condition inhibits them from driving. Additionally, they report the doctors at the clinic researched had poor knowledge on medical restrictions for driving. This gave concerns about the knowledge of other medical professionals to correctly advise patients to not drive.

Another study focuses on how patients with intractable epilepsy decide to drive even when it is discouraged by a medical professional. In Noah J. Webster, Peggy Crawford and Farrah Thomas’s article, “Who’s Behind the Wheel? Driving with Medically Intractable Epilepsy”, researchers took a sample of patients from the Cleveland Clinic Epilepsy Center with valid licenses and investigated what demographics affected the decision to drive (Webster et al. 2011). The results of their study showed that around one third of their population continued to drive. Additionally, 30% of the patients in the study had reported continuing to drive even after having had a seizure-related motor vehicle accident. Out of the patients questioned for the study who have had multiple seizure-related motor vehicle related accidents, 90% decided not to continue driving. The authors were able to confirm their hypothesis on the decision to continue driving being not only related to having multiple seizure-related motor vehicle accidents, but also to employment. They furthered this by finding that those who worked full time and had little means of alternative transportation were more likely to drive.

These two articles have shown difficult decisions those with restrictions on driving face. People with these conditions first must decide whether or not to drive. Then if they decide to not drive, it must be figured out how they will get around with these driving restrictions in a modern society that almost depends upon vehicular travel. Some do not even get the option to drive if their conditions are severe enough due to safety focused laws and regulations. With this, researchers have begun to access how new technologies can potentially increase travel and safety.

Corey D. Harper and the other authors of the article, “Estimating Potential Increases Travel with Autonomous Vehicles for Non-Driving, Elderly and People with Travel-Restrictive Medical Conditions” report on the possible increase of travel in different demographics due to autonomous technology. The authors state that, “The results from this analysis are intended to provide insight on the magnitude of potential future increases in total travel demand from these underserved populations under vehicle automation.” (Harper et al. 2016). With using the National Household Transportation survey from 2009 as the primary source, this study found that non-drivers, the elderly, and people with driving restrictions due to medical conditions will have an increase in travel if autonomous vehicles are introduced. Females would have the largest increase in vehicle miles traveled, the results showed, and working age adults would have the most increase in magnitude. Additionally, the study concluded that light-duty vehicle travel may increase by 14% and non-drivers would increase light-duty vehicle travel by 9%.

In Brian Reimer’s article, “Driver Assistance Systems and the Transition to Automated Vehicles: A Path to Increase Older Adult Safety and Mobility?” he discusses how using an advanced driver assistance system (ADAS) will aid the transportation of those who no longer drive due to age or medical conditions (Reimer 2014). He uses the National Highway Traffic Safety Administration’s system of classifying automated systems on a level between zero and four. At level zero, the system will only provide information but has no control over the vehicle. Level one, a step up, will expect the driver to continue to give their attention to driving the vehicle, cruise control is an example of this. As the levels progress to four, the driver gives less oversight and more trust to the automated system to drive the vehicle. Reimer acknowledges that current technology has not reached the fourth level of automation, and urges that drivers be better educated on the lower levels of automated technology that currently exist within their vehicles. Many currently are hoping for fully automated cars, he wants more education on the current ADAS available that will be able to support many of the current safety and mobility needs. With level 4 ADAS not being available anytime soon, he suggests that, “Policymakers, researchers, and industrialists should focus on developing a cohesive vision for increased vehicular automation that promotes, where effective, the utilization of current safety systems to reduce traffic fatalities, personal injury, and property damage” (Reimer 2014). With the populations’ hopes for increasing travel through ADAS, he recommends that drivers be educated on the reality of the technology along with supporting policies for regulations on this technology.

Harper et al. (2016) showed how predicted statistics on different demographics could have an increase in vehicular travel due to autonomous vehicles while Reimer (2014) focused on how education is needed for autonomous technologies to be useful. Reimer (2014) also focuses on how complete autonomous vehicles are a far away technology and lower level ADAS can also solve the problems that people want complete autonomy for. The following articles show different technologies used or researched that could be or are applicable to autonomous technology.

In Joshua Seth Herbach and Nathaniel Fairfield’s patent, “Methods and Systems for Determining Instructions for Pulling over an Autonomous Vehicle” they describe with different examples, what methods are used and which systems are applied to pull over the vehicle (Herbach 2016). The method can use the speed of the car to determine how to break tin order to reduce speed, along with how far the car will travel once the breaks are applied. Also, the method could read in several components of the road, like its boundaries or the lanes, to access the edge of the road. With this, it may identify a computing device and its stored memory to pull over the vehicle in the designated space. Many examples and scenarios are defined with how the methods interact within this patent.

Methods on how to signal a driver are researched in the article, “Multimodal urgency coding: auditory, visual, and tactile parameters and their impact on perceived urgency” by Baldwin et al. (2012). The research examined visual cues in regard to color, word choice, and flash rate. Auditory cues were tested with different frequencies, pulse rates, and different volumes. Then to access tactile cues they gave different pulse rates. With this, the researchers managed to, “determine urgency scaling within and across visual, auditory, and tactile modalities – and specifically, to develop and test a methodology for determining these cross modal scales” (Baldwin et al. 2012). Also, they found that tactile signals were able to display varying urgency to drivers.

3.1 Participants

Within this study, 21 volunteers were recruited from the Mt. Pleasant area. Volunteers could be students of the university or nonstudents, however this information was not recorded. Volunteers did not have to be licensed drivers to participate; four volunteers were unlicensed at the time they participated. The volunteers who are required to wear glasses while driving also wore them during the simulation. The volunteers were recruited by responses to fliers posted around Central Michigan University’s campus.

3.2 Materials

The open-source racing simulator, Torcs was used to simulate a driving environment in this study. As reported on the main website for the program, Torcs.org, this program has been used as a racing game, an AI racing game and for research. The source code is under a GNU General Public License and the associated artwork is under a Free Art License. This allows users of the code to use it for any purpose, modify the code, and distribute the code with any changes made.

The Torcs simulation was displayed on a computer monitor. Participants drove the vehicle in the simulation using a GameStop PS3 Steering Wheel with foot pedals. The steering wheel and pedals were connected to the computer and controlled the steering, acceleration, and brakes of the car in the simulation.

The Unity real-time development platform was used to create a dashboard simulation. The program created using this was displayed on the monitor of a Dell laptop. The functions of the program were controlled using a separate keyboard.

A survey was used to assess the user preference. Participants responded to several questions with a scale of one to ten: one meaning not at all and ten meaning very. Users specified opinions on aspects of each alert like the clarity and urgency it provided as well as if it would add stress to the situation. Additionally, users gave preference to icons, and how they felt about will be asked to respond to how they felt about the manner in which the vehicle pulled over. Space was left for users to write comments.

3.3 Design and Procedure

This study used the open-source racing simulator, Torcs, as driving environment for the participants. The user initially had control over the vehicle modelled within the program. They controlled the vehicle with a connected steering wheel and analog pedals to work like gas and brake pedals. At a time that is predetermined, but will appear random to the user, the participant will no longer be in control of the vehicle. To simulate the vehicle pulling over and the user experiencing a medical anomaly, the monitor displaying the vehicle simulation was turned off and an alert on the monitor displaying the simulated dashboard created from Unity will begin an alert. The program will not be detecting medical anomalies and the participants will not be experiencing one during the duration of the simulation.

The user repeated the process several times. For each separate trial the system announced that the autonomous system is pulling over the vehicle with different alerts. For the visual alerts, several different vehicle icons to be displayed on the simulated Unity dashboard were created. These icons were based upon standards and categories that were discussed in Chi and Dewi’s article (Chi 2014). Within this there are three main categories: graphical, textual, and combined. A a textual icon, which is based on text, was created for this study and displayed the phrase, “PULLING OVER”. This study also included three graphical icons, which based on Chi and Dewi’s study could be image-related, concept-related, semi-abstract or arbitrary. The icons created for this study were both image-related and concept-related, since they all included an image of a vehicle as well as trying to convey that the vehicles autonomous methods were taking over the vehicle. With the concept-related portion we also wanted users to understand that they should not try to regain control of the vehicle. The icons created can be seen below in Fig. 1.

Icons created for and displayed in alerts reviewed by users.

Full size image

The icons were presented on a separate dashboard simulation that was created with Unity. This program was displayed on a monitor that was positioned in between the steering wheel and the larger screen that was displaying the Torcs driving simulation. It was positioned in this way so that it would be in the location that is related to that of a dashboard while driving a normal vehicle. In Fig. 2, the simulation can be seen which consists of a picture of a dashboard (M.P. 2017) and would have the icons appear or blink as well as play an alarm tone (nmscher 2009). The alarm tone used was edited in Unity to have the pitch of C, which is considered a standard pitch for alerts (Block 2000). The alarm tone volume coming from the dashboard simulation was also adjusted to be at 15 dB above the volume of the driving sounds from Torcs which was recommended for auditory alerts (Patterson 1990). Also, the volume of Torcs was adjusted to be the same was the volume recorded inside of the average running vehicle. This was checked by measuring the decibel reading using a decibel reader of both the car and the program. The decibel reading coming from the program was adjusted to match that of the car which was a reading of 47 dB.

Dashboard simulation created in Unity showing a graphical icon.

Full size image

These functions of the program were operated by the proctor of the session by a connected keyboard. The order of the icons presented, the time the alert began, and whether they were accompanied by a blinking effect or an alarm tone was all predetermined. Several sets of trials were created, this way all users experienced one of three sets. Each set had 5 trials, each of the 4 icons were used and there was an extra trial where no icon appeared and only an alarm tone sounded. The different set allowed for the icons to be tested in different orders and with a variation of effects.

After the participants have completed the simulations, they were asked to give feedback on the different alert systems used. Initially participants were required to briefly describe any visual or auditory notifications that occurred. This tested whether they were able to accurately perceive the alerts presented. Participants will then respond to several questions on a 1 to 10 scale, 1 meaning least likely and 10 being the most likely. There were some questions for users to answer if there was an alarm tone in the presented alert of the trial. If they noticed an alarm tone, they then rated the urgency of the tone, how likely it would be to startle them, and how likely it would be for them to notice the alarm tone during a medical emergency. If they didn’t notice an alarm tone, they would answer questions about if there was an alarm tone, would they have noticed the alert and/or the icon better. If the user noticed a blinking effect, they would rate if it made the icon more noticeable and if it made the alert appear more stressful. Then regardless of the content of the alert for that trial, users would answer questions that rated the alerts on how distracting it was, how understandable the icon was if one appeared, how likely they would be to understand it during a medical emergency, and if it would have added stress during an emergency. When the trials and the complimentary questions are answered, users will answer a final set of questions that ask what icon they preferred as well as how they felt about all added alarms or blinking effects. With this, participants also responded about how likely they would be to trust self-driving technology to pull over their vehicle.

To assess the surveys and find significant results, this study used R to preform an analysis of variance test (ANOVA) on the data sets. These data sets were manipulated to remove any user mistakes. Mistakes were defined as an icon going unnoticed during a trial and users responding to a question about an effect that did not occur within the trial. All mistakes were recorded separately and replaced with averages of the other data from that trial case.

4.1 Icon Preference

In the finishing questions, which were given after all trials were completed, users were asked to review and rate the four different icons that they were presented with during their trials. With this they put in order the four icons from best to worst, which would be 1 (best) to 4 (worst) The top pick among users was the textual icon displaying the phrase, “PULLING OVER”. This one was ranked first the most number of times, which was six times. It also had the best overall average ranking of 1.857 (SD = 1.283). This was significantly great than the second most preferred icon, F(1,9) = 25.88, (p < 0.05). The second-best rated icon on with an average of 2.09 (SD = 0.75) was the graphical icon with an X through it. Following this was the graphical icon with the cross, which had an average of 2.52 (SD = 0.85). These two graphical icons with the X and cross were comparable based upon the similarity with their average values and there was no significant variance between the two sets of rankings. The worst rated icon was the graphical icon with the signals, and it had an average rating of 3.52 (SD = 0.66).

4.2 Responses on Alert Effects

When analyzing user responses, questions for the same icons as well as at least one similar effect were compared. When comparing the two trials that had a graphical icon with a cross through it that also had a blinking effect, there was a significant difference found in the questions that assess the response to the blinking effect. One trial had and alarm tone along with the blinking effect and the other just had the blinking effect alone. After performing an analysis of variance a significance was found with the question that asks how noticeable the overall alert was (F(1,5) = 8.804, p < 0.0313). The average response on how likely it would be for this alert to be noticed for the trial with the sound was 7.286 (SD = 1) and the average response for the trial without sound was 6.286 (SD = 0.47). Since the average response was higher for the trial with the alarm tone than the one without, it shows that users thought that the alarm tone along with a blinking effect made an alert more noticeable than an alert with a blinking effect and no sound.

When comparing other trials with the same icon and a similar effect, there were other significant findings. However, there were a few to be noted that were near the desired p-value of 0.05. It is possible that if this study had more users these findings could have been significant. More significant data was found however with the data attained from the final survey which asked overall questions about all the alerts, trials and their effects.

The final set of summarization questions were analyzed all together, regardless of the sequence of trials the user was showed. These were not about any specific trial, just about the effects and icons used throughout the study. Two questions, both separately inquiring about how noticeable the blinking effect and the alarm tone were, showed preference towards the noticeability of the alarm tone, with an average response of 9.381 (SD = 1.939) when compared to the noticeability of the blinking effect, with an average response of 7.952 (SD = 2.572); this data was found to be significantly greater F(1,19) = 11.54 (p < 0.05). Therefore, users found that after experiencing all five trials that overall the noticeability of the alarm tone was great than that of the blinking effect.

In addition, the finishing questions also showed the understandability of the alert with the alarm tone, had an average response of 6.524 (SD = 3.002) versus the alert with a blinking effect, which had an average response of 5.762 (SD = 3.176). This data also showed a significant difference with preference to the alarm tone F(1,19) = 8.569 (p < 0.05).

Upon concluding this study, a user preference was found toward textual icons as well as finding the effects that were most noticeable and understandable. To continue to define the most user preferred alert for this autonomous safety system more studies need to occur to answer other related questions. For instance, would users prefer a different textual icon than the one used? How do icons not related to the standard compare to the ones tested in this study? Additionally, in this study several different icons were tested, but as a constant, the alarm tone, blinking rates, volume, and the placement on the dashboard used were all the same. Testing different alarm types at different volumes with the similar parameters in this study would add to the detail of the alert.

However, this autonomous method will need more than just an alert, so this is just the beginning. More research is needed to fully define this system. This will include research on how to detect if the driver is experiencing a medical anomaly. Additionally, it would be beneficial to research a method to autonomously pull over the vehicle that is comfortable and will not add stress for someone in a medical emergency.

The outlined methods used to find the preference for this study can be replicated for the related future work to come. Using the simulation method was a safe and consistent source for testing driving scenarios. This can easily be used to accommodate for studies with new alarm tones, icons, and other new alert effects such as vibrations. Additionally, keeping the questioned variables: noticeability, understandability, how likely is it to cause stress, and if it was distracting, will help to add to the significant alert effects found in this study.

Within this study, the user preference of alerts system of an alert system. This system would need to warn a driver experiencing a medical anomaly that the vehicle will be pulling over using autonomous methods. A simulated environment was created in which 21 users drove. Upon completing five trials, a survey was given to these users to collect opinions on the alerts and effects shown. These results were analyzed using R.

Significant results were found using an ANOVA test. It was concluded that the dashboard icon which included text instead of images was preferred. Also, users responded with the opinion that an alarm tone with a blinking effect was better than just a blinking effect applied to an icon alone. An alarm tone was considered more noticeable in this situation than a blinking icon. Additionally, alerts that included an alarm tone were found to be more understandable than a alert with a blinking effect applied.

These results can be used to make the alerts for a safety system in vehicles that will pull over autonomously for drivers experiencing a medical anomaly that leaves them unable to drive. Hopefully this data can be used to create a full alert for this system, then, the completion of the entire autonomous procedure to ensure safety for all.