Keywords

1 Introduction

Air transport industry faced all kinds of difficulties in the early days. At that time, the safety of the mechanical system in aircrafts is the main cause of various aviation accidents due to the underdevelopment of science and technology. However, aviation safety has been greatly improved with the development of avionics and mechanical manufacturing technology in recent decades. Nowadays, air transportation has become one of the safest type of transportation. How to continue to enhance the security of this area has been the greatest challenge for the aviation safety organizations. Though human factors accounts for different proportions in aviation accidents in different researches, most of them showed a percentage between 70 % and 90 % [1]. As a result, aviation human factors gradually attracted people’s attention. Previous studies have shown that cockpit ergonomic is the most direct cause of flight crew’s misoperation [2]. Therefore, researches on human factors in aircraft cockpit are especially important in order to improve aviation safety. For sake of the adaptation between the design of cockpit and the task requests as well as execution capacity of the pilots, the human (pilots) centered design concept was gradually formed during the process of aircraft design, especially in the design of the cockpit.

Due to the appearance of new technologies like vertical takeoff and landing, low visibility landing as well as new noise reduction methods, the workload level of modern pilots shows a significant rise than before, and this change has a great impact on aviation safety. There has been studies indicated that pilot’s mental workload plays a crucial role in solving problems during flight [3], and in recent years a greater interest in assessing workload during flight testing was aroused. There are two different ways to measure pilot’s workload, subjective assessment and objective assessment. The former was used quiet a lot in the early days. Later, people found that the physiological parameters of pilots like heart rate, respiration and blood pressure would change when pilots are under different level of workload since they always try to maintain their performance at an acceptable level which would lead to a decline in parasympathetic nervous activity and an increase in sympathetic nervous activity [4]. For now, the physiological parameters used in subjective workload assessment includes electroencephalogram (EEG), electromyography (EMG), galvanic skin reflex (GSR), electrocardiogram (ECG), blood volume pressure (BVP), skin temperature and respiration.

Basing on the background described above, this paper mainly aims at building a workload comprehensive evaluation model with effective features deriving from the physiological parameters of the pilots.

2 Workload

2.1 The Definition of Workload

Though the concept of “workload” often appears in studies about human factors, there is no complete agreement reached over the definition of it so far. Some researchers have different opinions on the origin of workload, while others disagree over the mechanisms, consequences and measuring method of it [5]. However, the main aspects of workload seems to be concentrated in three categories: the quantity of work or events that need to be finished; time or one specific aspect of time that people focus on; the subjective mental feelings of pilots [6]. Here are some common definitions of workload:

  • The information processing capability and resources that operators have in order to meet the system requirements;

  • The difference between the information processing capacity that operators need to have in order to meet the performance expectations and the ability that operators really have;

  • The efforts that operators make in order to realize control and monitor. When operators perform different tasks at the same time, it achieves the maximum;

  • The decrease of capacity in performing other tasks which need to use the same processing resources when operators are performing one task;

  • The relative reaction ability, the key point is to predict what operators would be able to achieve in the future.

There may be new operational definitions and verifications of workload in the future, and people in different areas will continue to define it in different ways.

2.2 Measuring Criteria and Measuring Technology of Workload

The main reason of measuring workload is to quantify the physiological cost of pilots when they are carrying out tasks so that we can predict the performance of the operation staff and the whole system. One problem in measuring workload is that the workload measured in laboratory environments is usually different from that in true operating environment. In order to minimize the difference, O’Donnell and Eggemeier proposed several criteria in measuring workload [7]:

  • The measuring technology must be sensitive to the change of task difficulty and it should be reliable.

  • The measuring technology should be diagnostic enough to point out the origins of workload change and quantify their contribution.

  • The measuring technology should not be intrusive. It can not influence the performance of subjects.

  • The measuring technology should be acceptable for subjects.

  • The number of devices which will affect performance should be as less as possible.

  • The measuring technology should be able to capture instantaneous changes of workload timely and quickly.

  • The measuring technology should be reliable enough to ensure repetitive measurement.

  • The measuring technology should not be sensitive to the requirement of other tasks.

These criteria can help researchers to select and develop the measuring technology better. However, the difference between different subjects can also have an effect on the experimental results. For sake of obtaining more accurate results, some researchers set weight for the subjective rating scale of subjects before calculating, others set baselines, especially in case of physical measurement.

Since there is still no unified definition about the concept of “workload”, researchers in different fields use different technologies to measure it. Mainly, the measuring technologies can be classified into three categories: subjective rating scales, performance evaluation, and physical measurement. Different technologies are sensitive to different aspects of workload.

Subjective Rating Scales.

Subjective measurement of workload mainly focuses on quantifying the explanation and judgement of subjects about what they experienced. Some have pointed out that subjective rating scales are lack of reliability and validity.

Performance Evaluation.

Performance evaluation of workload can be divided into primary task assessment and secondary task assessment. The former is dedicated to evaluating operator’s performance directly when they are carrying out the designed tasks, which is quite effective when the work request exceeds the ability of the operator. The latter can provide a good indicator of operator’s remaining operating capacity when they are carrying out the main tasks. It will be more diagnostic when they are used together.

Physical Measurement.

The physiological parameters commonly used in researches about physical measurement of workload are electroencephalogram (EEG), electromyography (EMG), galvanic skin reflex (GSR), electrocardiogram (ECG), blood volume pressure (BVP), skin temperature and respiration. Almost all researchers agree that different physiological parameters significantly correlate to different aspects of workload.

The future direction of workload measurement is to develop measuring methods that are more formal and can couple multiple physiological parameters.

3 Multi-channel Physiological Parameter Measurement System

The experiment in our study was carried out in the Man-Machine-Environment System Engineering laboratory in Shanghai Jiao Tong University. The BioHarness Wireless Data Logger was the crucial equipment used in our experiment. Besides, the integrated flight monitor system was important as well.

3.1 Physiological Parameter Detecting Device

We used the BioHarness Wireless Data Logger produced by Zepher to collect the physiological parameters we need (Fig. 1). The Data Logger contains an infrared temperature sensor and a tri-axial acceleration transducer. It has 16 data recording channels totally, including ECG, respiration and ROM (range of motion).

Fig. 1.
figure 1

BioHarness wireless data logger

Full size image

The physiological parameters used in our study are heart rate [8], respiration [9], ROM and IBI (inter beat interval) [10]. ROM is composed of the total range of motion, the peak acceleration as well as the minimum acceleration and peak acceleration in three different axes. The IBI data was used for heart rate variability analysis.

3.2 Flight Monitor Integrated System

The integrated flight monitor system includes the simulated cockpit, visual system, video monitoring system and equipment condition monitoring system.

The Simulated Cockpit.

The simulated cockpit contains the flight control workstation, the display area, control area and peripherals. The flight control system can provide flight control models for the whole system and the simulated flight mission was designed basing on the flight dynamic parameters generated by it. The display area includes primary flight display and navigation display and the control area contains the common control unit such as the throttle, the steering wheel and the handle (Fig. 2).

Fig. 2.
figure 2

The simulated cockpit

Full size image

The Visual System.

The visual system contains a three-channel projector and a circular projection screen, which can make our pilots feel more depth and more real during the simulated flight experiment so that the physiological parameters we collect would be more realistic.

The Video Monitoring System.

The primary role of the video monitoring system is to record the hand action, facial expression of the subjects and the scene condition during our experiment. It can help the subjects to recall their actual experience during operations when they are asked to finish a self-assessment of workload after the simulated flight experiment.

The Equipment Condition Monitoring System.

The equipment condition monitoring system is designed for monitoring the working condition of the whole integrated flight monitor system. It is able to adjust the test flow and ensure the test is conducted strictly in accordance with the requirements.

4 Flight Simulation Experiment

In order to get the physiological parameters we need, we designed a full process simulated flight mission, basing on the existing Boeing 777-200ER simulated cockpit in our laboratory. It includes a total of six different stages: takeoff, climb, cruise, falling, approach and landing, which is designed to be similar to the true flight condition as much as possible. The participants can control the aileron and elevator of the simulated airplane with the joystick in the cockpit during the experiment. They can also get control of the rudder and brake using the pedal. Besides, the throttle lever, the flaps and landing gear can all be controlled.

Six subjects (age 20.83 ± 2.04) from School of Aeronautics and Astronautics in Shanghai Jiao Tong University participated in our experiment. Since the BioHarness Wireless Data Logger used in our experiment needs to be worn on the breast, which is quite inconvenient for women, all of the six subjects were men. For avoiding disturbance caused by special factors, all the subjects were in good condition of health and had normal or rectified vision. None of them had musculoskeletal injury or mental disease related to angiocarpy, respiration or the central nervous system within a year. Besides, they all had ample sleep before taking part in the experiment. Hoping that our research result would be more reliable, all the subjects were asked to repeat the same experiment twice and the interval between them was 7 to 30 days.

After being told about the goal and content of our experiment, the participants began to carry out the simulated flight mission, and their multiple physiological parameters, including heart rate, respiration, range of motion and IBI, were measured with the BioHarness Wireless Data Logger at the same time. Since different physiological parameters contain different information of the sympathetic nervous system and the parasympathetic nervous system [11], a single physiological parameter can hardly express all the information related to the workload of the pilots, so we recorded these four aspects of physiological parameters in our experiment, hoping that the final workload assessment model we get would be more reliable. At last, all the subjects were asked to recall the workload they actually felt at different stages during the simulated flight experiment and finish the NASA-Task Load Index [12] separately.

5 Feature Extraction and Principal Component Analysis

According to the study of Karavidas M.K., Lehrer P.M., et al., the workload of the pilots was relatively lower during take off and cruise, and it turned higher in landing [13]. So the collected physiological parameters were divided into three sections according to the three different stages in the simulated flight mission (takeoff, cruise and landing), basing on the premise that synchronization was achieved between all the physiological parameters recorded and all the points of time during the completion of the simulated flight mission. All the physiological parameters were preprocessed and standardized before processing so that the interference arising by contamination of artifacts and the difference between all participants can be minimized [14].

5.1 Feature Extraction

In order to extract the specific features related to pilot’s workload, each feature derived from the raw physiological parameter was tested for its validity and reliability separately.

Heart Rate.

Heart rate is controlled by both the sympathetic nervous system and the parasympathetic nervous system. Usually, the increase of workload will lead to the increase of heart rate [15]. In our research, we tested the mean value, the standard deviation and the mean absolute value of the first order difference of heart rate.

Respiration.

Previous research has shown that pilot’s breathing is significantly related to the change of workload. Pilots may be hyperventilating in case of extreme workload, which would have serious effects on flight safety. The mean value, the standard deviation and the mean absolute value of the first order difference of respiration rate was tested in our research.

Range of Motion.

The movement, posture and gesture of a man can be measured in three ways: using acceleration sensors or inertial measurement systems, using pressure sensors and using tracking systems basing on videos [16]. In our research, we used the data collected by the eight ROM channels of the BioHarness Wireless Data Logger. They are range of motion, peak acceleration and the minimum acceleration and peak acceleration in three different axes.

Heart Rate Variability.

Both the parasympathetic nervous system and the sympathetic nervous system are origins of heart rate variability. The increase of workload will lead to the decrease of parasympathetic activity. Researchers have found that heart rate variability will also be weaker when task complexity increase. We preprocessed the original IBI data through interphase correction [17], detrending [18] and resampling. The four features derived in time domain are mean value of IBI, standard deviation of NN intervals (SDNN), root mean square of successive differences (RMSSD) and the percentages of NN intervals which exceed 50 ms (pNN50). Basing on Fast Fourier Transform (FFT), we get other 9 features in frequency domain, they are LF peak, HF peak, LF power, HF power, the percentage of LF power, the percentage of HF power, normalized LF power (nLF), normalized HF power (nHF) and LF/HF.

According to the criterion that there should be significant difference (p < 0.05) between the features during take off, cruise and landing, we totally extracted 12 features from the 27 features derived from the raw physiological parameters by using independent T-test, they are the mean value of heart rate, mean |H1-H2| (the mean absolute value of the first order difference of heart rate), the mean value of respiration rate, mean |R1-R2| (the mean absolute value of the first order difference of respiration rate), range of motion, peak acceleration, the mean value of IBI, LF power, HF power, nLF, nHF and LF/HF. All of them showed good reliability.

5.2 Principal Component Analysis

Principal component analysis (PCA) [19] was used to reduce the dimensions of the feature space. After applying PCA to the 12 features extracted in the previous step, we got four main factors related to pilot’s workload and the cumulative contribution of them is 82.322 %. A workload comprehensive evaluation model was generated at the same time with the coefficients we got in PCA (Table 1).

Table 1. Component matrix without small coefficients
Full size table

Since all the features in factor 1 were in the time domain and all the features in factor 2 were in the frequency domain, we call factor 1 “time domain factor” and factor 2 “frequency domain factor”. The other two factors are named “activity factor” and “acceleration factor”. The absolute value of all these four main factors will decrease when the level of workload increases.

Our study showed that heart rate, respiration, range of motion and heart rate variability of the pilots would all change under different level of workload, which demonstrated the feasibility of using the special features extracted from these physiological parameters to assess the workload of pilots during flight. According to the result of PCA, time domain features may contain quite different information about workload in comparison with frequency domain features. Besides, the activity data of pilots is also important in research about workload.

6 Conclusion

This paper is a step toward automatic assessment of pilot’s workload, which will be quite useful in the airworthiness approval of new aircrafts. Previous research has shown that the workload of pilots can not be evaluated precisely by using one single physiological parameter. Though we synthetically studied heart rate, respiration, range of motion and heart rate variability in our research, but our division of flight phase is relatively simple. In the future, the assessment of pilot’s workload could be more precise through collecting more types of physiological parameters and classifying different flight phase more precisely. Besides, none of the subjects in our research was professional even though they all received a short-term training before participating our experiment. Their piloting skill was relatively poor in comparison with professional pilots, which may have led to a relatively low level of simulation. Collecting the physiological parameters of professional pilots may help to get better results. Finally, simply evaluating the overall workload may have its limits since workload is a variable with multiple dimensions. Finding the specific features that can be used for the assessment of different aspects of workload may be the future research direction.