The role of vision in sensory integration models for predicting motion perception and sickness

A wide range of studies indicate that motion sickness is caused by mismatches between perceived sensory signals—i.e., from the eyes, otoliths, semicircular canals, etc.—and expected sensory signals from the central nervous system (e.g., Reason (1978)). These mismatches are particularly triggered during passive motion—i.e., when the motion is due to external forces that cannot, or only imperfectly, be anticipated—with strongly moving visual surroundings, for example as experienced on ships and riding horses (Huppert et al. 2017). With the current trend towards fully automated vehicles, motion sickness is expected to become much more widespread (Griffin and Newman 2004; Bertolini and Straumann 2016; Kuiper et al. 2018; Salter et al. 2019), as all vehicle users will be passengers that passively experience the vehicle’s motion, while preferably engaged in other activities. Minimizing the incidence of motion sickness in automated vehicles requires improved knowledge of motion sickness and how it relates to human motion perception mechanisms, as well as accurate models with which its development over time can be predicted, to design comfortable vehicle motion control strategies.

In motion sickness models, the ‘sensory conflicts’ that are assumed to cause sickness (Reason 1978) are generally defined as the difference between the sensed and expected sensory signals. Hence, these models are often referred to as ‘sensory conflict models’, due to their ability to quantitatively predict the conflicts that, when accumulated over time, lead to the worsening of motion sickness symptoms (Oman 1990; Bos and Bles 1998; Kufver and F¨orstberg 1999; Irmak et al. 2020, 2022). For example, Bos and Bles (1998) conceptualized the Subjective Vertical Conflict (SVC) model, which is based on the assumption that motion sickness is caused by conflict in the sensed vertical (i.e., orientation with respect to gravity). The SVC model has been extended by Kamiji et al. (2007) to account for all six degrees of freedom (DOF) in the 6DOF-SVC model, which matched some available motion sickness data sets at reasonable accuracy. However, the 6DOF-SVC model in Kamiji et al. (2007), only accounts for vestibular sensation and how this impacts the subjective vertical, and motion sickness.

It is well known that the visual system has a crucial impact on both our perceived motion, as well as motion sickness. For example, illusory perceived subjective orientations that occur with only vestibular stimulation no longer occur when a visual reference (e.g., horizon line or straight walls) is perceived. Furthermore, the sickening drive data from Irmak et al. (2020) show that participants become at least 1.83 times as sick when not being able to see the vehicle’s movement (internal view) as compared to when looking outside the vehicle (external view). Motion sickness is also observed in studies using virtual reality or using fixed-base driving simulators. Since there is no physical motion in these cases, this is commonly called Visually Induced Motion Sickness (VIMS). To account for such effects, a number of perception models include visual perception contributions of angular rotation velocity, verticality, or both such as the Multi-Sensory Observer Model (MSOM) by Newman (2009), the extension of the 6DOF-SVC model with vision by Wada et al. (2020, 2015); Wada (2021); Liu et al. (2022), the spatial orientation and motion sickness model by Bos et al. (2008), and the sensory weighting model by Zupan et al. (2002). At this moment, some of the sensory conflict models that include visual perception have been validated for specific motion perception paradigms (e.g., MSOM by Newman (2009)) or for motion sickness prediction in real world naturalistic driving (e.g., Yunus et al. 2022). However, so far, no single model has been shown to describe both the perceptual effects of vision, and its effects on motion sickness development, which is required for physiologically valid and interpretable predictions of motion sickness (Irmak et al. 2023).

Hence, the goal of this paper is to verify the accuracy of available sensory conflict models that include visual motion perception, for predicting human perception responses in well-known motion perception paradigms, as well as motion sickness data from laboratory experiments (Waespe and Henn 1977; Vingerhoets et al. 2006; Merfeld et al. 2001; Correia Gracio et al. 2013) and real-world driving experiments (Irmak et al. 2020). In this paper, we focus on comparing only the most recent versions of the motion sickness and motion perception models that include visual rotational velocity and visual orientation perception, namely the Subjective Vertical Conflict (SVC) model (Wada et al. 2020; Liu et al. 2022; Inoue et al. 2022), and the Multi-Sensory Observer Model (MSOM) (Newman 2009; Clark et al. 2019), as the most promising candidates. Similar to Irmak et al. (2023), we present a two-part analysis, where we first focus on these models’ match to well-known frequency and amplitude sensitivity characteristics of motion sickness (McCauley et al. 1976; Golding and Markey 1996; Irmak et al. 2021, 2022; Griffin and Mills 2002b; Howarth and Griffin 2003). Furthermore, the extent to which the models can replicate the effect of vision conditions in the real-world sickening drive study of Irmak et al. (2020) is analyzed. In the second part, we assess the model’s ability to replicate well-known fundamental motion perception tests, i.e., earth vertical axis rotation (EVAR) (Waespe and Henn 1977; Vingerhoets et al. 2006), off-vertical axis rotation (OVAR) (Vingerhoets et al. 2006), centrifuge (Merfeld et al. 2001), somatogravic illusion (Correia Gracio et al. 2013) and pseudo-Coriolis (Newman 2009).

Apart from the aforementioned suggested models, alternative theories about the origin of motion sickness are also present. According to the work of Riccio and Stoffregen (2010), another proposition is that motion sickness arises due to ‘postural instability’. This implies that animals feel sick when they are in situations where maintaining proper postural stability becomes difficult. Consequently, this theory suggests that postural instability acts as a direct precursor to the symptoms of sickness. However, as the majority of existing mathematical models for motion sickness are grounded in the sensory conflict theory, this paper will not delve into the postural instability theory.

Thus, this paper directly extends the work of Irmak et al. (2023), who focused on vestibular-only perception and sickness modeling, to also include the essential visual component. Based on this quantitative performance comparison of available sensory conflict models, we formulate recommendations on the most critical and promising model structures for predicting motion perception and motion sickness, as well as provide crucial suggestions for much-needed experiments for further model validation.

Sensory integration models

In this paper, we evaluate several versions of the subjective vertical conflict (SVC) model (Fig. 1) with parameters in Table 1, and the Multi-Sensory Observer Model (MSOM) (Fig. 2) with parameters in Table 2.

Both models consider two vestibular inputs: the specific force sensed by the otoliths and the angular velocity sensed by the semicircular organs. Both models also include two visual inputs: a visual cue of rotational velocity (visual rotational velocity or VR; ω_vis in the Figs. 1 and 2) and a visual cue of verticality (visual vertical or VV; v_vis in the Figs. 1 and 2). The reasoning behind visual rotational velocity is that the human eye can perceive rotational velocities through rotational (optic) flow (Ehrenstein 2003). Visual vertical is included because human eyes tend to find earth vertical or horizontal objects such as trees, buildings, and horizons to orient themselves with respect to the earth (Cano Porras et al. 2020). The two approaches look promising. Human vision, as discussed in Berthoz and Droulez (1982), also is able to perceive linear (translational) motion (linear velocity and linear position). These have been modeled in the MSOM (Newman 2009) as well. However, in the model, these do not contribute to either verticality, acceleration, or angular velocity estimates. They are solely used for improving linear velocity and linear position estimates and do not affect sickness predictions. Hence, these visual linear motion pathways are not considered in the analysis in this paper.

Subjective Vertical Conflict (SVC) Model: a high-level diagram and b–d details of different subsystems. Vestibular loops (black) are as in Kamiji et al. (2007); ‘VR’ (visual rotational velocity) loops (blue), as in Wada et al. (2020); ‘VV’ (visual vertical (direction of gravity or orientation)) loops (red), as in Liu et al. (2022). The SVC_I model integrates the acceleration conflict (dotted box in d). This integrator (I) is replaced by a unity gain (no integration, NI) in the SVC_NI model in Inoue et al. (2022)

Full size image

Multi-Sensory Observer Model (MSOM) as in Newman (2009): a high-level diagram and b–d details of different subsystems. Vestibular loops (black); ‘VR’ (visual rotational velocity) loops (blue); ‘VV’ (visual vertical (direction of gravity or orientation)) loops (red); ∗calculates the rotation required to align the actual and expected sensory estimates (see Newman (2009) for more details)

Full size image

SVC model

Bos and Bles (1998) developed the Subjective Vertical Conflict (SVC) model, which estimates the conflict between the sensed signals from the sensory systems and the expected sensory signals derived from the central estimates computed by the central nervous system. The model has three parts: the ‘Visual-Vestibular System’, the ‘Internal Model’ of the visual-vestibular system, and the ‘Feedback’ of errors between sensed signals and internal model predictions, see Fig. 1. The main concept of this model is that the dominant conflict causing motion sickness is the mismatch between the estimation of verticality (orientation with respect to gravity) and the sensed verticality. The otolith dynamics are assumed to be unity (OTO = 1), while the semicircular organs are modeled as a high-pass filter (SCC = τ_scc s²/(τ_scc s + 1)²). These same sensory models (SCC and OTO) are also used in the internal model. The LP block calculates the subjective vertical (direction of gravity) using specific force and angular velocity through a low-pass filter with a time constant τ (Bos and Bles 1998; Mayne 1974), see the black lines and blocks in Fig. 1b.

Kamiji et al. (2007) extended the SVC model by Bos and Bles (1998) to include all six degrees-of-freedom (DOFs). The working principle is identical to the original SVC model and the parameters of each pair of three DOFs (linear and rotational) are identical. So for example, vertical, longitudinal, and lateral motions will all give the same conflict. Kamiji et al. (2007) also added loops that aid in state estimation during active motion (dashed pathways in the bottom left of Fig. 1a). These loops model active knowledge about the sensory consequences of the anticipated motion of the body, as was also mentioned by Oman (1990). This helps to improve estimation in the internal model by including the effect of the efference copy for the active movement, the predictable motion in passive cases, and unmodeled sensory signals such as proprioception. We tested the simulations with and without these (K_ω, K_a) parameters where we found only a 10% difference in the conflict magnitude while carrying out motion sickness frequency sensitivity analysis. However, the frequency dynamics are fully consistent between both cases. Also, the results for the motion perception tests are similar across all motion paradigms. Hence, as also done in previous work (Irmak et al. 2023), this loop was disregarded in the current paper as the conditions studied involve purely passive motion only, without any prior information to predict the motion stimuli. All model parameters are taken from Wada et al. (2020), Liu et al. (2022), and Inoue et al. (2022), which were set by tuning them to match the vertical motion sickness data by O’Hanlon and McCauley (1974).

SVC _I model

The SVC model by Kamiji et al. (2007) was further extended by Wada et al. (2020) and Liu et al. (2022) who added loops for simulating vision, see Fig. 1. The pathways in blue show the visual rotational velocity loop and the pathways in red show the visual vertical loop. The models with a visual rotational velocity loop will be referred to with a ‘VR’ at the end of their abbreviated names and the models with a visual vertical loop will be referred to with a ‘VV’. The sensory dynamics for both these visual inputs (VIS_g and VIS_ω) are unity matrices, as it is generally assumed that the eyes approximate a perfect sensor (Wada et al. 2020; Liu et al. 2022). The SVC_I model includes integration (I) for the acceleration conflict feedback term (see the dotted box in Fig. 1d) and is hence referred to in this paper as the SVC_I model.

SVC _NI model

The SVC_I model was found to not reproduce fundamental motion perception paradigms like roll-tilt perception during centrifuge (Inoue et al. 2022). However, Inoue et al. (2022) found that the removal of integration for the acceleration conflict in the feedback loop (see the dotted box in Fig. 1d) greatly improved model performance for roll-tilt perception during centrifuge, though only for the case without vision. The model by Inoue et al. (2022) (no integration, NI) is referred to in this paper as the SVC_NI model.

Multi-sensory observer model (MSOM)

Merfeld et al. (1993) proposed a one-dimensional observer model to predict the angular vestibular-ocular reflex to yaw rotation about the earth’s vertical axis. Furthermore, they extended this model to a 6-DoF model by including the otolith organs to account for acceleration storage and gravity storage central estimates analogous to angular velocity storage in the one-dimensional model. Here, the sensory dynamics of the semicircular canals (SCC) were modeled as a first-order high-pass filter with a time constant (τ_scc) of 5.7 s while the otolith dynamics (OTO) is unity. The G block calculates the subjective vertical (direction of gravity) using the angular velocity. The internal models of the central nervous system were assumed to be identical to these sensory dynamics. Using this approach of an observer feedback model, and adding additional processing layers, Zupan et al. (2002) proposed a 3D model consisting of three processing layers: frequency completion, conversion of sensory estimates to central estimates, and multi-cue weighted averaging of the central estimates. The internal models were made such as to complement the sensor dynamics and complete the frequencies to which perception remains sensitive. Additionally, using central estimates and calculating weighted averages helps in improving the central estimates. Newman (2009), using the vestibular core of this observer model, added additional estimates (position and velocity) to form the Multi-Sensory Observer Model (MSOM). This model, like the SVC model, has three parts—the models for the visual-vestibular system (green), the internal model of the visual-vestibular system (purple), and the feedback (orange) as can be seen in Fig. 2. The model’s parameters are listed in Table 2.

The MSOM was developed as a motion perception model (Newman 2009; Merfeld et al. 1993; Clark et al. 2019) and at the time of writing, did not define a conflict term for calculating motion sickness. Despite this, Irmak et al. (2023) used the MSOM model to predict motion sickness, for which he evaluated various conflict terms and concluded that only the otolith conflict, f_oto, was correlated to motion sickness. However, the otolith conflict was found to be unable to capture the well-known frequency sensitivity for sickness with vertical acceleration stimuli as in McCauley et al. (1976). This is further discussed in Sect. 4.1. Still, in this paper, we consider the integrated otolith conflict as the MSOM’s proxy for motion sickness.

The MSOM includes an explicit two-way coupling between the semicircular canals and the otoliths that helps to capture the somatogravic effect, which is closely linked to motion sickness induced by horizontal accelerations in absence of vision (Irmak et al. 2021; Wood 2002; Wood et al. 2007). The MSOM, like the SVC models, has two pathways for simulating vision. The pathways in blue show the visual rotational velocity loop (‘VR’) and the pathways in red show the visual vertical loop (‘VV’).

Tests for validation

The models (either VR, VV, or both) are validated using the model parameter values that were reported in their respective publications. While it is possible to enhance the modeling accuracy through a parameter optimization step, such retuning of model parameters is considered to be beyond the scope of the current paper. A brief exploration in appendix A.1 shows that the applied vision loop gain parameters are adequate for sickness prediction. To not further complicate the comparison of the relative effectiveness of the VV and VR loops in the SVC and MSOM models in this paper, the specific published forms of the models and their parameters were used. This means that these results are most applicable to SVC models and MSOM models when using the specific parameter sets we employed. Furthermore, given the lack of available comprehensive experiment data that cover the different vision cases studied in this paper, we present a qualitative assessment of the relative effects of modeled vision, rather than a quantitative one. By adhering to these defined parameters, we have achieved meaningful insights and valuable findings in this context.

Table 3 specifies the experimental motion paradigms used for validation in predicting sickness and perception, with reference to published data. Each of these validation paradigms was simulated for four different vision cases. These vision cases, and how they are implemented for the two possible vision inputs (visual vertical, v_vis, and visual rotation rotational velocity, ω_vis) were implemented as follows:

• ‘External vision’ is when the subject has the eyes open and has an outside view of a moving vehicle or motion simulator providing world-referenced visual information. v_vis is the same as the direction of the true vertical. ω_vis is the same as the true head angular velocity.

• ‘Internal vision’ is when eyes are open, but the subject’s vision is limited to the stationary interior of the vehicle. Assuming that the head rotates with the vehicle v_vis is set as constant pointing down (v_vis = [0, 0, − 9.81]) and ω_vis is set to zero.

• In the ‘Only vision’ case, there is no physical motion, but only visual inputs

• (also referred to as VIMS for motion sickness and vection for motion perception in literature). v_vis is the same as the direction of true vertical. ω_vis is the same as the true head angular velocity.

• In the ‘No vision’ case, the eyes are closed and only an inertial motion input is given. The vision loops are disabled.

The visual inputs are determined by making an assumption that the vision is a perfect sensor. The definitions given are true for simple motion paradigms. However, in complex motion paradigms such as real driving, these may not hold true due to the interaction of the view, inside and outside of the vehicle. Nonetheless, we assume the visual input to be only the outside view, thus simplifying the calculation of the visual inputs.

The expected result for the models’ motion sickness predictions is that ‘internal vision’ will give higher conflicts than both ‘external vision’ and ‘no vision’ (Wada and Yoshida 2016; Griffin and Newman 2004), as will be further discussed in Sects. 4.1 and 4.2. The comparison of these vision cases with the case of ‘only vision’ (VIMS) for the same motion paradigm and on the same participants is not known in the literature yet. Nonetheless, the simulation results for the ‘only vision’ case are shown to demonstrate the capability of the model in ‘only vision’ cases, as for example occur in fixed-base (driving) simulators.

The performance of the models for different vision conditions will be evaluated by measuring the linear integration (accumulation) of conflict over time. This is used as an indicator of motion sickness levels or motion sickness incidence. This approach eliminates the use of nonlinear integrators (Kotian et al. 2023; Kamiji et al. 2007), which are usually placed at the end of the model to convert conflict into a true motion sickness metric (i.e., MISC or MSI). In this paper, we use a simple linear integrated conflict to directly compare the results of various models in varying vision conditions. Using a simple linear integrated conflict makes it easier to compare the results of various models as well as the experimental data to computational results by normalizing them. Hence, we use conflict to compare the models and their performance in various vision conditions.

Motion sickness frequency sensitivity

Mapping of the frequency response of a model gives a quantitative measure of the sickness magnitude as a function of frequency and motion amplitude. In our case, this is shown with 3D plots of the accumulated conflict across a range of stimulus frequencies and amplitudes.

For the motion sickness frequency sensitivity analysis for linear DOFs, the simulated input was a fore-aft motion with frequencies from 0.06 to 0.63 Hz and amplitudes from 0.1 to 0.7 g, where g is the acceleration due to gravity (9.81 ms⁻²). The simulations were done for 60 min to predict the MSI at the end of this exposure. The simulation time, frequencies, and accelerations were chosen to be identical to that of the validation data given by McCauley et al. (1976) and simulation data presented by Kamiji et al. (2007). This is done to enable a comparison of the frequency and amplitude dynamics of conflict generation with the Motion Sickness Incidence (MSI) in the experimental dataset. As we are comparing accumulated conflict output from the models, the scale differs greatly from the 0–100% scale of MSI.

For frequency sensitivity analysis for rotational DOFs, the simulated input was a sinusoidal pitch motion with frequencies of 0.01–1 Hz and of varying amplitudes from 2 to 22°, which corresponds to horizontal specific forces of 0.34–3.68 ms⁻², respectively. The simulations were done for 30 min to compare the accumulated conflict at the end of this exposure. The simulation time and frequencies were chosen to match the available experimental data of Howarth and Griffin (2003).

Real-world sickening drive

A real-world slalom driving experiment was conducted by Irmak et al. (2020), where a passenger was driven through a route with slaloms of an amplitude of 0.4 g and a frequency of 0.2 Hz followed by braking and turning (Fig. 3).

Head motion signals in slalom driving (10–35 s) followed by braking and turning (40–50 s) from Irmak et al. (2020)

Full size image

Measured 3D head motion was always used as the vestibular input to the models, as well as the visual input when an external view is available (in ‘external’ and ‘only’ vision scenarios). For ‘internal vision’, the visual inputs are set to constant specific values: ‘visual vertical’ pointing downward and ‘visual rotational velocity’ set to zero. Lastly, in the case of ‘no vision’, the visual loops of the models are fully disabled.

Motion perception paradigm tests

Five fundamental motion perception paradigms were simulated to verify the fidelity of the models (Table 3, lower half). In all these paradigms, participants are subjected to passive motion without vision. They are requested to indicate the perceived velocity and/or perceived verticality through a handheld device. Thereby, such experiments demonstrate perceived motion resulting from the sensory integration of otolith and semicircular canal information.

Motion sickness frequency sensitivity analysis

Sickness with head translation

Figure 4 shows the frequency and amplitude sensitivity with sinusoidal translational head acceleration without vision (i.e., vestibular inputs only). For pure translation in the SVC models, identical results are obtained when the direction of acceleration is along the longitudinal (fore-aft), lateral (left–right), or vertical (up–down) axis. On the other hand, the MSOM has an identical response, conflict increasing with frequency, with longitudinal (fore-aft) and lateral (left–right) acceleration, but conflict does not depend on the frequency with vertical (up–down) acceleration.

Frequency (horizontal axis) and amplitude (vertical axis) sickness sensitivity in linear acceleration without vision. Comparison of experimentally observed MSI (%) with accumulated conflict (ms⁻²) from simulations of SVC_I, SVC_NI, and MSOM models. (Conflicts for SVC models are the same in horizontal and vertical motion)

Full size image

It can be observed in Fig. 4 that the peak conflict occurs around 0.16 Hz for the SVC-based models, which is consistent with experimental observations of McCauley et al. (1976) for vertical accelerations, as shown in Fig. 4e. In McCauley et al. (1976)’s experiments, participants had to keep their eyes open, which is analogous to our ‘Internal vision’ condition. Still, in Fig. 4, we have compared this experiment data to the modeled ‘No vision’ condition. We have done this for consistency with previous work, e.g. Kamiji et al. (2007), on benchmarking purely vestibular motion sickness models. Furthermore, currently no comprehensive ‘Internal vision’ dataset comparable to the McCauley et al. (1976) dataset is available. We show this comparison only to verify our implementation of the models.

While similar comprehensive data is not available for horizontal motion, still some data with either frequency or acceleration fixed is available as a reference. For longitudinal (fore-aft) motion, Irmak et al. (2020) have used motion perturbations at a peak acceleration of 2 ms⁻² and frequencies of 0.15, 0.2, 0.3, 0.4, and 0.5 Hz to measure motion sickness measured in terms of the MIsery SCale (MISC). Also, Irmak et al. (2022) have conducted experiments at a fixed frequency of 0.3 Hz and varying amplitudes of 1.0, 1.5, 2.0 and 2.5 ms⁻² to collect MISC ratings. Furthermore, Golding and Markey (1996) and Golding et al. (1997) have perturbed subjects with lateral motion with a peak acceleration of 3.6 ms⁻² and frequencies of 0.2, 0.35, 0.5, 0.7 and 1.0 Hz. A direct comparison of the data from these studies (black lines) with our model simulations (colored lines) is shown in Fig. 5.

Model predictions of frequency and amplitude sensitivity of motion Sickness for linear acceleration without vision. Data from various literature sources plotted for the group in black (in different line styles if there are multiple sets of data) (a, c, d) or individuals in various colors with transparency (b)

Full size image

It is interesting to note that the group-averaged results (Fig. 5d and a) are close to the integrated conflict from the simulations of the SVC-based models, while at an individual level (Fig. 5b, transparent lines), the differences can still be very large. It is to be noted that the SVC_I and SVC_NI model parameters are tuned by Kamiji et al. (2007) using data for vertical accelerations from McCauley et al. (1976), and hence may not hold true for other directions (Kamiji et al. 2007). It should be noted that all figures show the normalized conflict (normalized for each figure, not on the whole) to easily compare different models. Hence, we show the ability of the models to reproduce relative trends between conditions as a function of motion frequency and amplitude, and as a function of visual condition.

Figure 6 shows the effect of vision on the frequency and amplitude sensitivity with fore-aft sinusoidal accelerations for the compared models. In the left column (No Vision), the ‘VR’ and ‘VV’ loops are disabled and hence do not affect the results. Vision loops as implemented with visual rotational velocity (‘VR’) in the SVC_I, SVC_NI, and MSOM models do not affect the perception of fore-aft accelerations and hence have no effect on the development of motion sickness when subjected to purely translational accelerations (see ‘VR’ models in Fig. 6). However, when the visual vertical (‘VV’) loop is active, the responses differ from the ‘no vision’ case in particular at the lower frequencies. In the absence of rotations, the visual vertical remains unchanged (i.e., upright) which is expected to counteract the somatogravic illusion. However, this effect is only observed in the MSOM and not for the SVC models (see Sect. 3.3). Nonetheless, the visual vertical does cause a shift in the peak frequency of conflict to lower frequencies in the SVC models, with both ‘internal’ and ‘external’ vision. However, in the MSOM, which shows a limited effect of stimulus frequency on conflict without vision or with VR, adding the VV loop causes the effects of frequency on conflict to become marginal (Fig. 6). None of the models show any conflict in the ‘only vision’ case. All models with visual vertical input are sensitive to vision during pure translations. The influence of vision in pure translation motion in humans is not substantial, as observed by Butler and Griffin (2006), where no significant change in motion sickness was observed during pure sinusoidal fore-aft motion under various vision circumstances. Therefore, we conclude that the SVC models with visual rotational velocity (‘VR’) more closely replicate motion sickness during pure translation in response to changes in visual conditions.

Frequency sensitivity of integrated conflict (ms⁻¹) with sinusoidal fore-aft motion for different vision conditions: ‘no vision’ (eyes closed), ‘external vision’ (eyes open and with outside view), ‘internal vision’ (eyes open and without outside view), ‘only vision’ (no motion) and comparison of vision cases at 0.2 Hz and 0.7 g fore-aft motion for SVC_I, SVC_NI and MSOM (Plots for the’no vision’ case are the same as in Fig. 4)

Full size image

Sickness with head rotation

Pure head pitch or roll rotations can result in equivalent changes in the subjective vertical orientation as those that result from linear horizontal (fore-aft or left–right) accelerations. To compare the effects of rotational inputs in the different models, the predicted frequency sensitivity of motion sickness with varying vision conditions is shown in Fig. 7. The responses to pitch and roll in all models are identical due to identical parameters for these two DOFs. It is to be noted that yaw rotations do not affect the perception of verticality by the vestibular system in these models and hence will have an entirely different response, which is addressed later in this section.

Frequency sensitivity of integrated conflict (ms⁻¹) with sinusoidal pitch motion for different vision conditions: ‘no vision’ (eyes closed), ‘external vision’ (eyes open and with outside view), ‘internal vision’ (eyes open and without outside view), ‘only vision’ (no motion) and comparison of vision cases at 0.2 Hz and 20^◦ pitch motion for SVC_I, SVC_NI and MSOM

Full size image

The effect of vision on conflicts in rotation shown in Fig. 7 is much stronger compared to the effect of vision in translation shown in Fig. 6. This difference is due to the nature of the vision inputs: they are assumed zero for the translation case, but non-zero and sinusoidal for the rotation case, which amplifies the effect of vision during rotations. It can be observed that the presence of ‘external vision’ results in a reduction in conflict as compared to the ‘no vision’ case for six out of nine models, which is in line with experimental results of Wada et al. (2020), Wada and Yoshida (2016) and Griffin and Newman (2004). This is followed by ‘only vision’, and lastly ‘internal vision’ in the SVC_I-VR and SVC_NI-VR models. The ‘internal vision’ case shows the largest conflicts across all models. This is due to the contradicting signals between the visual and the vestibular sensors. One thing that is common among all plots of SVC_I and SVC_NI models is that the peak conflict frequency is at 0.2 Hz. This peak conflict frequency is the same as the peak motion sickness frequency observed by O’Hanlon and McCauley (1974) in vertical motion in ships. This is achieved by careful tuning of parameters by Wada et al. (2020); Liu et al. (2022); Inoue et al. (2022) and further verified in Appendix A.1 where it was found that the parameters as reported in Wada et al. (2020), Liu et al. (2022), Inoue et al. (2022) are optimal for imitating frequency dynamics of motion sickness. The MSOM, however, has very different frequency dynamics that predict increased sickness with higher stimulus frequencies for the ‘no vision’ and ‘external vision’ cases in the MSOM-VV and MSOM-VR + VV. For the ‘internal vision’ and ‘only vision’ cases, the same models show a frequency sensitivity that is invariant with frequency. As a clear peak sensitivity frequency is lacking, both these results are in disagreement with available experiment data. This is due to the selection of otolith conflict as the best proxy for motion sickness, by Irmak et al. (2023). This choice may not be suitable for predicting motion sickness in conditions with vision, which Irmak et al. (2023) did not investigate. This has been further discussed in Sects. 4.1 and 4.2.

The simulation results in Fig. 7 can, however, only be partially verified, as published data is lacking on motion sickness frequency and amplitude sensitivity in rotational motion paradigms. The most closely related data is from Howarth and Griffin (2003), where motion sickness during roll motion was evaluated at frequencies of 0.025, 0.05, 0.1, 0.2, and 0.4 Hz and a peak amplitude of 8°. The experiment data and the results from the model simulations are shown in Fig. 8. Even though the frequency of 0.2 Hz had the highest number of people reaching an illness rating (IR) of 2 and above, no significant effect of frequency was found by Howarth and Griffin (2003). The peak conflict frequency observed in our simulations in the SVC_I and SVC_NI model simulations are 0.2 Hz as well. However, the drop in sickness is much steeper in the models than in the data from Howarth and Griffin (2003). The MSOM, on the other hand, shows a response more like a high-pass filter, which is very different from the SVC models and the experiment data.

Model predictions of the frequency sensitivity for sinusoidal roll motion with an amplitude of 8° with eyes closed (‘no vision’ case) compared with experimental data from Howarth and Griffin (2003) showing the number of participants (out of 20) who reached an illness rating of 2

Full size image

In addition to the results in Figs. 7 and 8, the conflicts with pitch motion are compared to linear accelerations providing an identical horizontal component of the specific force. These have been shown in Fig. 9. The expected result is a smaller conflict for pitch motion than for linear accelerations (Howarth and Griffin 2003), even though pitch angles of 5.74–44.43° correspond directly to longitudinal specific forces of 0.1–0.7 g. In SVC_I and SVC_NI models, the peak conflict frequency has shifted from 0.16 to 0.2 Hz, with no difference in conflict magnitude for the SVC_I model and the SVC_NI model. In MSOM, there is an increase (over 2 times) in conflict magnitude for the pitch input at low frequencies. This is because the otolith conflict, used as a proxy for motion sickness, has a low sensitivity to rotations. This makes the MSOM, while using the otolith conflict as a proxy for motion sickness, unsuitable for motion sickness predictions in cases with high rotational velocities, like in vehicles. Experimental data with motion sickness at various frequencies and amplitudes are not available to verify the predicted difference in conflict magnitude.

Integrated conflict (ms⁻¹) with pitch motion for pitch angles, of 5.74–44.43° (bottom row), are compared to linear accelerations, of 0.1–0.7 g’s (top row, also shown in Fig. 6 with a different scale), providing an identical horizontal component of the specific force

Full size image

A very important assumption in the Subjective Vertical model by Bos and Bles (1998) and all SVC-based models tested in this paper, is that motion sickness occurs due to a conflict in the perceived and expected vertical (both direction and magnitude). As pure yaw rotations will not affect the perception of verticality, no conflict and subsequently motion sickness accumulation will be predicted by SVC-based models. The same is true for the MSOM, as the otolith conflict is only affected when the orientation vector is altered, which is not the case during pure yaw rotations. To verify this, all models were simulated for pure yaw motion, and the results showed that no conflict was predicted in any of the vision conditions, as expected, see Fig. 19 in the Appendix A.2. However, this predicted absence of motion sickness during yaw motion is in contradiction with Golding et al. (2009), where 9 out of 12 participants reached a sickness rating of 2 while being earth vertical, and a larger sickness when the visual stimulus was tilted. A potential explanation is that perfect verticality is impossible to achieve experimentally, e.g., due to other motions such as misalignment of the head with the axis of rotation. To explore this, and to investigate the effect of visual feedback, we simulated yaw motion with a constant pitch attitude of 10° for all models and vision conditions.

Figure 10 shows that a slight pitch attitude indeed results in substantial conflict in the ‘no vision’ condition, as expected based on Golding et al. (2009). In the model predictions, the highest conflict is found in ‘internal vision’ case, as expected due to the substantial disparity between visual and vestibular inputs. Unexpectedly, some conditions show a constant conflict being invariant with frequency and amplitude of the applied yaw motion. In these cases, the conflict depended only on the applied constant pitch. This applies to ‘internal vision’ and ‘only vision’ conditions for models with a ‘VV’ loop. This result indicating conflict in static pitch with sinusoidal yaw motion is not in agreement with experimental data. This effect occurs due to the constant conflict between the visual and vestibular estimates, a consequence of their constant inputs. For instance, in the case of ‘internal vision,’ the vestibular inputs maintain a constant value representing the true specific force following a 10° pitch, while the visual input retains a specific force vector pointing downward ([0, 0, − 9.81]). In contrast, in the ‘only vision’ case, these are reversed. Here the visual input is the true specific force following a 10° pitch and vestibular input is a specific force vector pointing downward ([0, 0, − 9.81]). For the case of ‘only vision’, all three models with visual rotational velocity input show negligible conflict. This is because of the constant pitch with the visual rotational velocity loop, which only produces yaw angular velocity conflict. This does not influence the otolith and subjective vertical conflict. It is to be noted that the visual rotational velocity loop is only concerned with rotation velocities and does not account for lateral specific forces, induced due to changes in rotation after each cycle. These, however, are captured by the visual vertical feedback loop as explained earlier.

Frequency sensitivity of integrated conflict (ms⁻¹) with sinusoidal yaw with a constant pitch of 10° for different vision conditions of—‘no vision’ (eyes closed), ‘external vision’ (eyes open and looking out of the car), ‘internal vision’ (eyes open and on objects inside the car), ‘only vision’ (no motion) for SVC_I, SVC_NI and MSOM

Full size image

Overall, for yaw motion with a constant pitch of 10^◦, the MSOM predicts smaller conflicts compared to the SVC_I and SVC_NI models with the conflict increasing with frequency. This is again due to the lower sensitivity of the MSOM’s otolith conflict to rotations. By interpreting the results, it can be deduced that in yaw motion, the SVC_I -VR and SVC_NI -VR models better match the expected relative sickness incidence for different vision conditions than the MSOM.

Motion sickness predictions for real-world sickening drive

Slalom experimental data from Irmak et al. (2020), as introduced in Sect. 2.2, was used to verify the accuracy of the models’ motion sickness predictions for real-world driving scenarios. These data include measured head accelerations and rotations that are used as input to the models.

Figure 11 shows the integrated subjective vertical conflict generated by all the models for the considered ‘external vision’, ‘internal vision’, ‘only vision’, and ‘no vision’ cases. For the SVC_I models, conflicts in ‘internal vision’ and ‘only vision’ are observed to be lower in the SVC_I-VV model, followed by the SVC_I-VR model, and the highest in SVC_I-VR + VV. This suggests that conflict contributions from the different vision loops are cumulative in the combined SVC_I-VR + VV model. For the ‘external vision’ and ‘no vision’ cases, the conflict is the same for all three versions of the SVC_I model. This is as expected: in the ‘no vision’ case, the vision loops do not affect the responses, while for ‘external vision’ the vision matches the vestibular motion inputs, and conflict is always minimized. In all three versions (i.e., VR, VV, and VR + VV) of the SVC_I model, the ‘no vision’ and ‘external vision’ case responses show minimal differences, while the responses of the ‘internal vision’ case always show the largest conflict. This matches the expected effect of vision on motion sickness. Although the SVC_I-VV model correctly predicts the expected order of severity for the other vision cases, it shows the least amount of conflict for the ‘only vision’ scenario, which is contrary to expectations.

Effect of vision on the accumulated conflict (ms⁻²) for SVC_I, SVC_NI and MSOM during slalom drive of 47 s for ‘no’, ‘external’, ‘internal’, and ‘only’ vision conditions; Red cross—Incorrect, Yellow Exclamation—Uncertain, Green Tick—Correct order of responses for vision conditions

Full size image

The SVC_NI model also predicts that the ‘external vision’ case is less sickening than the ‘internal vision’ and ‘only vision’ cases. However, the ‘no vision’ case shows, unexpectedly, the largest conflicts of all compared cases. This is due to the increase in the magnitude of acceleration conflicts as a result of removing the integration in the acceleration feedback loop. Hence, the SVC_NI sacrifices accuracy in replicating conflicts.

For the MSOM model, Fig. 11 shows that in the ‘only vision’ case, the predicted levels of motion sickness are low. Conversely, the responses for the other vision conditions are essentially the same. This implies that the main contributor to the otolith conflict in the MSOM is the physical motion perceived by the vestibular system and not the contributions from vision.

From the results of all three models in Fig. 11, the SVC_I models seem best for the prediction of the effect of vision on motion sickness, as the order of severity of the different considered vision conditions matches with expectation. The SVC_NI models, however, predict that ‘internal vision’ is less or about the same sickening as the ‘only vision’ case, which is contrary to the expected effects. For the MSOM, the otolith conflict term selected by Irmak et al. (2023) as a predictor for motion sickness, is found to be unsuitable for motion sickness simulations with vision, as it does not output the expected order of motion sickness severity of vision conditions in the model’s output. These results are consistent with the observations made in Sect. 3.1 where the SVC_I model was found to be reliable in replicating sickness results from the literature with a plausible order of severity of vision conditions.

Motion perception paradigm tests

To further evaluate the realism of the models’ simulated perception mechanisms that predict motion sickness, the models’ capacity for explaining well-known motion perception responses in fundamental motion perception paradigms, and how the outcomes vary due to the presence of the considered vision loops, was investigated. A summary of the outcomes is shown in Table 4; the detailed results are discussed per paradigm in the remainder of this section.

EVAR (Earth vertical axis rotation) and OVAR (off-vertical axis rotation)

As observed by Waespe and Henn (1977) in monkeys and reproduced here in Fig. 12, it is expected that the perception of rotational velocity in EVAR will converge to a value that approximates the true rotational velocity when vision is active. In the absence of vision, however, the perception of rotation should decay exponentially to zero. Even though similar measured neural responses are not available, the same effects of vision on self-motion perception in EVAR have been observed in humans [e.g., (van der Steen 1998)]. Pure yaw motion, by itself, does not impact the visually perceived orientation relative to gravity. Therefore, in models with visual vertical as the sole visual input, the correct prediction of the expected visual effect is not expected. Figure 12 shows the models’ results for the angular velocity perception, which are similar across all three models because of the similar visual loop implementation. As expected, and consistent with the findings of Waespe and Henn (1977), the models that include a visual rotational velocity input (‘VR’) predict the well-known variation in perceived rotational velocity for the different vision conditions. The models with only visual vertical input (‘VV’) have an identical response for all vision conditions involving physical motion, i.e., all except ‘only vision’ for which the models show zero perception of rotational velocity. The models with only visual vertical are thus indeed not affected by the vision condition. The model responses are the same as the case with ‘no vision’ except for the case of ‘only vision’, which has no response due to the visual vertical not registering any yaw rotations. These results show that the models with visual rotational velocity (‘VR’) more realistically model the effect of varying vision conditions in EVAR. This is consistent across all the models including SVC_I, SVC_NI, and MSOM.

Angular velocity perception during EVAR at 60° s⁻¹; Red cross—Result in disagreement with literature, Green Tick—Result in agreement with literature

Full size image

In OVAR, it is expected, based on the findings of Vingerhoets et al. (2006), that the perception of angular velocity in the dark (‘internal vision’) will decay exponentially over time. Perception responses for other vision conditions are not available in the literature. The model predictions show that the perception of angular velocity during OVAR is identical to EVAR as in Fig. 12 and are hence not shown.

Somatogravic illusion

The ‘somatogravic illusion’ is the phenomenon where, in absence of visual cues, low-frequency forward linear accelerations are incorrectly perceived as changes in pitch angle (tilt); lateral accelerations similarly induce a perception of roll. This effect was, for example, observed and quantified in Correia Gracio et al. (2013). Furthermore, Tokumaru et al. (1998) found that the strength of the somatogravic illusion was reduced in the presence of a visible horizon (‘external vision’). However, the presence of a vection stimulus (‘only vision’) did not cause a similar illusory effect. It is expected that this illusion occurs when there is no outside view, i.e. ‘internal vision’ and ‘no vision’. This is an important motion perception test, as somatogravic illusion is known to be closely linked to spatial disorientation (Groen et al. 2022) as well as motion sickness induced by accelerations in the horizontal plane (Irmak et al. 2021; Wood 2002; Wood et al. 2007).

Figure 13 shows the response of the tested models for this paradigm. The lower graphs in each subplot show the input acceleration in bold black, along with perceived linear accelerations in the different vision conditions of ‘external vision’ (blue), ‘internal vision’ (red), ‘only vision’ (yellow), and ‘no vision’ (purple). The upper graph shows the corresponding perceived pitch angles for the different vision conditions and also the pitch angle corresponding to the gravito-inertial force vector tilt, which is equal to 22.18° for the 4 ms⁻² forward acceleration. Contrary to expectations, Fig. 13 shows that the SVC_I model predicts this illusion to occur for all vision cases, i.e., the responses for all SVC_NI models are identical. Furthermore, the SVC_I model also does not show the capability to model this illusion in the presence of vision. Even though adding the visual vertical input (‘VV’) reduces the perception of pitch in the presence of vision, it is still not reduced to zero, which is the expected output. Thus, both visual inputs do not help in the perception of acceleration in SVC-based models. However, the vision input does affect the perception of acceleration in the MSOM with visual vertical input. The response of the MSOM-VV is exactly as expected based on existing literature, with the illusion occurring only during the ‘no vision’ case. This shows, for the first time in our analysis, that the visual vertical input positively contributes to predicting human motion perception responses in motion perception models. This is carried forward into the MSOM-VR + VV where the visual vertical again helps in capturing this illusion. Thus, the results in Fig. 13 show that only the MSOM with the visual vertical input (i.e., MSOM-VV and MSOM-VR + VV) is able to accurately predict the expected variation in the somatogravic illusion due to vision.

Pitch perception during constant acceleration of 4 ms⁻² (somatogravic illusion); For each condition the upper graph shows the perceived pitch and the lower graph the perceived acceleration. Black lines describe the applied acceleration in the lower graph, and the equivalent rotation in the upper graph. Red cross—Result in disagreement with literature, Green Tick—Result in agreement with literature, Yellow Exclamation—Some of the results are in agreement with literature

Full size image

Centrifugation

When humans are rotated in a centrifuge facing the direction of the local velocity vector in absence of visual cues, they perceive a roll tilt Merfeld et al. (2001). This tilt perception is induced by the constant (lateral) centrifugal force’s contribution to the specific force vector perceived with the otoliths, from which humans are unable to differentiate the inertial and gravitational parts. However, in the presence of upright visuals, no roll-tilt is perceived. This well-known suppression of tilt perception is effectively used in moving-base vehicle simulators through tilt-coordination (Berger et al. 2010). Figure 14 shows that the different tested models provide a different response for each vision condition. Furthermore, the SVC_I model is found to be unable to predict the expected roll-tilt perception, while the SVC_NI (Inoue et al. 2022) does show the expected response for the ‘no vision’ case. However, in the presence of ‘external vision’, the responses of the SVC_NI model show a strong perception of roll tilt, which is not the case in real life. The MSOM, on the other hand, is able to simulate all vision cases accurately, showing no roll tilt for the perception for the ‘external vision’ and ‘only vision’ cases, but the expected tilt for the ‘internal vision’ and ‘no vision’ cases. The only exception is the MSOM with only visual vertical (MSOM-VV), which does not predict tilt perception for ‘internal vision’. Thus, only the MSOM with visual rotational velocity (MSOM- VR and MSOM-VR + VV) is able to accurately capture the effect of roll tilt perception in a centrifuge paradigm.

Roll-tilt perception during centrifuge at 250° s⁻¹; The black lines describe the applied motion translated to an equivalent tilt angle. Red cross—Result in disagreement with literature, Green Tick—Result in agreement with literature, Yellow Exclamation—Some of the results are in agreement with literature

Full size image

Pseudo-Coriolis

The pseudo-Coriolis perception paradigm (Dichgans and Brandt 2009) is elicited by tilting the head out of the axis of rotation of a circular moving surrounding visual. This tilting of the head elicits a stimulus in the third (unexcited) axis of rotation. The resulting sensation is identical to that which arises during actual rotating motions (i.e., Coriolis). Figure 15 shows that only the MSOM model with visual rotational velocity (MSOM-VR and MSOM-VR + VV) is able to capture the excitation of the third rotational axis, shown by the pitch angle and pitch velocity perception. The models with visual vertical (SVC_I-VV, SVC_NI-VV, and MSOM-VV) are seen to be insensitive to the visual yaw rotation, as this motion is not captured by the model’s inputs: the visual vertical input is unaffected by yaw rotations and even when there is a head tilt, there may be roll angle perception, but there is no roll or pitch velocity perceived by the model. The SVC_I models with visual rotational velocity inputs (visual rotational velocity) do show pitch responses, but the perceived rotation angles do not return to zero after the end of the stimulus, which is unrealistic. The SVC_NI model, on the other hand, does show a response to the stimulus and convergence back to zero, however, the responses are oscillatory with ‘VV’ active. Figure 15 shows that a visual rotational velocity input is required for predicting human motion perception during the pseudo-Coriolis paradigm. However, there is no literature with continuous measurement of the perceived pitch rotation angle or rotational velocity during pseudo-Coriolis to validate these model responses.

Angular velocity, pitch angle, and roll angle perception during Pseudo Coriolis stimulation at yaw rotation of 138° s⁻¹ with a stimulus of 45^◦ head tilt applied from 42 to 52 s; Red cross—Result in disagreement with literature, Green Tick—Result in agreement with literature, Yellow Exclamation—Some of the results are in agreement with literature

Full size image

For the first time, the implementation of the effects of vision in state-of-the-art motion sickness and motion perception models was broadly validated. Vision loops were selectively disabled or enabled to compare the models’ responses to various stimuli. These included the well-known frequency sensitivity of motion sickness, sickness in a slalom drive, and perception responses in fundamental motion paradigms. Of the nine tested model variants, none was able to reproduce all experimental sickness and perception results. All models showed (some) realistic effects of vision, but all were unable to reproduce at least one experimental finding. Overall, the SVC best matched the experimental motion sickness data, whereas the MSOM showed the best match for motion perception. In the following, findings will be discussed and recommendations will be formulated on the most critical and promising model structures for simulating motion perception and motion sickness.

In this paper, the models were validated using the parameters reported in their respective publications. While enhancing modeling accuracy through parameter optimization was possible, it falls beyond the paper’s scope. For simplicity in comparing the VV and VR loops in the SVC and MSOM models, we utilized the models’ original published forms and parameters. Consequently, these results should not be generalized to SVC or MSOM models with different parameters. However, while adhering to these defined parameters, we have achieved meaningful insights and valuable findings in this context.

Motion sickness frequency sensitivity

Comparing the frequency and amplitude sensitivity of motion sickness incidence in the vertical direction, from McCauley et al., (1976), to the conflict predictions from motion sickness and motion perception models (SVC_I, SVC_NI, and MSOM), it is observed that the SVC_I and SVC_NI accurately capture the frequency and amplitude dynamics. In its current published form (Newman 2009; Clark et al. 2019) and with the acceleration conflict used as a proxy for motion sickness Irmak et al. (2023), the MSOM cannot capture this frequency dynamics and predicts no frequency dependency of conflict in the vertical direction due to the choice of the otolith conflict as an indicator of motion sickness, which is not affected by purely vertical accelerations.

Visual cues are implemented in two ways: visual rotational velocity (‘VR’) and visual vertical (‘VV’). The sensitivity of the predicted conflicts to the gains of each visual loop in the three models was also tested, see appendix A.1. The results show that the gains reported by the respective authors are optimal for sickness simulations. To analyze the effects of vision, frequency sensitivity analysis during pitch and fore-aft motion was carried out. A small effect of vision on the conflicts due to translation was observed (see Fig. 6). However, a large effect of vision on the conflicts due to rotations was observed (see Fig. 7). It was observed that only the SVC models with visual rotational velocity (namely, SVC_I-VR and SVC_NI-VR) are able to accurately model the sickness severity of various vision conditions relative to each other (from most severe to least severe—‘internal vision’, ‘only vision’, ‘no vision’ and ‘external vision’ (Irmak et al. 2020; Wada and Yoshida 2016; Griffin and Newman 2004)). The MSOM, though models the order of vision severity correctly, produces low levels of conflict during pitch motion as compared to SVC models. The peak conflict frequency in the MSOM during pitch motion, around 1 Hz, remains unchanged when vision loops are added. Also, in for-aft motion, the peak conflict frequency is around 1 Hz, but this changes in the case of visual vertical loop is added. Nonetheless, the peak frequency of 1 Hz is very different from what is found in the literature (see Fig. 4e). This may be due to the inherent frequency sensitivity of the conflict term used. Using other conflict terms also does not aid in improving the frequency response of the MSOM (Irmak et al. 2023). This different peak conflict frequency combined with its no sensitivity to vertical accelerations shows that the MSOM is not (yet) suitable for motion sickness simulations.

Unfortunately, to the best of our knowledge, there exists only a single experimental dataset (Howarth and Griffin 2003) to show the frequency sensitivity of motion sickness with rotations. Hence, it is difficult to conclude with certainty which of the models is accurate. Hence, there is a need to plan and carry out experiments to capture the frequency sensitivity with varying pitch/roll angular velocities. Along with this, motion sickness data at different cases of visual stimuli (‘external’, ‘internal’, ‘no’, and ‘only’ vision) need to be investigated to better understand the effect of vision and verify the effects of vision on frequency sensitivity of motion sickness as predicted by Figs. 4, 5, 6, 7, 8, 9.

Another important conclusion is that none of the models, the SVC_I, SVC_NI, and MSOM, show any conflict in vertical due to pure yaw rotations. While there may be conflict generated between perceived and estimated rotational velocities, this is not used for motion sickness predictions. However, this contradicts the finding of Golding et al. (2009), who showed that while being earth vertical and with optokinetic stimulation, motion sickness is still observed during yaw motion: 9 out of 12 participants reached a sickness rating of 2. There is a possibility that some degree of head tilt occurred, leading to an imperfect alignment with the rotation axis, as mentioned in Bos et al. (2008). However, if this were the case, we would expect a significant increase in motion sickness scores when the tilt is introduced, which is not observed in Golding et al. (2009). Another plausible explanation is the presence of inherent irregularities or asymmetry in the vestibular organs, potentially causing motion sickness during pure yaw motion. Nevertheless, it is reasonable to assume that the human brain habituates to such irregularities, updates an internal model to account for the affected vestibular organs, and compensates for them. It is noteworthy that none of the models considered this adaptation process. Alternatively, it is conceivable that multiple sources of sensory conflict exist, with specific conflicts contributing to motion sickness during yaw motion as in Khalid et al. (2011). Additionally, it was found that even a small constant pitch of 10^◦ during yaw motions can incite substantial levels of motion sickness in all models, as has been also verified by Golding et al. (2009). This was also backed up by the conflict generation in high yaw motions during slalom maneuvers as shown in Fig. 11 and Appendix A.3.

Motion sickness in slalom drive

Until now these motion sickness and motion perception models were only tested either by using fundamental inputs like a sine wave (Kamiji et al. 2007; Wada et al. 2020) or by real-world driving data while ignoring the vision loops Yunus et al. (2022); Wada et al. (2015); Wada and Yoshida (2016). In this paper, the ability of the vision loops to differentiate various vision conditions was studied using experimental data by Irmak et al. (2020). It is found that only the SVC_I-VR and SVC_I-VR + VV models are able to accurately model the varying sickness severity of various vision conditions (from most severe to least severe: ‘internal vision’, ‘only vision’, ‘no vision’, and ‘external vision’ (Irmak et al. 2020; Wada and Yoshida 2016; Griffin and Newman 2004)). The use of only visual vertical (SVC_I-VV) deteriorates the ability of the SVC_I model by increasing the conflict during the ‘external vision’ case. Hence, it is necessary for the SVC_I model to have visual rotational velocity as an input to capture the effects of vision. The SVC_NI models, however, predict an incorrect order of severity of vision conditions. Possibly the parameters need to be further tuned to improve performance in vision conditions as opposed to only in case of ‘no vision’ as was done in Inoue et al. (2022).

The MSOM does predict sickness in the slalom drive but predicts small any effects of vision. In the MSOM, the otolith conflict term selected by Irmak et al. (2023) is found to be unsuitable for motion sickness prediction with vision, as it does not show the expected sensitivity to vision in the model’s output. This choice may not be suitable for predicting motion sickness in conditions with vision, which Irmak et al. (2023) did not investigate. Hence, alternate conflict terms, like a combination with angular velocity conflict, can be considered. This was also recently proposed by Allred and Clark (2023), who used a weighted sum of various conflict terms of MSOM. They found the highest weighting factor for conflict in f_v (GIF in their paper) as compared to conflict in ω and conflict in f_oto (a in their paper). However, we expect ∆f_oto and ∆f_v to yield similar results as they derive from the same signals. Allred and Clark (2023) do not explicitly reflect on the consequences for their model’s fit when any of these two conflict terms are omitted. Also, they do not make any comparisons with other models, such as the SVC_I model, which our paper shows to have better motion sickness frequency dynamics as compared to the MSOM.

In addition to comparing the effects of vision, the contribution of each degree of freedom to the conflict was investigated (see Appendix A.3). This was done by switching each degree of freedom off and seeing its effect. This revealed important insights. It was observed that the conflict from SVC_I and SVC_NI models have a low sensitivity on linear degrees of freedom (translational motion). The MSOM, on the other hand, is highly sensitive to linear degrees of freedom while not so sensitive to rotation degrees of freedom. It was hence concluded that only the SVC_I-VR is able to match sickness and how it is affected by vision in this naturalistic driving dataset.

Motion perception tests

From the motion perception tests, it is evident that MSOM-VR + VV can predict the effects of vision in all motion paradigms. The visual rotational velocity (VR) input is essential for capturing the effects of vision on human motion perception. However, the visual vertical (VV) input is only useful during the somatogravic illusion, and only in the MSOM. It is understandable that visual vertical will not be of any help during yaw angular velocity perception, as there is no feedback from the visual vertical due to no change in verticality. In SVC-based models, the visual vertical is not even able to help in cases of rotation angle or acceleration perception (as in somatogravic illusion (Fig. 13), centrifuge (Fig. 14) and pseudo-Coriolis (Fig. 15)). The visual vertical does neither perform well for motion perception tests nor does it aid the visual rotational velocity when the combined approach is used in the VR + VV models. This was expected from models based on SVC, as these were not designed for motion perception; rather they were designed with the sole purpose of fore- casting motion sickness. In the SVC-based models, there exists feedback from the semicircular canals to the otoliths, but not the other way around. This is the reason that the SVC-based models show only a small perception of pitch during somatogravic illusion in dark (‘internal vision’). The MSOM, however, is the best out of all the models, as it accurately predicts all considered perception paradigms. Also, the visual vertical loop actually helped in estimating pitch (bringing it down from 22.2° to 1.7°) and acceleration (increasing it from 0 to 2.9 ms⁻²) during somatogravic illusion (see Fig. 13). This shows the first evidence of visual vertical aiding in the simulations of motion perception in our analysis. This superior performance in motion perception tests as compared to the SVC models was expected, as the MSOM was designed to be a motion perception model and not a motion sickness model like the SVC-based models. This is also supported by Groen et al. (2022), where the MSOM reliably predicted the occurrence of somatogravic illusion in an airplane accident investigation. This advantage in modeling motion perception does not translate into motion sickness simulation, however, for which the MSOM performs poorly.

The SVC_NI model was developed with the intention of improving the motion perception quality of the SVC_I (Inoue et al. 2022). However, as seen from the results for the somatogravic illusion and centrifugation paradigms tested in our paper (Figs. 13 and 14), the model only showed improvement for the ‘no vision’ case. When vision loops are introduced, the responses are not accurate. This indicates that there is room to improve the SVC-based models, specifically for the cases with vision. One of the possible solutions is to add feedback from the otoliths to the semicircular canals in the SVC-based models to induce a perception of tilt when accelerated, which the current SVC-based models do not account for.

Individual vs. group-averaged models

In this paper, the integrated subjective vertical conflict, as predicted by sensory integration models, was used as a proxy for experimental Motion Sickness Incidence (MSI), a key metric for quantifying motion sickness evolution McCauley et al. (1976); Bos and Bles (1998). However, MSI is a group-averaged metric and is not representative of an individual’s response. For these models to be used in controlling motion comfort in automated vehicles, MSI is not ideal as it targets the average severity of sickness. MSI could be used to design controllers that output sickness levels for 50% of the users. However, this will ignore users outside the envelope of average susceptibility. Thus, we need models that also predict the lower/higher sickness levels, and capture variations between individuals. Thus, using an individual-specific metric like MIsery Scale (MISC), as proposed by Bos et al. (2010), will help not only solve the aforementioned problem, but also enable an improved understanding of how diverse the model parameters and subsequently the responses to a given stimulus are. This has already been shown by Irmak et al. (2020), where individual MISC responses were fitted to the Oman (1990) model and confirmed that individual models reduce prediction errors by a factor of two as compared to group-based models. This was further improved upon by Kotian et al. (2023) where a combination of SVC_I-VR and Oman model greatly increased fitting accuracy in varying vision conditions as well. Hence, we emphasize the importance of using the MISC as a metric in future motion sickness studies. The next step of the modeling will thus be to combine such conflict generation models with visual inputs with a conflict accumulation model to be able to predict an individual’s motion sickness level, in terms of a MISC score across varying vision conditions.

Comparison of models and their visual loops

Studies from Krapp and Hengstenberg (1996); Tokumaru et al. (1998); Bos et al. (2008) imply that we need to have both visual vertical and visual rotational velocity for estimation of self-motion. Visual vertical provides a visual reference for the direction of verticality and is affected by both rotations and linear accelerations. Visual rotational velocity provides visual angular velocity perception and is only induced by rotations. Visual rotation velocity was shown to be essential in both sickness prediction and motion perception prediction in SVC-based and MSOM models. Visual vertical helps only in the case of the MSOM, where it improves the predicted perception of tilt during the somatogravic illusion paradigm. Visual vertical has some effects on sickness prediction with SVC-based models, but given the lack of experimental data, we cannot yet conclude whether visual vertical can enhance sickness predictions. However, practically in our simulations of the aforementioned models, we have found no convincing use of visual vertical in motion sickness simulations of SVC-based models. This is due to the SVC-based models not having feedback from the otoliths to the semicircular canals, which may help in reducing tilt perception when visual vertical is included. Hence, changes need to be made in the SVC-based models to more realistically include the effects of visual vertical.

The recent Visual-Vestibular Motion Sickness (VVMS) model by Jalgaonkar et al (2021), Sousa Schulman et al (2023) has almost the same structure as the SVC model by Kamiji et al. (2007), but integrates vision directly in the sensory part of the model. Appendix B shows that sickness results are identical to the SVC_I-VR without vision. However, the inclusion of visual input does have an impact on sickness results, although not in the expected order of severity with vision. Thus, in its current form the VVMS is not a better predictor of vision effects in motion sickness than the SVCI -VR model.

Comparing the models, it is evident that SVC models with a visual rotational velocity loop should be used for motion sickness predictions. Adding the visual vertical loop has very limited, and partially negative, effects making SVC_I-VR the recommended model for motion sickness prediction. However, for the tested motion perception paradigms, the MSOM with both vision loops (MSOM-VR + VV) performs best. Thus, there exists no universal model to simulate both motion sickness and motion perception. In our recent paper (Happee et al. 2023), we also evaluated the suitability of MSOM and SVC_I- VR and SVC_I-VR + VV to explain neck stabilization across a range of passive translational and rotational motion conditions. Here we found both MSOM and SVC_I-VR + VV to well explain how vestibular and visual information is integrated for postural stabilization, where the correspondence with human postural stabilization data was not very sensitive towards model type or parameters, but the SVC_I-VR, did not correctly capture postural stabilization. This supports the idea that one unified model of sensory integration could explain motion perception, motion sickness and postural stabilization. To create such a unified model, one possible solution could be to implement better otolith-semicircular canal interactions in SVC models as done in the MSOM. This would help to better capture tilt perception during special motion paradigms such as roll tilt perception during centrifuge and pseudo-Coriolis. Another solution could be to apply a band-pass filter to the conflict term of the MSOM, thereby adjusting the frequency responses. However, to accurately see the effects of such modifications, more experiment data, especially with head rotation and under varying vision conditions, are sorely needed.

The goal of this paper was to validate the effects of vision as currently modeled in state-of-the-art motion sickness and motion perception models, i.e., the Subjective Vertical Conflict model with Integration of acceleration conflict (SVC_I) (Wada et al. 2015; Liu et al. 2022), the Subjective Vertical Conflict model with No Integration of acceleration conflict (SVC_NI) (Inoue et al. 2022), and the Multi-Sensory Observer Model (MSOM) (Newman 2009). The SVC_I -VR model, which includes visual rotational velocity perception, best predicts experimental data for the effects of different vision conditions on motion sickness development. However, at the perceptual level, the SVC_I -VR model’s predictions do not match available experiment data for a number of tested paradigms (i.e., somatogravic illusion, tilt perception in a centrifuge). All motion perception paradigm data are accurately matched by the tested MSOM-VR + VV, which includes both visual rotational velocity and visual orientation perception, however, no correct frequency sensitivity of motion sickness is shown in the MSOM. Thus, the performed model comparison shows that no single model exists that can accurately predict the effects of vision on motion sickness and motion perception. Our next steps include expanding our model comparison effort to include other conflict terms such as in (Allred and Clark 2023). Here we expect the model by Allred and Clark (2023) to provide results similar to our results for the MSOM when comparing the predicted conflicts. The main improvement comes from the use of the nonlinear function by Oman (1990) which greatly improves the motion sickness frequency dynamics as shown in Kotian et al (2023). Crucial steps towards realizing such a unified model are, based on the analysis in this paper, the implementation of more complete otolith-semicircular canal interactions in SVC-based models such as the SVC_I-VR, adding a band-pass filter to correct the frequency dynamics of the MSOM. Along with this, future experiments will be directed towards addressing the gaps in the existing literature identified in this study.