- 1School of Psychology, University of Wollongong, Wollongong, NSW, Australia
- 2Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada
- 3Division of Clinical Psychology, Faculty of Child Development, Nihon Fukushi University, Nagoya, Japan
- 4Research School of Psychology, Australian National University, Canberra, ACT, Australia
Visual-vestibular conflicts have been traditionally used to explain both perceptions of self-motion and experiences of motion sickness. However, sensory conflict theories have been challenged by findings that adding simulated viewpoint jitter to inducing displays enhances (rather than reduces or destroys) visual illusions of self-motion experienced by stationary observers. One possible explanation of this jitter advantage for vection is that jittering optic flows are more ecological than smooth displays. Despite the intuitive appeal of this idea, it has proven difficult to test. Here we compared subjective experiences generated by jittering and smooth radial flows when observers were exposed to either visual-only or multisensory self-motion stimulations. The display jitter (if present) was generated in real-time by updating the virtual computer-graphics camera position to match the observer’s tracked head motions when treadmill walking or walking in place, or was a playback of these head motions when standing still. As expected, the (more naturalistic) treadmill walking and the (less naturalistic) walking in place were found to generate very different physical head jitters. However, contrary to the ecological account of the phenomenon, playbacks of treadmill walking and walking in place display jitter both enhanced visually induced illusions of self-motion to a similar degree (compared to smooth displays).
Introduction
As we move through the world, we detect our self-motion using multiple senses, including vision, the vestibular system of the inner ear, proprioception, somatosensation, and audition (Gibson, 1966; Dichgans and Brandt, 1978; Howard, 1982; Benson, 1990). The visual and vestibular contributions to this self-motion processing are thought to be especially important (Dichgans and Brandt, 1978; Howard, 1982). The key role that vision plays is clearly demonstrated by the fact that compelling illusions of self-motion can be induced by visual self-motion stimulation alone (known as “vection”; Fischer and Kornmüller, 1930; Tschermak, 1931). Notably, while the visual system is able to detect both constant and accelerating self-motions (based on the optical flow presented to the moving observer), the inertial sensors comprising the vestibular end organs only respond to acceleration (Howard, 1982; Benson, 1990). Given this limitation of the vestibular system, it was long considered that: (a) visually simulated smooth/constant self-motions should induce the strongest vection – since such displays would be expected to produce minimal visual-vestibular conflict in stationary observers; and (b) visually simulated self-acceleration would impair, or possibly even destroy, vection – since the vestibular stimulation normally accompanying this simulated self-motion would be absent (Zacharias and Young, 1981).
Contrary to both of these predictions, we now know that vection can be significantly enhanced by adding a variety of visually simulated self-accelerations to smooth, constant self-motion displays (Palmisano et al., 2000, 2003, 2007, 2008; Palmisano and Chan, 2004; Kim and Palmisano, 2008; Bubka and Bonato, 2010; Nakamura, 2010, 2012; Kim et al., 2012; Apthorp and Palmisano, 2014; Kim and Khuu, 2014). Despite the expected increases in sensory conflict, adding visually simulated viewpoint jitter to optic flow has been shown to significantly decrease vection onset latencies, lengthen vection durations and strengthen vection ratings (e.g., Palmisano et al., 2000; see Palmisano et al., 2011 for a review).
To date, one of the most persistent explanations for this “jitter advantage” for vection is based on the observation that smooth optical/retinal flow rarely occurs in the real world. Until recently, vection was typically induced by displays which simulated constant velocity linear/rotary self-motion along/about a single axis. However, walking and running through the world actually generates complex six degree-of-freedom (6DOF) head movements. In addition to an overall forward displacement, such self-motions also generate random and oscillatory “bob,” “sway,” and “lunge” head displacements, as well as 3D head rotations (Grossman et al., 1988; Cutting et al., 1992; Hirasaki et al., 1999; von Grünau et al., 2007). The end result is a rich multi-axis mix of head motion amplitudes and frequencies. Since head jitter in these situations can be as high as 15 Hz, and can also include linear components, the visual perspective jitter generated by such head movements can only be partially compensated for by eye-movements (Grossman et al., 1989; von Grünau et al., 2007). Accordingly, it has been proposed that self-motion perception might be specialized for (i.e., tuned to) the jittering retinal flow accompanying everyday locomotion1. That is, jittering self-motion displays might induce superior vection to smooth, constant velocity self-motion displays because they are more ecological/naturalistic and therefore better matched to visual self-motion processing (e.g., Bubka and Bonato, 2010). In apparent support of this proposal, Bubka and Bonato (2010) reported that movies filmed while walking with a handheld camera induced superior vection to other control movies which were filmed from a rolling cart (the former and latter situations should have produced more naturalistic and minimal/unnatural jitter respectively).
Despite the intuitive appeal of this ‘ecological’ explanation of jitter effects on vection, and Bubka and Bonato’s (2010) observations, it is actually a rather challenging hypothesis to test. Simply comparing the vection enhancements provided by adding ecological and artificially generated display jitter (compared to no jitter or smooth display conditions) does not provide a conclusive test. The artificial jitter stimuli required for this type of study should, by necessity, match the complexity and other characteristics of real head jitter as closely as possible, while still serving as viable control stimuli. However, it is difficult to adjust any one of the multiple display factors thought to make displays more/less ecological, without also changing other, potentially critical, low-level display factors. For example, one cannot create a viable jittering control stimulus by simply scrambling real head position and orientation data, since the resulting frequency spectrum of this artificial jitter would be dramatically different from that of the real head jitter upon which they were based.
In the current study, we attempted to test the ecological account of the jitter advantage by comparing the effects of more and less ecological patterns of simulated viewpoint jitter on vection. In both cases, complex 6DOF visual viewpoint jitter was added to the radially expanding optic flow display which simulated constant velocity forward self-motion. Two types of walking were used to generate this visual jitter: (1) Walking on a motorized treadmill – where participants repeatedly stepped in a forward direction while walking on the moving belt of the treadmill; and (2) Walking in place – where participants repeatedly moved their legs forward and then reversed this leg motion so as to return their feet to their original position on the stationary ground surface. Of these two types of walking, treadmill walking is generally regarded as the more naturalistic (Pozzo et al., 1990; Templeman et al., 1999; Usoh et al., 1999; Yan et al., 2004; Warabi et al., 2005; Ivanenko et al., 2006; Riley et al., 2007; Lee and Hidler, 2008). In fact, treadmill gait appears to be qualitatively and quantitatively similar to overground gait (e.g., Riley et al., 2007), and as a result, head kinematics are similar to those during normal walking (e.g., Pozzo et al., 1990; Lee and Hidler, 2008).
In order to test the ecological account, it was important to first show that the physical characteristics of 6DOF head jitter generated by the “more ecological” treadmill walking and the “less ecological” walking in place conditions differed significantly2. Thus we initially conducted a series of head movement analyses that compared the head jitter frequencies and amplitudes generated by the two types of walking. These analyses examined whether the two types of walking produced: (1) different dominant peak head movement amplitudes and frequencies; (2) different distributions of spectral content; and (3) differences in the balance of the motion energy across the three orthogonal axes of head movement.
This study was however primarily interested in whether: (1) the visual jitter generated by the treadmill walking would induce superior vection to that generated by the walking in place; and (2) both types of jittering radial flow would induce superior vection to the smooth radial flow. During the walking trials, head movements generated by either the treadmill walking or walking in place were directly incorporated into the self-motion display in real time (as changes to both the position and orientation of the virtual camera). These jittering radial flow displays were also later played back to participants when they were standing still. Self-motion perceptions in these subject stationary “playback” trials were crucial as they compared the effects of the two visual jitter types on vection (Note: vection is traditionally defined as a visual illusion of self-motion induced in a physically stationary observer). By contrast, the walking conditions allowed us to compare the effects of the two types of visual-and-head jitter on multisensory (i.e., visual + non-visual) perceptions of self-motion for the first time.
Materials and Methods
This study was approved by the University of Wollongong Human Research Ethics Committee (HE10/120) and written consent was obtained from all participants prior to participating in the study.
Participants
Nineteen Psychology students (15 females and 4 males; mean age = 23.28, SD = 2.87) from the University of Wollongong participated in this experiment. All of these participants had normal or corrected-to-normal vision and no self-reported vestibular or neurological impairments.
Apparatus
The general experimental setup is shown in Figure 1. The computer-generated self-motion displays were generated on a Dell Optiplex GX620 PC and rear-projected onto a flat projection screen (1.48 m wide × 1.20 m high) by a Mitsubishi Electric (Model XD400U) color DLP data projector. The displayed images subtended a visual area of ∼79° wide by ∼67° high and had a 1024 (horizontal) × 768 (vertical) pixel resolution as well as a refresh rate of 72 Hz. Each display (viewed in an otherwise completely dark room) simulated either a 4 or 5 km/h forward self-motion through a 3D cloud of randomly positioned objects. The cloud – the dimensions of which were simulated to be 1.7 m wide by 2.6 m high by 10.8 m deep – consisted of 1000 blue spheres (300 cd/m2) on a black background (0.1 cd/m2). On different trials, these self-motion displays were viewed either while standing still, walking in place, or walking forward on the motorized treadmill (ProForm PF 4.0). Participant head position and orientation were continuously recorded during both types of walking trial, via an ultrasonic Logitech 3-D head tracker (see Ash et al., 2013 for details). For safety reasons, participants wore a ceiling mounted B-Safe body harness throughout the entire experiment (during both walking and standing still blocks).
FIGURE 1. The general experimental setup. Please note that moving 3-D clouds of spherical objects were used as the visual stimuli for this experiment (not the tiled virtual corridor which is depicted). All room lighting was turned off during the actual testing.
Procedure
In this experiment, we examined self-motion percepts generated by viewing radially expanding optic flow in the following three situations: (1) treadmill walking, (2) walking in place, and (3) standing still. On each type of trial, participants were exposed to 30 s of jittering or smooth radially expanding flow. The display jitter (when present) was generated by incorporating the participant’s own tracked head position (horizontal, vertical, and depth) and orientation (yaw, pitch, and roll) changes directly into the self-motion display (Note: end-to-end system lag measured as ∼60 ms – see Ash et al., 2011 for measurement details). This visually simulated viewpoint jitter was created in real time when participants were actually walking, and was also later played back to them when they viewed these self-motion displays while standing still. During smooth display conditions, these head tracking data were simply ignored and purely radial (i.e., non-jittering), expanding flow displays were presented instead3.
During the treadmill walking conditions, the speed of forward self-motion simulated by the radially expanding component of the display matched the speed of the treadmill (Both simulated either 4 or 5 km/h self-motions depending on the trial). By contrast, in the walking in place and the standing still conditions, forward self-motion was only simulated by the display motion (i.e., the treadmill belt was not moving). During the walking conditions (both treadmill walking and walking in place), participants were instructed to try to walk so as to match the speed of their self-motion to that indicated by the radially expanding optic flow display. Two different normal/naturalistic walking speeds were simulated: 4 or 5 km/h. These were deliberately chosen to be close to treadmill belt speeds previously found to enhance optic flow discrimination (Durgin and Gigone, 2007; Durgin, 2009). This rather narrow range of simulated self-motion speeds was also chosen for practical reasons: (1) the acceleration profile of our treadmill was not optimal for slower belt speeds; and (2) our pilot studies indicated that faster belt speeds were too quick for comfortable walking.
Each participant was run through six different blocks of trials. The three different walking types (i.e., treadmill walking, walking in place, and standing still) were examined in separate blocks (with each block being run twice). Four self-motion trials were tested in each block, with the following displays being presented in a fully randomized order: (1) smooth radial flow at 4 km/h, (2) jittering radial flow at 4 km/h, (3) smooth radial flow at 5 km/h, and (4) jittering radial flow at 5 km/h. The order of block presentation was (by necessity) not fully random: One standing still (playback) block was run between two treadmill walking blocks, and the other standing still (playback) block was run between two walking in place blocks4.
On all trials, the participant initially held the treadmill’s handrails until he/she was comfortable walking/standing and then released them (only using these again later if needed for support or if he/she became uncomfortable or disoriented). Directly after the 30-s self-motion display, the participant rated the perceived strength of their experience of self-motion for that trial [via a modified version of Stevens’s (1957) method of magnitude estimation]. Each strength rating could range from 0 to 100 (with a rating of “0” indicating no experience of self-motion). These strength ratings were made relative to a standard reference stimulus shown at the beginning of each block of trials. This standard stimulus was a smooth (i.e., non-jittering) pattern of radially expanding optic flow, which simulated forward self-motion at 4 km/h, and was always viewed for 30 s by participants when they were standing still. Participants were told that the strength of the self-motion they experienced during this standard condition corresponded to a rating of “50.” Note that vection (a visually induced illusion of self-motion in a physically stationary observer) could only be potentially induced during standing still conditions (both walking conditions involved whole body motion, even if there was little or no actual displacement relative to the room). However, self-motion was perceived in both walking and stationary conditions – based on multisensory stimulation in the case of the former and visual-only stimulation in the case of the latter.
Results
Head Movement Analysis
In the introduction we hypothesized that head jitter amplitudes and frequencies would differ significantly in the “more ecological” treadmill walking and “less ecological” walking in place conditions. Throughout each walking trial, we continuously recorded head position (along the x/horizontal, y/vertical, z/depth axes in m) and head orientation (relative to these same three axes in degrees). In order to test the above hypothesis, we chose to focus only on our participants’ head position changes when walking (because the visual consequences of head position changes could not be nulled by eye movements and were more salient than those resulting from head orientation changes). Figure 2 provides examples of the raw horizontal, vertical and depth head position changes made during one treadmill walking trial, and one walking in place trial, for one representative participant (S2).
FIGURE 2. Horizontal, vertical and depth head position over time for one participant (S2). (A) Plot shows the head position changes (in meters) of this participant while walking on a treadmill at 4 km/h while viewing a matched speed self-motion display. (B) Plot shows head position changes for this participant viewing a similar display (also simulating a 4 km/h forward self-motion) while walking in place.
We first performed a power spectrum analysis on this head position data to identify the head movement frequency content during both types of walking condition (i.e., treadmill walking and walking in place). These data were first partially pre-whitened by fitting a first-order autoregressive (AR) model and then filtered with the inverse of this model. This filtering process removed overall broad low-pass characteristics from the data and made the signal whiter, improving the ability to extract the spectral peaks (see Kay, 1988). Then the power spectrum for each trial was estimated using Welch’s (1967) smoothed periodogram and a Hamming window to suppress spectral side lobes. Figure 3 shows results for one randomly selected participant (S2) during treadmill walking (TOP) and walking in place (BOTTOM). To characterize the spectral composition of the signals, we identified the frequencies and amplitudes of the largest four peaks in the spectrum for each subject and trial type. As the largest peak was typically at least 5 to 10 dB above the next largest peak, we chose to only statistically compare the dominant frequencies and amplitudes of these translational head movements along each axis (these values were likely to be more representative of the subject’s actual cadence).
FIGURE 3. Power spectrum analyses (blue/unbroken trace = original; red/dashed trace = whitened) of the head position data along each axis for treadmill walking (A) and walking in place (B). These traces are both also for participant S2 here walking at 4 km/h. Note that partial pre-whitening lowers the absolute magnitude of the first peak.
Head movement amplitude
Dominant peak amplitude (individual axes). Do the two types of walking produce different dominant peak head movement amplitudes (and if so for which of the three axes)? To answer these questions we performed separate repeated-measures ANOVAs on the magnitude of the dominant peak in the head movement spectra estimated for each axis. The independent variables examined in each of these ANOVAs were walking type (treadmill walking or walking in place) and simulated speed (4 or 5 km/h). We found significant main effects of walking type on dominant x-axis (horizontal) and y-axis (vertical) head amplitudes (F1,18 = 4.60, p < 0.046 and F1,18 = 98.79, p < 0.0001 respectively), but not on dominant z-axis (depth) head amplitudes (F1,18 = 1.24, p = 0.281). Walking in place produced larger x-axis, and smaller y-axis, dominant peak amplitudes than treadmill walking (Figure 4). In the case of y-axis dominant peak head movement amplitudes, we also found a significant main effect of simulated speed (F1,18 = 21.51, p < 0.0001) and a significant interaction between simulated speed and walking type (F1,18 = 18.51, p < 0.0001). These findings indicated that y-axis dominant peak head movement amplitudes increased significantly with simulated speed during treadmill walking (but not when walking in place). No other main effects or interactions were found to reach significance.
FIGURE 4. Effects of Walking Type (Treadmill Walking versus Walking in place) on the magnitude of the dominant (pre-whitened) peak head movements along the horizontal (x), vertical (y), and depth (z) axes. This corresponds to the peak spectral density of the motion signal in each case. Error bars represent SEMs.
Distribution of the spectral content (individual axes). The spectral content was distributed quite differently across multiple spectral peaks in the treadmill walking and walking in place conditions. To quantify the spread of power across the peaks, we used the ratio of the magnitude of the second lowest frequency peak to magnitude of the lowest frequency peak. We statistically compared these ratios for head motions along each axis using Bonferroni-corrected contrasts. These ratios were the highest for z-axis head motions, and in fact, were not significantly different for treadmill walking (M = 0.43; SD = 0.16) and walking in place (M = 0.40; SD = 0.19), t18 = 0.69, p> 0.05. The ratios were, however, significantly lower for y-axis head motions during treadmill walking (M = 0.07; SD = 0.05) than during walking in place (M = 0.26; SD = 0.18), t18 = -4.83, p < 0.0003. By contrast, the ratios were significantly higher for x-axis head motions during treadmill walking (M = 0.17; SD = 0.09) than during walking in place (M = 0.06; SD = 0.05), t18 = 5.16, p < 0.0003. These findings suggest that y-axis head motions during treadmill walking (but not during walking in place), and x-axis head motion for walking in place (but not treadmill walking), were each dominated by a single oscillatory motion.
Head motion energy across axes. Head motion energy was observed to be better balanced between horizontal and vertical motion during treadmill walking. Bonferroni-corrected contrasts revealed that: (i) ratios of the dominant x-axis peak motion to the dominant y-axis peak motion were significantly lower for treadmill walking (M = 1.1; SD = 1.0) than for walking in place (M = 19.3; SD = 20.5), t18 = -3.97, p < 0.002; and (ii) ratios of the dominant y-axis peak motion to the dominant z-axis peak motion were significantly higher for treadmill walking (M = 4.9; SD = 3.0) than for walking in place (M = 0.73; SD = 0.64), t18 = 6.26, p < 0.0002. Thus, there was a greater bias toward x-axis over y-axis head motion during walking in place and a greater bias toward y-axis over z-axis head motion during treadmill walking.
Head movement frequency
We also performed separate repeated-measures ANOVAs on the frequencies of the dominant peaks in the head movement spectra estimated for each of the three axes. The independent variables examined were walking type (treadmill walking or walking in place) and simulated speed (4 or 5 km/h). We found significant main effects of walking type on the frequency of the dominant peak head motion along each of these three axes (x-axis F1,18 = 31.83, p < 0.0001; y-axis F1,18 = 6.79, p < 0.02; z-axis F1,18 = 8.444, p < 0.009). Walking in place generated significantly higher frequency dominant peak x-axis and y-axis head movements, and significantly lower frequency dominant peak z-axis head movements, than treadmill walking (see Figure 5). We also found significant interactions between simulated speed and walking type on the dominant peak frequencies of the x-axis (F1,18 = 77.86, p < 0.0001) and y-axis (F1,18 = 8.91, p < 0.008), but not z-axis (F1,18 = 2.11, p < 0.164) head movements (see Figure 5). The frequencies of the dominant spectral peaks for both x- and y-axis head motions increased with simulated speed during treadmill walking (by 18 and 16% respectively), whereas they slightly decreased during walking in place (by 2 and 1%). By contrast, the frequencies of the dominant spectral peaks for z-axis head motions increased with simulated speed for both types of walking (by 25% for treadmill walking and 5% for walking in place).
FIGURE 5. Effects of Walking Type (Treadmill Walking versus Walking in place) and simulated speed (4 versus 5 km/h) on dominant (pre-whitened) peak head movement frequencies (Hz) along the horizontal (x), vertical (y) and depth (z) axes. Error bars represent SEMs.
Self-Motion Perception Data Analysis
Ratings of forward self-motion when standing still (i.e., vection)
The first repeated-measures ANOVA examined the vection strength ratings produced in the display playback conditions (when participants were always standing still). The independent variables examined were display type (smooth, treadmill jitter and walking in place jitter) and simulated speed (4 and 5 km/h). We found a significant main effect of simulated speed (F1,18 = 97.97, p < 0.0001), indicating that displays which simulated 5 km/h self-motions induced stronger vection than those simulating 4 km/h self-motions (see Figure 6). We also found a significant main effect of display type (F2,36 = 29.23, p < 0.0001; see Figure 7). Bonferroni corrected post hoc comparisons showed that: (1) adding either type of jitter (i.e., treadmill walking and walking in place generated jitter) to the radial flow induced significantly stronger vection than the smooth control displays (p < 0.05 in each case); and (2) displays with treadmill walking generated jitter did not produce significantly different vection strength ratings to those with walking in place generated jitter (p > 0.05). The interaction between display type and simulated speed was also not significant (F2,36 = 0.877, p> 0.05).
FIGURE 6. Effect of Simulated Speed (4 or 5 km/h) on the strength of the vection induced when participants were standing still (i.e., Display Playback). Error bars represent SEMs.
FIGURE 7. Effect of Display Type (Smooth flow, Treadmill Jitter, and Walking in place Jitter) on the strength of the vection induced when participants were standing still (i.e., Display Playback). Error bars represent SEMs.
Ratings of forward self-motion when treadmill walking and walking in place
A second repeated-measures ANOVA compared the multisensory perceptions of forward self-motion when participants viewed optic flow displays while treadmill walking and walking in place. The independent variables examined were walking type (treadmill walking and walking in place), display type (smooth and jittering) and simulated speed (4 and 5 km/h). We found no main effect of walking type (F1,18 = 0.04, p> 0.05), which indicated that ratings of the strength of forward self-motion were not significantly different during walking in place and treadmill walking conditions. We also found a significant main effect of simulated speed (F1,18 = 43.31, p < 0.0001), indicating that faster simulated speeds generated stronger ratings of forward self-motion (see Figure 8). Importantly we also found a significant main effect of display type (F1,18 = 25.27, p < 0.0001), which indicated that jittering displays produced stronger self-motion ratings than smooth displays (see Figure 8). The walking type by display type interaction was not significant (F1,18 = 1.84, p> 0.05), suggesting that treadmill-walking generated jitter and walking in place generated jitter produced very similar advantages/enhancements in terms of the rated experience of self-motion. No other two- or three-way interactions were significant (p > 0.05 in all cases).
FIGURE 8. Effects of Walking Type (Treadmill Walking versus Walking in Place), display type (smooth versus jitter) and Simulated Speed (4 or 5 km/h) on the strength of the perceived self-motion in depth. Error bars represent SEMs.
Discussion
This study compared the self-motion perceptions induced by treadmill walking and walking in place generated jitter in order to test the ecological account of the jitter advantage for vection. Both types of visual jitter were generated by whole body observer motions, which resulted in complex 6DOF head movements and jittering optic flow. It was predicted that the “more ecological” treadmill walking and “less ecological” walking in place conditions would generate significantly different head (and therefore visual) jitter amplitudes and frequencies. Consistent with this prediction, our head tracking data displayed multiple differences between the two types of walking. Head motion energy was observed to be better balanced between horizontal and vertical motion during treadmill walking. Compared to walking in place, treadmill walking also produced: (1) significantly smaller horizontal, and larger vertical, dominant peak amplitude head jitter; and (2) significantly lower frequency dominant peak horizontal and vertical head jitter, as well as higher frequency dominant peak z-axis head jitter. We also found that the simulated speed manipulation had different effects on the head jitter generated by treadmill walking and walking in place. While increasing the simulated speed of forward self-motion increased the dominant peak amplitude of vertical head jitter as well as the dominant horizontal and vertical head frequency during treadmill walking, it did not alter these during walking in place5. It is likely that without the added guidance provided by treadmill belt motion, participants found it difficult to adjust the pace of their walking in place to match the noticeable but rather modest differences in visually simulated speed.
Importantly, the present self-motion strength data provided little support for the ecological account of the jitter advantage for vection. While we found marked differences in the head movement amplitudes and frequencies generated by treadmill walking and walking in place, “more ecological” treadmill walking and “less ecological” walking in place generated jitter produced very similar advantages/enhancements (compared to conditions with smooth, purely radial, optic flow). When participants walked while viewing the computer generated self-motion displays: (1) both types of jitter significantly increased the perceived self-motion in depth induced by the radial component of the flow; and (2) there was no difference between perceptions in the treadmill walking and walking in place conditions. When the jittering self-motion displays recorded during the two types of walking were later played back to stationary observers, the vection advantages produced (again compared to smooth flow control conditions) were virtually identical.
So if visually simulated viewpoint jitter does not improve vection by making the flow appear more ecological, what causes these types of jitter advantages? Currently two explanations have strong empirical support. Adding simulated viewpoint jitter to smooth radial flow might improve vection in depth by: (1) increasing (or perhaps altering6) the observer’s overall global retinal motion (Palmisano and Kim, 2009; Palmisano et al., 2012; Nakamura, 2013); and (2) reducing local motion adaptation to the smooth component of the flow (e.g., Seno et al., 2011; Kim and Khuu, 2014). These explanations are not mutually exclusive, and in fact, there appears to be mounting evidence that there may be multiple mechanisms underlying these jitter effects (see Apthorp and Palmisano, 2014). In support of the “increased global motion” explanation, Palmisano et al. (2012) found that the “slalom illusion,” where observers track an oscillating fixation point while viewing smooth radial flow, produced a similar vection advantage to stationary fixation conditions when the display was oscillated. Interestingly, slalom illusion and display oscillation conditions did not produce significantly different motion aftereffects to control conditions with stationary fixation and a smooth non-oscillating display (probably because display durations in all three cases were very short in this study; only 15 s).
In support of the “reduced local motion adaptation” explanation, Kim and Khuu (2014) found reduced motion aftereffects and increased vection in depth for smooth flow when either angular or linear simulated viewpoint oscillation was added. Seno et al. (2011) had earlier found both reduced motion aftereffects and increased vection in depth for linearly jittering and oscillating, compared to smooth, radial flow. Interestingly, they also reported that random jitter, but not oscillation, increased the duration of aftereffects of visual self-motion perception. These vection aftereffects are separate and distinct from general motion aftereffects. Thus, in the Seno et al. (2011) paper, we speculated that these vection aftereffects might reflect the adaptation of a “pure vection mechanism,” over and above lower level motion mechanisms. According to this notion, simulated random viewpoint jitter might have stimulated this pure vection mechanism to a greater extent, while simulated viewpoint oscillation might have tapped into relatively lower-level motion processing.
In the current paper, we utilized complex optic flow patterns containing 6DOF jitter. This walking-generated jitter contained both oscillatory and random components along/about all three axes and thus potentially would have tapped both proposed jitter-enhancement mechanisms (i.e., “increased global motion” and “reduced local motion adaptation” and perhaps others as well).
Conclusion
We have shown that the jitter advantage for visually mediated self-motion perception is very robust. We found a substantial advantage for jittering over smooth self-motion displays, irrespective of whether the participant was walking or standing still when viewing these optic flow displays. Importantly, the advantages found in both scenarios (i.e., when walking or standing still) were remarkably similar for “more ecological” and “less ecological” types of jittering optic flow. Thus, the present data provide little support for the ecological account of these jitter advantages for visually mediated self-motion perception and vection.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by an Australian Research Council Discovery grant (DP1093493) to awarded to Stephen Palmisano and Robert S. Allison.
Footnotes
- ^ One reviewer noted that while walking is arguably the most natural form of self-motion, many passive self-motions (e.g., when in a vehicle) are common in the modern world. These are also accompanied by characteristic patterns of jitter and could thus potentially be deemed ecological as well.
- ^ While vection is a conscious subjective experience, it is widely regarded to be the result of the very rapid (i.e., preconscious) pickup of optic flow based information (e.g., Warren, 1995; Norman, 2002). Thus, the participant’s ability to consciously discriminate between the different walking conditions was not regarded as a critical test here (since preconsciously detected differences might not be consciously available). It was however essential to show that the physical characteristics of the head movements generated by the more and less ecological walking conditions were in fact significantly different each other.
- ^ When smooth radial flow displays were presented during walking (as opposed to standing still), they would have visually jittered (due to the relative motion between the walker’s head and the display). While the rotational components of this jitter might have appeared reasonably natural, the linear components contained no motion perspective. Therefore, the visual jitter in this situation would have been incompatible with the 3D environment being simulated by the self-motion display.
- ^ Prior exposure to optic flow and walking can both alter subsequent vection (e.g., Apthorp and Palmisano, 2014; Seno et al., 2014). Here to control for possible block order effects we averaged the data obtained in the first and last walking blocks together. In order to minimize trial-to-trial carry over effects (i.e., sensory adaptation/readjustment), the minimum inter-trial interval was set at ∼35 s (i.e., longer than the actual duration of the self-motion stimulations).
- ^ This latter finding is intriguing since research shows that optic flow based speed discriminations are enhanced by treadmill walking at very similar speeds (Durgin and Gigone, 2007; Durgin, 2009). It provides further evidence that walking in place was less ecological than treadmill walking (i.e., the conscious control of walking in place may have bypassed the normal/automatic processing associated with the control of natural walking).
- ^ Kim and Khuu (2014) have recently suggested that it may not be the overall increase in retinal motion that is important, “but rather, variation in the direction and velocity of visual motion across the retina” (p. 9) which is critical for the jitter advantage for vection.
References
Apthorp, D., and Palmisano, S. (2014). The role of perceived speed in vection: does perceived speed modulate the jitter and oscillation advantages? PLoS ONE 9:e92260. doi: 10.1371/journal.pone.0092260
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Ash, A., Palmisano, S., Apthorp, D., and Allison, R. S. (2013). Vection in depth during treadmill walking. Perception 42, 562–576. doi: 10.1068/p7449
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Ash, A., Palmisano, S., Govan, G., and Kim, J. (2011). Display lag and gain effects on vection experienced by active observers. Aviat. Space Environ. Med. 82, 763–769. doi: 10.3357/ASEM.3026.2011
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Benson, A. J. (1990). “Sensory functions and limitations of the vestibular system,” in Perception and Control of Self-motion, eds R. Warren and A. H. Wertheim (Hillsdale, NJ: Erlbaum).
Bubka, A., and Bonato, F. (2010). Natural visual-field features enhance vection. Perception 39, 627–635. doi: 10.1068/p6315
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Cutting, J. E., Springer, K., Braren, P. A., and Johnson, S. H. (1992). Wayfinding on foot from information in retinal, not optical, flow. J. Exp. Psychol. Gen. 121, 41–72. doi: 10.1037/0096-3445.121.1.41
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Dichgans, J., and Brandt, T. (1978). “Visual-vestibular interaction: effects on self-motion perception and postural control,” in Handbook of Sensory Physiology, Vol. 8, eds R. Held, H. Leibowitz, and H.-L. Teuber (New York: Springer), 755–804.
Durgin, F. H. (2009). When walking makes perception better. Curr. Dir. Psychol. Sci. 18, 43–47. doi: 10.1111/j.1467-8721.2009.01603.x
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Durgin, F. H., and Gigone, K. (2007). Enhanced optic flow speed discrimination while walking: contextual tuning of visual coding. Perception 36, 1465–1475. doi: 10.1068/p5845
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Fischer, M., and Kornmüller, A. (1930). Optokinetisch ausgelöste bewegungswahrnedhmungen und optokinetischer nystagmus. J. Für Psychol. Neurol. 41, 273–308.
Grossman, G. E., Leigh, R. J., Abel, L. A., Lanska, D. J., and Thurston, S. E. (1988). Frequency and velocity of rotational head perturbations during locomotion. Exp. Brain Res. 70, 470–476. doi: 10.1007/BF00247595
Grossman, G. E., Leigh, R. J., Bruce, E. N., Heuber, W. P., and Lanksa, D. J. (1989). Performance of the human vestibuloocular reflex during locomotion. J. Neurophysiol. 62, 256–272.
Hirasaki, E., Moore, S. T., Raphan, T., and Cohen, B. (1999). Effects of walking velocity on vertical head and body movements during locomotion. Exp. Brain Res. 127, 117–130. doi: 10.1007/s002210050781
Ivanenko, Y. P., Poppele, R. E., and Lacquaniti, F. (2006). Motor control programs and walking. Neuroscientist 12, 339–348. doi: 10.1177/1073858406287987
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Kay, S. M. (1988). Modern Spectral Estimation: Theory and Applications. Englewood Cliffs, NJ: Prentice-Hall.
Kim, J., and Khuu, S. (2014). A new spin on vection in depth. J. Vis. 14, 1–10. doi: 10.1167/14.7.1
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Kim, J., and Palmisano, S. (2008). Effects of active and passive viewpoint jitter on vection in depth, Brain Res. Bull. 77, 335–342. doi: 10.1016/j.brainresbull.2008.09.011
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Kim, J., Palmisano, S., and Bonato, F. (2012). Simulated angular head oscillation enhances vection in depth. Perception 41, 402–414. doi: 10.1068/p6919
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Lee, S. J., and Hidler, J. (2008). Biomechanics of overground vs. treadmill walking in healthy individuals. J. Appl. Physiol. 104, 747–755. doi: 10.1152/japplphysiol.01380.2006
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Nakamura, S. (2010). Additional oscillation can facilitate visually induced self-motion perception: the effects of its coherence and amplitude gradient. Perception 39, 320–329. doi: 10.1068/p6534
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Nakamura, S. (2012). Rotational jitter around the observer’s line of sight can facilitate visually induced perception of forward self-motion (forward vection). Multisens. Res. 26, 553–560. doi: 10.1163/22134808-00002431
Nakamura, S. (2013). Effects of additional visual oscillation on vection under voluntary eye movement conditions – retinal image motion is critical in vection facilitation. Perception 42, 529–536. doi: 10.1068/p7486
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., Allison, R. S., Kim, J., and Bonato, F. (2011). Simulated Viewpoint jitter shakes sensory conflict accounts of self-motion perception. Seeing Perceiving 24, 173–200. doi: 10.1163/187847511X570817
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., Allison, R. S., and Pekin, F. (2008). Accelerating self-motion displays produce more compelling vection in depth. Perception 37, 22–33. doi: 10.1068/p5806
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., Bonato, F., Bubka, A., and Folder, J. (2007). Vertical display oscillation effects on forward vection and simulator sickness. Aviat. Space Environ. Med. 78, 951–956. doi: 10.3357/ASEM.2079.2007
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., Burke, D., and Allison, R. S. (2003). Coherent perspective jitter induces visual illusions of self-motion. Perception 32, 97–110. doi: 10.1068/p3468
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., and Chan, A. Y. C. (2004). Jitter and size effects on vection are robust to experimental instructions and demands. Perception 33, 987–1000. doi: 10.1068/p5242
Palmisano, S., Gillam, B. J., and Blackburn, S. (2000). Global perspective jitter improves vection in central vision. Perception 29, 57–67. doi: 10.1068/p2990
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., and Kim, J. (2009). Effects of gaze on vection from jittering, oscillating and purely radial optic flow. Atten. Percept. Psychophys. 71, 1842–1853. doi: 10.3758/APP.71.8.1842
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Palmisano, S., Kim, J., and Freeman, T. C. A. (2012). Horizontal fixation point oscillation and simulated viewpoint oscillation both increase vection in depth. J. Vis. 12, 1–14. doi: 10.1167/12.12.15
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Pozzo, T., Berthoz, A., and Lefort, L. (1990). Head stabilization during various locomotor tasks in humans. I. Normal subjects. Exp. Brain Res. 82, 97–106. doi: 10.1007/BF00230842
Riley, P. O., Paolini, G., Della Croce, U., Paylo, K. W., and Kerrigan, D. C. (2007). A kinematic and kinetic comparison of overground and treadmill walking in healthy subjects. Gait Posture 26, 17–24. doi: 10.1016/j.gaitpost.2006.07.003
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Seno, T., Palmisano, S., and Ito, H. (2011). Independent modulation of motion and vection aftereffects revealed by using coherent oscillation and random jitter in optic flow. Vision Res. 51, 2499–2508. doi: 10.1016/j.visres.2011.10.007
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Seno, T., Palmisano, S., Riecke, B. E., and Nakamura, S. (2014). Walking without optic flow reduces subsequent vection. Exp. Brain Res. doi: 10.1007/s00221-014-4109-4 [Epub ahead of print].
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Templeman, J. N., Denbrook, P. S., and Sibert, L. E. (1999). Virtual locomotion: walking in place through virtual environments. Presence 8, 598–617. doi: 10.1162/105474699566512
Tschermak, A. (1931). “Optischer raumsinn [Optical sense of space],” in Handbuch der Normalen und Pathologischen Physiologie, eds A. Bethe, G. Bergnann, G. Emden, and A. Ellinger (Leipzig: Springer-Verlag).
Usoh, M., Arthur, K., Whitton, M., Bastos, R., Steed, A., Slater, M.,et al. (1999). “Walking > walking-in-place > flying, in virtual environments,” in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques ACM, New York, 359–364.
von Grünau, M. W., Pilgrim, K., and Zhou, R. (2007). Velocity discrimination thresholds for flowfield motions with moving observers. Vision Res. 47, 2453–2464. doi: 10.1016/j.visres.2007.06.008
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Warabi, T., Kato, M., Kiriyama, K., Yoshida, T., and Kobayashi, N. (2005). Treadmill walking and overground walking of human subjects compared by recording sole-floor reaction force. Neurosci. Res. 53, 343–348. doi: 10.1016/j.neures.2005.08.005
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Warren, W. H. (1995). “Self-motion: visual perception and visual control,” in Handbook of Perception and Cognition: Perception of Space and Motion, 2nd Edn, eds W. Epstein and S. Rogers (San Diego, CA: Academic Press).
Welch, P. D. (1967). The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans. Audio Electroacoust. 15, 70–73. doi: 10.1109/TAU.1967.1161901
Yan, L., Allison, R. S., and Rushton, S. K. (2004). “New simple virtual walking method-walking on the spot,” in Proceedings of the 8th Annual Immersive Projection Technology (IPT) Symposium, Ames, Iowa.
Zacharias, G. L., and Young, L. R. (1981). Influence of combined visual and vestibular cues on human perception and control of horizontal rotation. Exp. Brain Res. 41, 159–171. doi: 10.1007/BF00236605
Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar
Keywords: vection, self-motion, optic flow, ecology, sensory conflict, locomotion, treadmill
Citation: Palmisano S, Allison RS, Ash A, Nakamura S and Apthorp D (2014) Evidence against an ecological explanation of the jitter advantage for vection. Front. Psychol. 5:1297. doi: 10.3389/fpsyg.2014.01297
Received: 15 August 2014; Accepted: 25 October 2014;
Published online: 11 November 2014.
Edited by:
Wataru Teramoto, Muroran Institute of Technology, JapanReviewed by:
Georg Meyer, University of Liverpool, UKJennifer Campos, University Health Network, Canada
Copyright © 2014 Palmisano, Allison, Ash, Nakamura and Apthorp. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Stephen Palmisano, School of Psychology, University of Wollongong, Wollongong, NSW 2522, Australia e-mail: stephenp@uow.edu.au