1 Introduction
We introduce a novel concept called
thermal masking, a newly observed thermal illusion demonstrating apparent masking properties in thermal referral. In thermal referral, when thermal and tactile stimuli are simultaneously applied to nearby locations of the skin, an illusory thermal sensation is also perceived at the location where the tactile stimulation occurs [
20,
21]. We argue that a strong intensity of the tactile signal can affect the redistribution of thermal sensation and dominate the original thermal sensation, exhibiting masking properties. That is, the masking effect will take place so that the original thermal sensation can be sufficiently attenuated and completely masked by the elicited thermal illusion created from the tactile location, perceiving the thermal sensation only at the tactile location.
Thermal masking is a phenomenon that exhibits masking effects in thermal referral through tactile and thermal interactions. Unlike existing sensory masking (i.e., vision [
19], auditory [
22], and tactile [
57]) that shows a stronger stimulus dominating the perception of a weaker stimulus within the same type of sensory stimuli, thermal masking shows that illusory thermal sensation could dominate the original thermal stimulus through thermal and tactile integration. Previous thermal referral studies have suggested that this phenomenon was based on spatial summation and thermal redistribution, resulting in a uniform distribution of temperature sensation across the thermal and tactile locations. When the thermal stimulus is applied, the amount of thermal sensation would be summed and uniformly redistributed to both the thermal and tactile locations, perceiving thermal sensations at both locations [
28]. However, our finding of the thermal masking phenomenon extends this theory, demonstrating several referral states of
weak,
strong, and
masking with non-uniform thermal redistribution.
Although the underlying mechanism needs further investigation, we believe that our finding of thermal masking has significant implications that potentially impact various human-computer thermal interfaces. Leveraging thermal masking could achieve localization in a large-scale thermal display with a minimum number of thermal actuators. That is, we can perceptually turn vibrotactile actuators into thermo-tactile actuators by transferring thermal sensation to the tactile location through the masking effect. Because thermal actuators and devices are generally more expensive and require more power to activate than vibrotactile actuators, adopting a cost-effective masking approach with a minimum number of thermal actuators could simplify the interface design and engineering complications. Thermal motion illusions from one location to another could be designed to provide rich thermal sensation. We can deliver thermal moving experiences throughout the user’s body to create various thermal effects for better thermal experiences in immersive environments such as VR and AR. Such new illusions could bring huge engineering benefits by bridging the gap between human sensory capabilities and device limitations, benefiting designers and engineers to make effective thermal interfaces. Furthermore, our efforts to investigate thermal masking and its property of non-uniform redistribution of thermal sensations can accelerate scientific research in understanding thermal perception.
In this study, we investigate the properties of thermal masking by interplaying thermal and vibrotactile actuators. We placed one thermal actuator and one or more vibrotactile actuators on the user’s arm to explore the effects of thermal masking in several aspects: temperature, distance, and the placement of opposite sides. The first study focuses on the occurrence of masking in different temperature conditions to investigate the temperature effects on thermal masking and thermal referral. The second study explores the effective distance range of thermal masking on the entire arm. The third study investigates its spatial features to explore the design space of thermal masking. The main contributions of this paper are i) new findings of masking effects in which the thermal sensation can be attenuated and completely weakened, exhibiting non-uniform thermal redistribution; ii) promoting the scientific research in thermal perception through new aspects from existing sensory masking and thermal referral; and iii) engineering benefits of employing perceptual approaches of thermal migration and thermal motion for various thermal displays and interfaces.
5 Study 2: Thermal Masking On Distance
This study aims to explore the effective distance range of thermal masking in the entire arm. The occurrence rates and the perceived thermal regions for the four referral states were measured to study the distance effect.
5.1 Participants
Twenty-two participants (mean age 23.7 (SD=3.6), nine females) who did not participate in the first user study participated in this study. We followed the same procedure of recruitment, compensation, and screening as in the previous study.
5.2 Study Design
Actuators Placement. In this study, we varied the location of the vibrotactile actuators with four distance levels. They were placed at 4 cm, 8 cm, 16 cm, and 24 cm apart from the thermal actuator. One thermal actuator was placed at the same location (the middle of the forearm) as in the previous study (see Figure
9). The four tactile actuators and a thermal actuator were worn together to reduce any perceptual bias by the setup. The onset time and duration for both thermal and vibrotactile actuators were the same as in our previous study.
Experimental Conditions. This study involved two independent variables: temperature and distance. Two temperature levels of cool (-5°C) and warm (+3°C) were selected based on the results from the first user study. Four distance levels of 4cm, 8cm, 16cm, and 24cm were determined for short and long distances based on the average human arm length. To explore the ceiling of thermal masking distance, we chose those 16cm and 24cm to reach the upper arm. The intensity and frequency of vibrotactile actuators were the same as in the first study for cool and warm conditions. The same three questions were provided to determine the referral state.
A block consisted of the randomized trials of four distances to avoid any bias. The order of the cool and warm conditions was balanced among participants. Each condition, by the temperature and distance, has five trials. We gathered a total of 880 trials with the combination of 22 participants by two temperatures by four distances by five repetitions.
5.3 Procedure
Participants were asked to wear five straps on their arms. The rest of the procedure was the same as the first user study.
5.4 Results and Discussion
Figure
10 shows the mean rates of
REFERRAL (i.e., a combination of
weak referral,
strong referral, and
masking) at each distance level in cool and warm conditions. It shows a generally higher rate and lower variances with warm conditions than cool conditions over four distance levels. The low occurrence rates of
REFERRAL were observed at the shortest distance (4cm) at both temperatures. The mean occurrence rates of warm conditions were 90% and 85% at 16cm and 24cm, respectively, showing higher rates at long distances. The results indicate that the thermal referral could be reached to the upper arm with a higher chance. The mean thermal detection rate was 99.9% (Q1 response).
Two-way RM ANOVA for the temperature and distance variables was conducted for data analysis. There was only one significant main effect of temperature (F(1, 21) = 20.72, p < 0.001, \(\eta _{p}^{2}\)=0.50). Post-hoc t-tests revealed that REFERRAL at the warm condition led to a higher occurrence rate on the distances of 8cm (p<0.01), 16cm (p<0.01), and 24cm (p<0.001). This result suggests that the warm temperature better generated thermal referral with the vibrotactile stimulus over long distances.
The individual occurrence rates of
weak referral,
strong referral, and
masking are shown in Figure
11. Both
masking and
strong referral accounted for high occurrence rates in warm conditions at all distances. It shows that the strength of the thermal illusion was likely to be stronger than the actual thermal sensation from the thermal actuator when thermal referral occurred.
The three referral states were analyzed using two-way RM ANOVA with the temperature and distance variables. Only masking showed significant main effect on temperature (F(1, 21) = 18.23, p < 0.001, \(\eta _{p}^{2}\)=0.46) and interaction effect (F(3, 63) = 8.99, p < 0.001, \(\eta _{p}^{2}\)=0.30), indicating the strongest effect of masking on temperature. We also note that the interaction effect involves the distance variable. It can be demonstrated with the results of post hoc t-tests as the high occurrence rates of masking at the distances of 16 cm and 24 cm impacting the overall occurrence rate of masking at the warm condition. This suggests that the warm temperature can yield a higher masking rate than the cool temperature as the distance increases. It may be due to the difference in density of the warm and cold receptors, which is similar to the results from Study 1.
The thermal distribution of the perceived regions on the two temperatures and four distances are shown in Figure
12 and Figure
13. Three types of thermal referral showed similar trends compared to Study 1: a larger and denser region from the stronger thermal illusions. In addition, we observed that the area of illusory thermal sensation is generally larger than the contact area of the tactile actuator over all distance levels. Specifically, in warm conditions, the strong thermal illusions with high occurrence rate formed more dispersed thermal sensations regardless of the distance. At the shortest distance (4 cm),
weak referral and
strong referral were likely to form a unified thermal region at both locations. This demonstrates that the more dispersed single sensation was perceived rather than two separated thermal sensations due to the close distance. This might be due to poor tactile spatial resolution for close distances and may result in a low occurrence rate of thermal referral at 4 cm. At the distance of 16 cm, the thermal sensation was likely to be redistributed around the elbow joint. This may indicate that the thermal referral can be generated by the vibration conveyed to joints through the skin. It is also interesting to see clear
masking at 24 cm, indicating thermal masking is feasible for larger distances. One possible interpretation is that the afferent nerve signals from two thermal and tactile locations far apart can be transmitted to the brain cortex within the acceptable time for sensory masking [
38,
46].
6 Study 3: Thermal Masking On the Opposite Skin Side
This study aims to explore the interaction space of masking effects by placing the tactile actuators on the opposite side of the thermal actuator. We measured the referral states on both the dorsal and ventral sides of the forearm to observe the thermal redistribution over the arm in two different temperature conditions.
6.1 Participants
Eighteen participants (mean age 23.4 (SD=3.4), nine females) who did not participate in the previous two studies completed this study. We followed the same procedure of recruitment, compensation, and screening as in the previous two studies.
6.2 Study Design
Actuators Placement. Three tactile actuators were placed on the dorsal side of the forearm, while one thermal actuator was placed at the same location as in our previous two studies (i.e., the middle of the forearm on the ventral side of the forearm). The locations of tactile actuators were -8cm (8cm toward the wrist), 8cm, and 16cm (8cm and 16cm toward the upper arm from the thermal actuator) (see Figure
14). Participants wore the three tactile actuators along with the thermal actuator at the same time to minimize the perceptual bias. The methods of presenting stimuli were the same as in our previous studies.
Experimental Conditions. We explored the thermal masking on the effects of temperature and placement on the opposite side of the forearm. We chose the distance of -8cm as it is a popular place for the wristwatch. We kept 8cm and 16cm distances to maintain consistency with the distance condition in Study 2. The temperature conditions were cool and warm, as in the second user study, and we used the same intensities and frequencies as in the previous studies. The thermal and tactile stimuli were delivered in the same way as in the previous studies. All trials were randomized, and the order of cool and warm conditions was balanced. Each condition by temperature and distance has seven repetitions. In total, we retained a total of 756 trials throughout the study, with the combination of 18 participants by two temperature levels and three distance levels by seven repetitions. The same three questions were provided to determine the referral state.
6.3 Procedure
Four straps were placed on the participants’ arms. Besides this, we used the same procedures as in the previous studies.
6.4 Results and Discussion
As shown in Figure
15, the mean occurrence rates of
REFERRAL at -8cm, 8cm, and 16cm with the warm conditions were relatively higher than in cool conditions, indicating that the thermal referral is likely to occur at the opposite side of the forearm with a higher rate with warm temperatures. A two-way RM ANOVA was conducted for statistical analysis. There were two significant main effects for both temperature (
F(1, 17) = 34.49,
p < 0.001,
\(\eta _{p}^{2}\)=0.67) and distance (
F(2, 34) = 4.70,
p < 0.05,
\(\eta _{p}^{2}\)=0.22) and interaction effect (
F(2, 34) = 6.35,
p < 0.01,
\(\eta _{p}^{2}\)=0.27). Post-hoc t-tests for the distance explicitly showed that the distance of -8cm had a significantly higher occurrence rate than the distance of 8cm (
p < 0.05). It suggests that the wrist is suitable for generating thermal referral on the dorsal side of the forearm. The interaction effect can be interpreted with the results of post-hoc t-tests as thermal referral is presented with a higher rate in warm conditions at all distances of -8cm (
p < 0.01), 8cm (
p < 0.001), and 16cm (
p < 0.001). The mean thermal detection rate was 98.1% (Q1 response).
Figure
16 shows the individual referral state. The results show the majority of
no referral and
weak referral at all distance levels in cool conditions, indicating poor referral performance at the opposite side of the arm. In warm conditions, we observed dominant
strong referral and
masking.
We statistically analyzed the individual occurrence rate of weak referral, strong referral, and masking with two-way RM ANOVA. We found that strong referral shows a significant main effect on the temperature(F(1, 17) = 9.82, p < 0.01, \(\eta _{p}^{2}\)=0.37) by the higher rates in warm condition. We also found that masking had a significant main effect of temperature (F(1, 17) = 29.68, p < 0.001, \(\eta _{p}^{2}\)=0.64) and interaction effect (F(2, 34) = 4.58, p < 0.05, \(\eta _{p}^{2}\)=0.21). The interaction effect can be interpreted as the higher occurrence rates at all distances of -8cm (p < 0.01), 8cm (p < 0.05), and 16cm (p < 0.001) on the warm temperature by post-hoc t-tests. The results suggest that the forearm’s opposite side (dorsal side) tended to have a similar trend in thermal referral and masking to the ventral side of the forearm: the higher occurrence rates on all distances with the warm temperatures.
Figure
17 and
18 shows the thermal distribution on the dorsal side of the forearm. The thermal referral was formed in the dispersed thermal location around the location of the vibrotactile stimulus on the dorsal side and the ventral side of the forearm. The thermal illusion also occurred on the opposite side of the target area on the ventral side of the forearm. This phenomenon was observed particularly on the wrist, the thinnest part of the forearm. It may indicate that the thermal illusions can be spread to both sides of the forearm, and the thickness of the arm can affect the thermal referral on the opposite side. Furthermore, it was noted that positioning actuators on the opposite side of the arm resulted in a more dispersed sensation, in contrast to the findings from Study 2, where actuators were on the same side. The afferent nerves on two sides of the forearm may convey the sensations to the brain cortex with a different modulation process, yielding poor sensory masking [
6]. It may even weaken the masking occurrence in cool conditions with a higher density of cold receptors.
7 Discussions
We demonstrated apparent thermal masking phenomenon and non-uniform thermal redistribution in thermal referral. Previous studies have claimed that thermal referral relies on spatial summation and uniform redistribution of thermal sensations between the initial and referred sites. We showed that original thermal sensation can be sufficiently attenuated and dominated by illusory thermal sensation.
Sensory masking has been extensively investigated in auditory, tactile, and visual stimuli, but the discovery of the masking phenomenon in thermal is new. It’s a novel phenomenon that extends the concept of thermal referral in that the original thermal sensation can be completely attenuated so that thermal sensation can be only felt at remote locations. To the best of our knowledge, there exists no known thermal masking theory that one thermal stimulus dominates another thermal stimulus in two different sites. That is, there are no "true" masking effects in thermal cues in that we can perceive two distinctive thermal stimuli when applied simultaneously in different locations, regardless of whether they are the same or different temperatures. Our discovery of thermal masking that demonstrates illusory thermal sensation dominating the existence of thermal sensation will present a new aspect of masking and can help advance scientific research in the understanding of thermal perception.
Another aspect of our contribution is engineering benefits. Our finding of non-uniform thermal redistribution in thermal referral contradicts the existing theory, and this could potentially impact various thermal user interfaces and displays. With non-uniform thermal redistribution properties in thermal referral, we can design effective thermal interfaces using a combination of thermal and vibrotactile actuators. This can be advantageous when developing a large-scale thermal display, as it allows large coverage areas with relatively few thermal actuators, such as gloves, vests, suits, sleeves, and shoes. Additionally, this approach can be cost-effective as thermal actuators are relatively more expensive and require more power. Eventually, adopting thermal masking could simplify the engineering complexity and enhance the efficiency of designing various thermal interfaces. Furthermore, a complete migration of thermal sensation can lead to creating immersive interaction experiences by designing thermal motion illusions that move from one point to another, delivering various thermal motion effects on the human body. It can be applied to various engineering hardware and human-computer interfaces like gloves, vests, and even hand-held controllers that can be integrated with VR and AR to deliver dynamic thermal patterns and animation for immersive user experiences. We believe that our findings can help overcome current hardware limitations of providing thermal feedback, benefitting designers and engineers in the HCI community.
We show several important takeaways from this study. We clearly observed that thermal masking can impact a wide effective range and space. This phenomenon could reach long distances (i.e., 24 cm) and the other side of the arm, demonstrating the feasibility of 3D thermal cues that could cover the entire arm. The wrist area (-8 cm) showed a high masking occurrence rate, implying optimal actuator placement location for wearable applications. Another interesting finding was the perceived thermal dispersion. We observed that the perceived thermal sensation in thermal masking (and other referral states) was considerably larger than the actual size of vibrotactile actuators. We also observed that in many cases, this perceived thermal illusion was even larger than the original sensations in other referral states. This is interesting as the size of vibrotactile actuators (diameter of 10 mm) is much smaller than that of thermal actuators (30 mm (width) × 40 mm (height)). Leveraging this property can expand the thermal interaction space around the location of actuators to cover a relatively larger area than the physical size of actuators.
Another interesting finding is that warm stimuli showed a relatively higher probability of thermal masking than cool stimuli. It could be due to the sparse distribution of warm receptors that resulted in a lower magnitude of thermal stimuli, which raises the masking occurrence rate. It exhibited a comparable observation in the previous thermal referral studies [
20,
28]. While the warm conditions outperformed the cool and cold conditions in thermal masking, the occurrence rate of other referral states is in the acceptable range for implementing large-scale displays with cool and cold sensations. In addition, we also noticed that the temperature should not be too far from the skin temperature to increase the masking probability. Earlier studies indicated that higher intensity of thermal feedback might reduce referral effects [
28]. We also observed similar properties in thermal masking in that a higher rate of masking was observed under warm conditions compared to hot conditions.
The underlying mechanism of thermal masking is still unknown. One hypothesis is the diversion of attention while experiencing simultaneous cross-modal stimuli. Masking is observed when both signals are presented simultaneously or in quick succession. The dominant signal interferes with the processing of the target signal, diverting processing resources away from it [
16,
18]. It is known that tactile and thermal stimuli are processed in different regions of the cerebral cortex. Thermal signals activate the insular cortex, while tactile signals activate the somatosensory cortex. Shubert et al. [
46] found that in a masking scenario, the weaker stimulus signal activated the primary somatosensory cortex even if it was masked. This is interesting as it could mean that both the strong and weak signals are processed in the brain, but only one of them is perceived. They conclude that masking results from a shift in attentional focus and conscious perception induced by activity in the reticular formation [
6]. We hypothesize that the change in attentional focus and conscious perception is caused by the sudden appearance of the new and unexpected phantom stimulus, which functions as an
interrupter for the ongoing processing of the initial stimulus. This interruption may lead to the masking of the original thermal cue.
An alternative hypothesis can be that the masking could potentially take place in the latter phases of signal transmission to the brain, perhaps as signals travel from the thalamus to the cerebral cortex. Meador et al. [
40] demonstrated asymmetric thresholds exist between the left and right hand for somatosensory stimuli. They hypothesized that masking occurs during the interaction within the thalamocortical pathway. This finding can provide another hypothesis for the thermal masking process. Previous studies have shown that masking is more probable when the target and the masked signals share a channel while getting transmitted to the cortical region [
40,
46,
51]. Thermal and tactile signals are received and processed differently by our neural system. Tactile stimuli are received by various mechanoreceptors present in the skin. Typically, the detection of tactile cues relies on one of four distinct mechanoreceptors (i.e., Pacinian corpuscle, Mesissner’s corpuscle, Merkel’s disk, and Ruffini ending). Thermal stimuli are detected by thermoreceptors located in the skin, with warm stimuli activating warm receptors and innocuous cold stimuli activating cold receptors. These receptors convert physical stimuli into electrical signals that are then transmitted along peripheral nerves to the spinal cord [
13,
38]. The signals are then transferred to the thalamus, which serves as a gateway to the cerebral cortex and redirects the signal to the specialized regions of the cortex. Since the thalamus is the common gateway for both signals, it is possible that masking occurs during the thalamus-to-cortex transition.
We plan to expand our work to investigate factors including the vibrotactile cue size, intensity, frequency, and moving cue. The exploration of vibrotactile cue size helps reveal the thermal redistribution between two stimuli. It’s known that tactile masking requires the masker signal with a specific signal-to-noise rate [
18,
51]. In previous studies, the vibrotactile frequency impacts tactile masking [
15,
51]. The study on intensity and frequency could show the strength range of thermal masking and unveil the relationship between those two masking mechanisms. We also plan to explore the moving cue to examine the thermal masking occurrence on moving body parts. Finally, the time duration that it takes to perceive the thermal illusion was not captured clearly. We believe exploring this factor could help reveal the occurrence process of thermal masking and its lasting time.