Introduction

When considering complex instructional texts, it is difficult to distinguish between learning-relevant and learning-irrelevant information. Learners who do not receive instructional help can easily be overwhelmed when trying to identify core information which is necessary for understanding the learning content (e.g., Anderson and Armbruster 1984; Mayer 2005). The signaling principle is a prominent design recommendation for supporting learners. Relevant information should be cued, in order to guide the learner’s attention to the most important parts of the text (van Gog 2014). This instructional support is especially necessary when the learning environment has a confusing or detrimental design (Sweller 2010). How the extraneous cognitive load (processing learning-irrelevant information due to suboptimal instructional design) induced through an instructional text influences the effects of signaling is not yet known. Therefore, the current research aims to investigate the signaling effect in interdependence on the legibility of the text, considering the disfluency effect. The aim is to clarify whether signaling is especially beneficial in instructional texts which induce a high cognitive load and are perceived to be very difficult.

Literature review

The signaling effect

The signaling or cueing principle refers to the finding that learning from instructional materials is enhanced when relevant elements or the organization of the material, are highlighted (for reviews, see Schneider et al. 2018; Richter et al. 2016; van Gog 2014). According to van Gog (2014), it is possible to distinguish two signaling modes: signaling within texts and signaling within pictures (including diagrams and animations). The current investigation focused on textual signaling, which can be divided into five types: organizational signals (e.g., headings or summaries), colors (e.g., font colors), text–picture references (e.g., “see the picture”), intonation (e.g., in auditory texts) or a mixture of types (e.g., coloring and text–picture referencing) (van Gog 2014).

The signaling principle can be explained considering the cognitive load theory (CLT; e.g., Paas and Sweller 2014; Sweller 1988). The CLT postulates that the total cognitive capacity is limited. According to recent approaches (Kalyuga and Singh 2016; Kalyuga and Plass 2017), two categories of cognitive load, concerned with the acquisition, storage and retrieval of information, can be distinguished. The first major type is the intrinsic (productive) cognitive load, which is the unavoidable load necessary for accomplishing a specific goal (Kalyuga and Plass 2017). The load arises from the cognitive processing of learning-relevant information and depends on the learner’s prior knowledge (Sweller et al. 2011). Germane processes, or the germane load, which are relevant for schema construction are subsumed under the productive load construct. The second major type is the extraneous (unproductive) cognitive load, which is not directly related to the achievement of a specific learning goal (Kalyuga and Plass 2017). The load arises from cognitive processing of learning-irrelevant information due to suboptimal instructional design (Mayer and Moreno 2010). With respect to CLT, signaling supports the learner by helping him to recognize which information in new instructional material is learning-relevant and which might be irrelevant for the learning goal. Signaling directs the learner’s attention to the most important, learning-relevant material. Especially in long and homogeneous texts, the selection of relevant information must be coordinated by using attention-guiding features within the instructional texts, to prevent cognitive overload. Eye-tracking studies pointed out that visual cues can direct the eye movements of learners to the most important information and limit the amount of time spent fixating the less relevant areas (e.g., Jamet 2014). Therefore, the signaling principle is also referred to as the attention-guiding principle. In consequence, signaling reduces unproductive extraneous load and increases productive intrinsic load (e.g., Amadieu et al. 2011).

In accordance with the theories described, studies have shown that signaling can enhance retention (e.g., Boucheix et al. 2013), and transfer performance during problem-based tasks (e.g., Liu et al. 2013). In addition, signaling also affects motivational and affective states (Schneider et al. 2018). Studies have shown that signaling decreases the learners’ stress levels (e.g., Skuballa et al. 2012), increases their enjoyment of the learning material (e.g., Johnson et al. 2015), increases the material’s attractiveness (e.g., Huk et al. 2003) and enhances their intrinsic motivation (Lin 2011).

Moderators of the signaling effect

Recent meta-analyses regarding the signaling effect have postulated moderator effects in terms of the signaling mode (text vs. graphic), types of signaling (color vs. labeling vs. pointing gestures, etc.), instructional domain (natural sciences vs. math vs. history, etc.), the level of prior knowledge (high vs. low) and the pacing (self-paced vs. system-paced) (Richter et al. 2016; Schneider et al. 2018). For example, Schneider and colleagues (2018) argued that learners with prior knowledge are better able to cope with the additional information provided by signaling than those with low prior knowledge, and thus, signaling is especially beneficial for high-prior-knowledge learners. Nevertheless, low-prior-knowledge learners benefit from the attention-guiding cues in learning environments as well.

However, so far, less attention has been paid to the influence of cognitive load induced through the general design of the learning environment, on the effects of signaling. The concept of element interactivity is the focal point here. An element is any information that must be processed (Sweller 1994). In material with low element interactivity, every element can be processed with no or minimal reference to other elements (for example: the acquisition of a language’s vocabulary). High-element-interactivity material includes elements that interact strongly with each other (for example: the acquisition of a language’s grammar); the elements must be processed simultaneously for the learning content to be fully understood. The higher the number of elements interacts with each other, the higher the cognitive load on the working memory (Tindall-Ford et al. 1997). According to Sweller (2010), “element interactivity is the major source of working memory load underlying extraneous as well as intrinsic cognitive load” (p. 125). Design features in learning environments can be viewed as additional elements, which interact with the actual content and inevitably have to be processed (Beckmann 2010). Whether these features can be viewed as intrinsic load or extraneous load depends on the learning goal (Schnotz and Kürschner 2007). If an instructional text is written in an illegible font which makes it hard to process the information, the font can be viewed as an extraneous load. However, if the learning scenario’s goal is to learn this particular font, the load can be viewed as intrinsic. In both scenarios, the font contributes to the element interactivity because it is an additional element, which must be processed in interaction with the actual content (Sweller et al. 2019). Based on this definition, the element interactivity effect postulates that instructional support through signaling should be especially beneficial in learning materials which induce a high extraneous cognitive load. This support is not necessary in low-element-interactivity materials since learners still have resources available for dealing with the material’s suboptimal design (Chen et al. 2015, 2017).

Induction of extraneous load

For the current research, extraneous load is induced through the legibility or perceptual disfluency of the text. Perceptual disfluency can be induced in instructional text, for example by illegible fonts (e.g., Besken and Mulligan 2013; Carpenter et al. 2013; Katzir et al. 2013; Pieger et al. 2016). According to the CLT, harder-to-read texts induce extraneous cognitive load, since additional cognitive resources are needed to decipher the illegible way in which the information is presented. This was supported by a recent study by Seufert et al. (2017), which indicated that strongly disfluent texts led to a greatly reduced legibility and increased extraneous cognitive load. According to Sweller’s definition of element interactivity (2010), the disfluent font can be viewed as an extraneous source of element interactivity, since the font’s acquisition is not the goal of the learning task.

It is especially important that disfluency must be particularly strong if it is supposed to induce a high cognitive load (Seufert et al. 2017). A low or moderate degree of disfluency can have the exact opposite effect (e.g., French et al. 2013; Weltman and Eakin 2014). In this context, disfluency can be viewed as a desirable difficulty (Bjork 1994). Difficult learning conditions can foster learning processes because the challenge posed by a disfluent learning environment stimulates cognitive engagement and fosters deeper cognitive processing and more elaborate strategies (e.g., Alter et al. 2007; Bjork 1994). Thus, metacognitive activities should be taken into account and measured, in order to determine whether extraneous load induction could act as a trigger for beneficial activities. In this case, extraneous load induction would not generally contribute to the element interactivity defined by Sweller (2010); extraneous elements trigger activities that might compensate for the cognitive processing difficulties induced by the design of the learning environment. Three main concepts are measured when discussing metacognitive activities in learning contexts (Nelson and Narens 1990): Ease of learning judgments is made before learning and affect the allocation of study time (Son and Kornell 2008). Ease of learning judgments is particularly affected by perceptual fluency, since no information about the complexity of the learning content is available at the time the judgment is made. Judgments of learning are made after learning from the text and predict future memory performance (Dunlosky and Metcalfe 2009). Finally, retrospective confidence assesses the confidence of the performance in a learning test (Dinsmore and Parkinson 2013).

The present experiment

The current experiment aimed to investigate the effect of signaling in dependence on the extraneous cognitive load which was induced through the text font used in the learning environment. To do this, instructional texts were manipulated in terms of signaling (color cues vs. no color cues) and induced extraneous load (fluent text font vs. disfluent text font). According to the reviewed literature, signaling should lead to improved learning outcomes (van Gog 2014). Color cues can be viewed as attention-guiding features, which help learners to select important information, thus fostering generative processing (Mayer 2014).

H1

Learners who receive an instructional text with color cues achieve higher learning scores than learners who receive an instructional text without color cues.

As well as the validation of the main effect of signaling (Schneider et al. 2018), the induced extraneous load’s moderating effect is especially interesting and relevant. With respect to the element interactivity effect, signaling is supposed to be especially beneficial in the condition with an illegible font and thus a high element interactivity through extraneous elements (Sweller 2010). Therefore, signaling might compensate for the negative effects of a high extraneous load. In contrast, if the extraneous load is low, no additional instructional support is needed to process the relevant information successfully. Thus, signaling has no influence on learning. On the contrary, extraneous load induction might trigger a more analytic and careful processing (e.g., Alter et al. 2007, Song and Schwarz 2008). Even if the extraneous load induction was especially strong, metacognitive benefits from the disfluency effect cannot be ruled out. Thus, two hypotheses were formulated:

H2a [based on the concept of element interactivity]

Learners who receive an instructional text with an illegible font achieve higher learning scores when receiving additional color cues than learners who receive an instructional text with an illegible font and no color cues.

H2b [based on the concept of desirable difficulty]

Learners who receive an instructional text with an illegible font achieve higher learning scores when receiving no additional color cues than learners who receive an instructional text with an illegible font and color cues.

Several process variables were investigated to obtain deeper insights into learning with instructional texts. Cognitive load was assessed to validate whether manipulating the illegible font succeeded in inducing a high extraneous load. In line with the disfluency effect, metacognitive variables were assessed to investigate whether the extraneous load manipulation had metacognitive benefits. Furthermore, subjective ratings for learning time and navigation behavior were measured to gain additional insights into the learning process, but since these variables were not in the main focus of the study, the procedures and results are displayed in “Appendix 2”).

Methods

Participants and design

Since recent studies and meta-analyses regarding signaling have reported at least medium effect sizes (e.g., Schneider et al. 2018), the estimated required sample size of this experiment is based on a medium effect. According to an a priori power analysis (f = .25; α = .05; 1 − β = 0.80; 2 × 2 design), at least 128 participants should be recruited for this study. Therefore, 143 university students from the Chemnitz University of Technology participated in the current experiment. Five participants had to be excluded due to technical and language problems. The remaining 138 students (85.5% female; age: M = 22.86; SD = 3.93) were included in the statistical analyses. The majority (71.7%) of the students had high school degrees and the remaining participants had academic degrees. Each participant received either 5€ or a 1-h course credit. No significant differences existed between the four treatment groups in terms of age (p = .19), gender (p = .70), educational level (p = .38) or prior knowledge (p = .55). The participants’ domain-specific prior knowledge was low to medium (mean percentage of correct answers in the prior knowledge test: M = 0.31, SD = 0.13). Each student was randomly assigned to one cell of a two-by-two factorial between-subjects design, by drawing lots (signaling: color cues versus no color cues and Extraneous load induction: illegible font versus legible font). Thirty-three students participated in the legible font and non-signaled condition, 34 students participated in the legible font and signaled condition, 35 students participated in the illegible font non-signaled condition and 36 students were assigned in the illegible font and signaled condition.

Materials

The learning material consisted of an instructional text the content of which dealt with the emergence and characteristics of tsunamis, and protection against tsunamis. The text had 1340 words and was divided into six segments, which were presented on different Web sites. On average, 223.33 words were presented per segment. The participants could click on the forward or backward buttons in order to navigate through the Web sites. They could navigate and re-read the segments as often as they wanted and there was a finish button on the last page. Once this button had been clicked, the Web sites could no longer be accessed. An additional graphic was used on the last page to illustrate the emergence of tsunamis. The participants decided how long they wanted to learn themselves, but were told that they would be automatically redirected to the experiment’s next steps after a maximum of 20 min.

Signaling

Color cues were used to implement signaling in the texts. This operationalization was chosen because recent meta-analyses indicated that signaling could be successfully implemented in texts and that visual cues, like colors, were effective for encouraging learning (Richter et al. 2016; Schneider et al. 2018). The main concepts in the texts were marked in red (Hex Color Code: #ff6347), but the color of the text itself was not changed. The color was based on the color of classical text markers in order to strengthen external validity. Furthermore, the color was fairly bright to prevent any additional disfluency from the bad readability of the words. In total, 315 words were signaled (52.20 words per segment). This equated to 23.51% of the instructional text (the most important hard facts and key concepts).

Extraneous load induction

A pretest was conducted in order to determine how to induce a high extraneous load. Since extraneous loads can be achieved by using illegible fonts (e.g., Miele and Molden 2010; Rummer et al. 2016), different fonts were pretested by 49 participants in order to find a particularly hard-to-read font that induced an especially high disfluency. In the online test, the holo-alphabetic sentence (pangram): “Franz jagt im komplett verwahrlosten Taxi quer durch Bayern. (Franz hunts in a completely dilapidated taxi across Bavaria.)” was presented in 22 fonts; participants had to rate the items: readability (scale 1 = very bad; 7 = very good); favor (scale 1 = very bad; 7 = very good) and suitability for learning (scale 1 = not suitable; 7 = very suitable). Since each participant rated all the fonts, dependent measures analyses of variance (ANOVAs) were used for calculation. There was a significant effect for readability with a very large effect size, F (1, 21) = 71.56; p < .001; η 2p  = .60. The “Arial” font had the best readability (M = 6.25; SD = 0.81) and “Mistral” the worst (M = 2.02; SD = 0.98). For favor, there was a significant effect with a very large effect size, F (1, 21) = 37.80; p < .001; η 2p  = .45. Again, the “Arial” font had the highest score (M = 5.58; SD = 1.09) and “Mistral” the lowest (M = 2.46; SD = 1.57). There was a significant effect for suitability with a very large effect size, F (1, 21) = 144.63; p < .001; η 2p  = .76). Again, the “Arial” font had the highest score (M = 5.80; SD = 0.73) and “Mistral” the lowest (M = 1.88; SD = 0.76). On the basis of these results, and since “Mistral” had already been used by Pieger and colleagues (2016), the Arial and Mistral fonts were used to implement good or bad text legibility in the current experiment. The “Times New Roman” font was used for the following questionnaires and learning tests because it scored highly for the three pretested characteristics readability (M = 5.58; SD = 1.37), evaluation (M = 4.67; SD = 1.28) and suitability (M = 4.88; SD = 1.05). This approach was chosen so that the font used for the test did not work as a memory cue. The instructional text is illustrated in Fig. 1.

Fig. 1
figure 1

Screen example of the experimental manipulation

Full size image

Measures

Revelle’s coefficient ω (e.g., McNeish 2018; Revelle and Zinbarg 2009) was chosen to calculate reliability estimates for all measures. The interpretation of the level of reliability is identical to that of Cronbach’s α. In the current manuscript, the focus is on the learning outcomes and the main explanatory variables (cognitive load and metacognition). Further assessed variables (navigation, learning and testing time, frustration and effort) are outlined in “Appendix 2.”

Cognitive load

Klepsch, Schmitz and Seufert’s cognitive load questionnaire (2017) was chosen because it refers to complexity of the content and the recognition of important information. Two items measured the intrinsic load (e.g., “This task was very complex.”) and three items measured the extraneous load (ω = .78; e.g., “It was exhausting to find the important information in this task.”). The participants had to rate the items on a 7-point Likert scale, ranging from 1 (absolutely wrong) to 7 (absolutely correct).

Metacognition and metacognitive accuracy

The procedure for assessing metacognitive judgments and metacognitive accuracy is based on Pieger and colleagues (2016). Ease of learning (EOL) was measured by the question, “How easy or difficult will it be to learn the text?” on a scale from 1 (very easy) to 101 (very difficult). Judgments of learning (JOL) were measured using the question, “What percentage of the questions about the text will you answer correctly?” on a scale from 1 (very easy) to 101 (very difficult). The JOL question was implemented for each text segment. Retrospective confidence (RC) was measured by the question, “How confident are you that your answer is correct?” on a scale from 1 (unconfident) to 101 (confident). The RC questions were implemented after every retention and transfer question.

Metacognitive accuracy was calculated as absolute accuracy and relative accuracy. The three metacognition scores (EOL, JOL and RC ratings) and the overall learning score were first z-standardized. For the absolute accuracy calculation, the performance score was subtracted from the three metacognition scores. For the relative accuracy calculation, the within-person Goodman–Kruskal gamma correlations between the metacognitive judgments and the retention and transfer test item scores were calculated.

Learning and prior knowledge

Prior knowledge was measured by one item, “Please describe how tsunamis emerge.” using an open-answer format. The item was evaluated by two independent raters. The inter-rater reliability (i.e., the intra-class-correlation coefficient (ICC) of aggregated prior knowledge score) was high, ICC (2, k) = 995; F(137, 137) = 189.33; p < .001. A maximum of four points could be achieved. Three further items (ω = .77; “Tsunamis were taught in school.”; “I have dealt with the topic of tsunamis in my spare time.”; “I have dealt with the emergence of tsunamis before.”) were implemented and rated on a scale ranging from 1 (not at all) to 7 (very detailed), in order to assess the participants’ prior experience of the topic.

Learning was measured using two scales: retention and transfer. According to Mayer (2014), retention can be defined as “remembering” content which has been explicitly presented in an instructional text. Transfer knowledge is defined as “understanding.” The learners had to solve novel problems which were not explicitly presented in the instructional text by using the acquired knowledge (Mayer 2014). Retention (ω = .65) was measured with a 10-item multiple-choice questionnaire. The students were asked to choose from six possible answers; each question could have one to six correct answers. They received points for selecting the correct answers and for not selecting the wrong ones and were therefore able to score up to six points per question. The questions referred to information that had been explicitly presented in the text, such as “Which statements about the causes of tsunamis are correct?” An additional open question (“Name three requirements for tsunamis.”) was also implemented to measure retention. The inter-rater reliability was perfect. The same question format was used to obtain the transfer scores. A 5-item scale was created in which every item presented a new scenario and the items had three to seven possible answers. Two open question were also used (e.g., “Calculate the speed of a tsunami with the formula you learned in the text.”). The inter rater reliability was perfect. Every item dealt with a different subtopic from the learning material, in order to assess transfer performance to greatest possible extent. In consequence, the reliability of the scale cannot be reported because the items assessed nearly independent subdomains of the overall knowledge; a detailed applicable knowledge in one domain does not necessarily mean that the participants could apply their knowledge correctly to other items.

Procedure

A computer laboratory at the university with ten identical computers was prepared before each experimental session. 24-inch monitors were provided for this setup, and the online questionnaire was opened at each workstation. The online learning environments could be accessed on the SoSci Survey (Leiner 2016) by entering a particular link. Up to ten participants were tested simultaneously. Sight-blocking partition walls were used to ensure that the students worked independently. At the beginning of the experiment, the participants were told that the experiment was an instructional study on a science topic and were asked to answer a previous knowledge test. Then, they were given the link and asked to take a preliminary look at the learning environment and the learning text. After 2 s, the participants were automatically redirected to a questionnaire and had to evaluate the EOL item. The learning phase then began. The students had to learn the six segments of the online environment at a learner’s pace (with a maximum duration of 20 min). They were able to navigate freely between the individual learning segments. When they had finished the learning phase, the students had to rate the JOL items for each segment. The dependent variables were measured after finishing the learning phase. The cognitive load was assessed first, followed by the two NASA-TLX items. Afterward, as in the previous research, a filler (distraction) task was implemented (see Weissgerber and Reinhard 2017). Participants were instructed to work on different tasks, but not to put a special effort into solving the task. The filler task consisted of questions about the capitals of German states, media preferences, lifestyle and food, a hidden object game, a counting task and a math test. The filler task lasted 25 min and was implemented to ensure rather robust learning effects. There is a lot of research in terms of multimedia learning that only assesses and interprets rather short-term learning effects which might be explained by the fact that the knowledge is still present in the working memory. We wanted to know whether rather robust learning effects could be observed and thus wanted to investigate whether the knowledge is still present when the working memory was distracted. Afterward, retention and transfer were measured. An RC item had to be answered after every retention and transfer question. Finally, the students had to answer a demographic questionnaire. When all the tests had been completed, the participants could leave the room. The experiment lasted a total of 60 min.

Results

Multivariate analyses of covariance (MANCOVAs) were conducted in order to investigate differences between the experimental groups. Signaling (signaled versus non-signaled) and extraneous load (ECL) induction (illegible font versus legible font) were used as independent variables for all the analyses and the students’ prior knowledge and prior experience were used as covariates. The test assumptions were examined, and only significant violations of these assumptions were reported. The effect sizes for all differences were only reported if they attained significance (p < .05). First, the learning outcomes were investigated in order to support or reject the hypotheses and to answer the research question. Next, the process variables were studied in order to obtain deeper insights into the learning process. Again, the focus is on the learning outcomes and the most important explanatory variables (cognitive load, metacognition). Results regarding navigation, learning and testing time, frustration and effort are displayed in “Appendix 2.” The descriptive results for all dependent variables are shown in Table 1, and bar diagrams for all dependent measures are displayed in “Appendix 1.”

Table 1 Mean and standard deviations of all dependent variables for the four experimental groups
Full size table

Learning outcomes

A MANCOVA was conducted using retention and transfer score as dependent measures. Prior knowledge was a significant covariate, Wilks’s Λ = 0.92; F(2, 131) = 5.79; p = .004; η 2p  = .08, but prior experience was not: Wilks’s Λ = 0.999; F(2, 131) = 0.08; p = .93. A significant effect with a small effect size was found for signaling: Wilks’s Λ = 0.95; F(2, 131) = 3.49; p = .03; η 2p  = .05. However, no significant effect was found for font legibility, Wilks’s Λ = 0.97; F(2, 131) = 1.72; p = .18, and no interaction was apparent: Wilks’s Λ = 0.99; F(2, 131) = 0.43; p = .66.

Follow-up ANCOVAs were conducted in order to obtain a detailed insight into the signaling effect. No significant effect was found for retention: F(1, 132) = 0.01; p = .91. A significant main effect with a small to medium effect size could be found for transfer: F(1, 132) = 6.24; p = .01; η 2p  = .05. Transfer performance was enhanced in the signaled conditions, in contrast to the non-signaled conditions.

Cognitive load

In order to obtain detailed insights into cognitive processes during learning, a MANCOVA was conducted using intrinsic load and ECL as dependent measures. Prior knowledge was a significant covariate, Wilks’s Λ = 0.92; F(2, 131) = 5.93; p = .003; η 2p  = .08, but prior experience was not: Wilks’s Λ = 0.98; F(2, 131) = 1.69; p = .19. A significant effect with a large effect size was found for signaling, Wilks’s Λ = 0.84; F(2, 131) = 12.80; p < .001; η 2p  = .16. A significant effect with a large effect size was found for font legibility, Wilks’s Λ = 0.69; F(2, 131) = 28.96; p < .001; η 2p  = .31. No interaction could be observed: Wilks’s Λ = 0.98; F(3, 131) = 1.17; p = .32.

Follow-up ANCOVAs were conducted in order to obtain detailed insights into both cognitive load facets. In terms of intrinsic load, no effect was found for signaling, F(1, 132) = 0.96; p = .33; however, a significant effect with a small effect size was found for font legibility, F(1, 132) = 6.54; p = .01; η 2p  = .05, and the intrinsic load was enhanced in the illegible font conditions, compared to the legible font conditions. No interaction could be observed, F(1, 132) = 0.01; p = .91. In terms of ECL, a significant effect with a large effect size was found for signaling, F(1, 132) = 21.91; p < .001; η 2p  = .14. ECL was reduced in the signaling conditions compared to the non-signaled conditions. A significant main effect with a large effect size was found for font legibility, F(1, 132) = 57.28; p < .001; η 2p  = .30. ECL was enhanced in the illegible font conditions, compared to the legible font conditions. Therefore, the induction of ECL through an illegible font can be viewed as successful. No interactions could be observed, F(1, 132) = 2.17; p = .14)

Metacognition and metacognitive accuracy

In order to investigate the metacognitive measures, a MANCOVA was conducted using EOL, mean JOL and mean RC score as dependent measures. Prior knowledge was a significant covariate, Wilks’s Λ = 0.92; F(2, 130) = 3.74; p = .01; η 2p  = .08, but prior experience was not: Wilks’s Λ = 0.99; F(2, 130) = 0.31; p = .82. There was no significant effect for signaling, Wilks’s Λ = 0.99; F(2, 130) = 0.54; p = .66. A significant main effect with a large effect size was found for font legibility, Wilks’s Λ = 0.78; F(2, 130) = 12.13; p < .001; η 2p  = .22, and an interaction with a medium effect size could be observed, Wilks’s Λ = 0.92; F(2, 130) = 4.04; p = .01; η 2p  = .09.

Follow-up ANCOVAs were conducted in order to investigate the main effect of font legibility and the interaction further. In terms of EOL, a significant main effect with a medium effect size was found for font legibility, F(1, 132) = 19.09; p < .001; η 2p  = .13. EOL was enhanced in the illegible font conditions, compared to the legible font conditions. A significant interaction with a medium effect size could be found: F(1, 132) = 7.86; p = .01; η 2p  = .06. In the non-signaled conditions, an illegible font led to higher EOL scores that a legible font. In the signaled conditions, the values only differed slightly (see “Appendix 1”). In terms of JOL, a significant main effect with a medium effect size was found font legibility: F(1, 132) = 16.26; p < .001; η 2p  = .11. JOL was reduced in the illegible font conditions, compared to the legible font conditions. No interaction could be observed, F(1, 132) = 1.04; p = .31. In terms of RC, neither a significant main effect for font legibility, F(1, 132) = 2.63; p = .11, nor an interaction could be found: F(1, 132) = 3.26; p = .07.

In order to investigate metacognitive accuracy, a MANCOVA was conducted using the EOL accuracy, JOL accuracy and RC accuracy scores as dependent measures. Neither prior knowledge, Wilks’s Λ = 0.95; F(2, 130) = 2.52; p = .06, nor prior experience, Wilks’s Λ = 0.99; F(2, 130) = 0.30; p = .82, was a significant covariate. There was no main effect for signaling: Wilks’s Λ = 0.98; F(2, 130) = 0.77; p = .51. A significant main effect with a large effect size could be found for font legibility: Wilks’s Λ = 0.78; F(2, 130) = 12.20; p < .01; η 2p  = .22. No interaction could be observed: Wilks’s Λ = 0.98; F(2, 130) = 1.10; p = .35.

Follow-up ANCOVAs were conducted to obtain deeper insights into the significant effect found for font legibility. In terms of EOL accuracy a significant main effect with a medium effect size could be found, F(1, 132) = 17.38; p < .001; η 2p  = .12. Students working with the illegible font had higher accuracy scores compared to those with the legible font. t-tests against zero (no over- or under-confidence) revealed that no condition was highly accurate. Students in the illegible font conditions were significantly under-confident, t(70) = 2.73, p = .01, whereas participants in the legible font conditions were significantly overconfident, t(66) = − 2.37, p = .02. There was no significant effect in terms of JOL accuracy, F(1, 132) = 3.23; p = .08, and RC accuracy, F(1, 132) = 0.04; p = .85. T test against zero revealed nonsignificant results.

In order to investigate relative accuracy, the gamma coefficients were calculated (see Table 1). Under the legible font conditions, no significant correlations could be found between the metacognitive judgments and the overall learning scores. In contrast, there were significant positive correlations between metacognitive judgments (JOL and RC) and the overall learning scores under the illegible font conditions. This indicates that students in the legible font conditions could not evaluate their learning and test performance accurately, whereas students in the illegible font conditions were able to estimate their performance more precisely, both during learning and the following retention and transfer test.

Discussion

The goal of the current study was to investigate how signaling affects learning and learning-relevant processes, in interdependence with an extraneous cognitive load which was induced by using illegible text fonts. The results revealed that signaling significantly enhanced transfer, but not retention performance. Therefore, hypothesis 1 was partially supported. Extraneous load induction had no significant effect on learning scores, even when the retention and transfer scores were descriptively reduced in the high extraneous load condition group. Concerning hypotheses 2a and 2b, no interaction, and therefore no moderation, occurred regarding both learning scores. In consequence, hypotheses 2a and 2b must be rejected.

The main effect of signaling on transfer can be explained by considering the CLT; color cues worked as attention-guiding features within the instructional text. The learners were assisted in identifying and selecting relevant. The attention of the learners was directed to the most important information which was necessary to build a complex schema. Additionally, learners were not distracted by less relevant information or overloaded by too much information at once. This is reflected in the extraneous load scores, which were lowered in the signaling condition group. The results suggest that signaling can be explained by the inherent cognitive benefits mentioned, since metacognitive judgments and accuracy were not influenced by color cues. Yet, two important restrictions have to be discussed. First, retention scores were not affected by color cues, and in consequence, the signaling effect could only be partially replicated. A possible explanation might be the low complexity of the learning material itself. Even if participants had low prior knowledge, the element interactivity of the material might be too low to overtax learners in terms of simple recognition of information. Color cues could not enhance retention performance since learners had only marginal difficulties in remembering basic facts, but were effective in helping the learner to obtain a comprehensive mental model which was necessary to adapt their knowledge to other problems. Thus, information got processed more deeply and constructed higher-quality schemata in their long-term memory. Second, according to the results of the additional measures (see “Appendix 2”), signaling was only beneficial for learners who had at least some prior knowledge. This supports the results of a recent meta-analysis by Schneider and colleagues (2018), who pointed out that learners with prior knowledge are better able to cope with the additional information provided by signaling, than those with low prior knowledge.

An illegible text font increased extraneous load with a high effect size. Therefore, the induction can be viewed as successful. Interestingly, intrinsic load was enhanced through an illegible font as well. The items from the used ICL scale (Klepsch et al. 2017) deal with the perceived complexity of the material. When learners had trouble, reading the material they might also think that the material is overly complex which might have led to the increased ICL rating. The induction of an extraneous load does not seem to influence learning outcomes in general. This is especially interesting since the perceived cognitive load was increased in the high extraneous load conditions. Seufert and colleagues’ (2017) findings in terms of the learning–inhibiting effect of a high disfluency could not be replicated.

This contradictory finding might be explained by considering metacognitive variables. On the one side, the enhanced ECL in learning environments had potential negative effects on learning. In line with recent research (e.g., Pieger et al. 2016), inducing extraneous load by using an illegible text font reduced confidence at the time the students first saw the learning environment. Learners receiving an illegible font were significantly under-confident and not accurate in regard to their later learning outcomes. Furthermore, learners reported an enhanced frustration, working with the illegible material (see “Appendix 2”). This might provide further support for the results reported by Reber, Schwarz and Winkielman (2004), who pointed out that disfluent perceptions of objects led to negative responses toward them.

On the other side, the overall EOL score was particularly enhanced by the extraneous load manipulation, which, in turn, encouraged metacognitive monitoring. Even if the EOL accuracy was reduced, the high extraneous load in the form of an illegible font acted as a trigger for more analytic and deliberate monitoring during learning (Alter et al. 2007), which was reflected in the relative metacognitive accuracy regarding JOL and RC. In contrast to the students with the low-extraneous-load conditions, those in the high extraneous load conditions were aware of potential difficulties regarding the learning task and thus were able to estimate their performance correctly, both during learning and the following retention and transfer test. In consequence, the metacognitive benefits of a high extraneous load might compensate for the negative cognitive effects of such a load so that the learning outcomes were not influenced. This is further supported by considering additional variables (see “Appendix 2”). Even if frustration is enhanced through an illegible text font, the learners reported that they put more effort into understanding the learning content.

The results of this experiment support the assumption that the induction of extraneous load is not detrimental to learning in general. Instead, the higher extraneous load induced through the manipulation of perceptual disfluency led to more elaborate metacognitive monitoring concerning JOL and RC which at least compensate for the negative effects of a high ECL. These metacognitive benefits have an effect even when the disfluency manipulation is particularly strong (Seufert et al. 2017).

No interaction effects regarding learning could be found. Statistically, signaling seems to be beneficial for transfer, independent of the induced extraneous load. However, interaction effects occurred in terms of EOL judgments. Especially in the high-extraneous-load condition group, signaling reduced EOL judgments, indicating that signaling might work as a visual cue for reducing perceptual disfluency. Learning behavior changed as a result (see “Appendix 2”). Students working under the high-extraneous-load condition with implemented color cues navigated through the pages more often without taking any more time for the learning itself than those working under the high-extraneous-load condition without color cues. These results patterns reversed in the low-extraneous-load conditions group. Implementing signals in learning environments with a high extraneous load might weaken the learning-relevant metacognitive monitoring processes and therefore change learning behavior. This is especially interesting since descriptively, the retention scores were enhanced in the low extraneous load—signaled—and high extraneous load—non-signaled conditions—indicating that extraneous load might have an influence on the effects of signaling. Implementing signals in disfluent learning environments might weaken the learning-relevant metacognitive monitoring processes and therefore change learning behavior. Nevertheless, the results regarding interactions in the learning scores did not reach significance, and therefore, these interpretations have to be viewed with caution.

Implications

On the theoretical side, the signaling principle is supported partially. Implementing attention-guiding cues in an instructional text can foster learning transfer, but not retention processes. Inducing an extraneous load through the learning environment does not seem to moderate the effect of signaling on learning outcomes. This is a theoretical implication which is in contrast to the concept of element interactivity through extraneous elements. Nevertheless, the fact that learners should have at least a low or moderate level of prior knowledge in order to cope with the additional load from the implemented signals must be taken into account. The element interactivity effect could not be replicated by manipulating element interactivity by inducing an extraneous load, instead of manipulating an intrinsic load. Thus, extending the concept of element interactivity to an extraneous load (Sweller 2010) might be subject to additional boundary conditions. For example, the learners’ prior knowledge is important. If they have no prior knowledge, inducing load through illegible fonts impairs their learning compared to those learners who have a low to medium degree of prior knowledge. Furthermore, the induction of extraneous load by manipulating visual presentation can lead to metacognitive benefits. These benefits can compensate for the negative effects of a multimedia learning environment’s learning–inhibiting design.

On the practical side, designers should be encouraged to signal important parts of instructional texts, to foster learning transfer and to reduce the learners’ irrelevant cognitive load. It is irrelevant whether instructors aim to create a simple learning environment or one that is more complex with potential extraneous cues. Signaling can be implemented in order to enhance transfer and reduce the subjective irrelevant load. The font is not significant for learning success, but designers should be aware that illegible fonts lead to frustration and a higher perceived load. If the learning material induces an extraneous load, designers should be particularly aware that signaling changes the learning behavior, compared to a text without it.

Limitations

The manipulation might have been too weak in terms of extraneous load. Although the fonts used were pretested, a stronger extraneous load manipulation might lead to differences in the effects of signaling, since the metacognitive benefits would be unable to compensate for the induced load. According to Seufert and colleagues (2017), multiple methods of inducing disfluency should be considered, because perceptual fluency, at the level used in this study, might not be a diagnostic cue for performance (e.g., Dunlosky and Thiede 2013). In consequence, it is difficult to generalize the current results to other kinds of extraneous load induction. To generalize the finding from the current investigation, it would be necessary to implement and investigate various manipulations. In addition, the disfluent font, which was used in this experiment, looks more like a handwritten font than the fluent font. The perception of it as a handwritten font could trigger social processes since the learner might feel that they were being addressed by an actual human being with their own intentions. This might trigger “para” social communication processes and foster processes, like attention (e.g., Grice 1975), thus further compensating for the negative effect of a high extraneous load. Recent meta-analyses (e.g., Schneider et al. 2018) have indicated that different types of signaling (e.g., color coding, organizational signals) have different influences on learning. Therefore, it is difficult to generalize the results for other signaling modes. Another limitation must be mentioned regarding the exploratory analyzes in terms of prior knowledge; dividing the participants into two groups and investigating them separately led to a significant drop in statistical power. This must be taken into account, especially when discussing the descriptive trends of individual learning measures. Finally, the filler task has to be discussed. Even if the filler task was just implemented to distract the participants from the material, it is possible that some participants put special effort into it. Thus, it is possible that retroactive interference occurred (e.g., Craig et al. 2015). Participants who put more effort into the task could have been adversely affected, and thus, learning performance could be lower.

Future directions

Based on the limitations of the current study, future research should focus on different types of signaling modes, signaling types and the induction of extraneous load. Extraneous load was induced through the text font in this study, but other methods, such as the use of dialects (Sweller 2010) or decorative pictures (Rey 2012), should also be taken into account. Different types of learning material should be investigated in this context. Signaling implemented in simple or potentially irritating graphics might be a promising field for future research. Finally, the current study was carried out with a student sample. In order to strengthen external validity, additional studies should be carried out with primary or secondary students, since these age-groups often work primarily with instructional textbooks.