Exploring the Relationship Between Implicit Scaffolding and Inclusive Design in Interactive Science Simulations

Educational interactive simulations can be powerful tools to support student engagement and achievement in science disciplines [1, 2]. Interactive science simulations contain an underlying model based on scientific phenomena (e.g., current and voltage in a circuit simulation). By allowing students to change model parameters and experience the outcome of these changes, interactive simulations can be particularly useful for supporting students’ engagement in authentic science practices in the classroom and enjoyment of the process [3–5].

PhET simulations (sims) are a popular suite of interactive simulations developed by the PhET Interactive Simulations project [6] at the University of Colorado Boulder. The PhET project impacts classrooms around the world through over 130 interactive science and mathematics sims and associated teacher resources. The sims are run over 75 million times per year, across K-16 levels, and have been translated into 75 languages. Despite the potential of PhET sims to foster engagement and participation in science education, they are currently inaccessible for many students with disabilities – due in part to their reliance on visual representations.

In 2014 PhET began to design new inclusive features to enhance the accessibility of the sims [6]. In collaboration with the Inclusive Design Research Centre, we began developing prototypes of inclusive features [7], prioritizing initial efforts on keyboard navigation and auditory descriptions (to be read by screen readers). Our long-term goals are to transform our growing suite of PhET sims into the most inclusive learning tools possible, and to develop and share research and design guidelines for the inclusive design of other interactive educational resources.

A hallmark of PhET sims is their design through the implicit scaffolding design framework, an approach developed by the PhET project for the design of interactive science sims [8–10]. Implicit scaffolding includes the use of affordances and constraints to cue and guide students to engage in pedagogically productive interactions, while maintaining students’ agency and choice during the learning process. The end result is that students can be (implicitly) guided along beneficial and efficient learning paths without the sim being directive or inhibiting active engagement or the students’ feeling of control – key components of successful science learning [11]. In other words, PhET’s sim designs guide students to engage in productive interactions without students feeling guided. To date, PhET’s work on the implicit scaffolding design framework has centered on visual cues and choice of interactions compatible with mouse or touch-interfaces.

As we began the design of inclusive features for PhET sims, we wanted to ensure that the outcomes of implicit scaffolding – e.g., productive learning interactions, active engagement, and agency in the learning process – extended to inclusive features, and ultimately could be experienced by all students. Foundational components of implicit scaffolding [8] include: (1) sequence and interactions, (2) framing engagement, (3) enabling sense making, and (4) continued engagement. To ensure that students engaging with PhET sims experience equivalently engaging and effective learning experiences regardless of what inclusive features they may utilize, we want to understand the relationships between implicit scaffolding and inclusive design. In this work, we analyze how users with visual impairments interact with a prototype PhET sim utilizing a screen reader and compare this with our prior work on how users engage with PhET sims visually, to broaden our original implicit scaffolding design framework to include non-visual design.

For this work we focused on design and user testing of the HTML5 prototype sim Capacitor Lab: Basics [12]. The design of this sim reflects a specific sequence (ordering of screens, representations, and interactions), and set of interactions (the choice of what representations are interactive, and the specifics of that interactivity) to address the selected learning goals of the sim. In this section, we introduce the learning goals, and associated sequence and interactions of the Capacitor Lab: Basics sim, to provide a concrete example of PhET’s implicit scaffolding approach.

3.1 Setting Scope of the Sim: Learning Goals

The Capacitor Lab: Basics sim focuses on the physics of capacitors at the advanced high school (16–18 year olds) and introductory college levels. The sim was designed to support students to be able to: (1) Predict how capacitance changes with changes in voltage, capacitor plate area or separation; (2) Explain the relationships between voltage, charge, stored energy, and capacitance, and (3) Describe how a capacitor can be used to light a light bulb.

3.2 Sequence and Interactions

To achieve these learning goals, Capacitor Lab: Basics contains two screens for users to explore, the Capacitance screen and the Light Bulb screen (Fig. 1). For the purpose of this paper, we will focus on the Capacitance screen only. In the Capacitance screen, there is a capacitor (two parallel metal plates) connected to a battery; a bar graph that displays the value of the capacitance (a measure of the stored electrical charge); a voltmeter that allows the user to measure voltage at various points on the circuit and across the capacitor plates; and a control panel with checkboxes to add or remove the bar graph, and the plate charge, electric field, or current representations.

Capacitor Lab: Basics Simulation. Capacitance screen (left), Light Bulb screen (right). Images reproduced with permission from PhET Interactive Simulations.

Full size image

The user can change the voltage on the battery, the capacitor plate separation and plate area, and learn how these changes affect capacitance. Visual representations of plate charge and electric field are overlaid on the capacitor and change dynamically as the user interacts with elements in the circuit. Plate charges are represented by “positive” and “negative” symbols evenly distributed across the plates that change in number as voltage, plate area, and plate distance are changed. Electric field is represented by vector arrows that change in length, number, and direction as the voltage, plate area, and plate distance are changed.

3.3 Inclusive Feature – Auditory Descriptions

For the Capacitor Lab: Basics sim, we focused on the design of the user experience for those accessing the sim with a screen reader – specifically, navigating the screen with a keyboard and utilizing screen reader shortcut keys to access auditory descriptions and interact with the sim. For technical details regarding the implementation of these inclusive features, please see [7].

Scene Description.

When a user first opens the sim, using a screen reader, they can navigate through a Scene Description. The Scene Description describes the layout and interactive features of the sim, and is structured hierarchically through the use of headings, paragraphs, etc. This description starts with a general overview of what is available on the screen (labeled as a Heading 1: “The Scene for the Capacitance Screen”), followed by section headings that describe each area of the sim screen (each area labeled as Heading 2: “Play Area”, “Graph Panel”, and “Control Panel”). After each heading are subheadings that list the interactive elements found in that area, the current state of these elements, and – in cases where the interaction may not follow standard conventions – a description of how to interact with that element. In Table 1 we present an example of the first three subheadings and descriptions found after “Play Area”.

Interactive Elements.

Users can also navigate to each interactive element in the sim and hear a description of that element. As the user makes changes in the sim with the interactive elements, they hear live updates of changes to that element and to all other impacted sim features. For example, the battery in the circuit can be navigated to by pressing the “Tab” key to navigate to the circuit group, and then pressing the “Enter” key to enter the circuit group. The battery is the first interactive element in the circuit group. The battery consists of a slider, allowing control of the voltage available to the circuit. As the user changes the battery voltage (using the arrow keys to change the voltage slider) the user is updated on the current voltage value, and any other changes in the sim that may occur as a result of this voltage change (e.g., the flow of current).

As part of the iterative design process, we conducted user interviews to explore and refine design ideas. Each interview was structured into three sections. First, users were asked three conceptual questions to help assess their knowledge of the sim topic, capacitance. Next, users were provided with a computer, equipped with the screen reader NVDA [13] with the sim open and ready for use, and were asked to “Explore the simulation, and try to think out loud as you explore.” The sim use portion of the interview was conducted in a think aloud format, where users were asked to think aloud as they explored the sim, with little to no interruption from the researcher. After exploring the sim, users were again asked three conceptual questions to help assess any knowledge they may have gained through using the sim, as well as general questions about usability and sim design.

During the interview, we collected audio recordings of users’ verbalizations, and video recordings of the computer screen and of users’ keyboard interactions and gestures. We analyzed these data sources for evidence of the sim’s non-visual scaffolding in supporting or inhibiting the users from productive interactions.

4.1 Participants

A total of three interviews have been conducted with Capacitor Lab: Basics prototypes, with the inclusive features of keyboard navigation and auditory descriptions. The first interview was conducted with an early prototype, and the user encountered significant technical barriers to sim use. In this work, we present results from the remaining two user interviews with the sim, both conducted with the same prototype of Capacitor Lab: Basics. We will identify users by the pseudonyms Amy and Leela. Amy was a recent college graduate, and Leela was a college senior. Neither had, nor were pursuing, degrees in a physical science. Both users identified as being blind, and used a screen reader on a daily basis. Each user was interviewed individually, for about one hour. In this work, we focus on the sim exploration section of the interview. Amy used the sim for a total of 21 min, and Leela used the sim for a total of 29 min. Both users had used the screen reader NVDA prior to being interviewed.

We start by summarizing the user interviews with the Capacitor Lab: Basics prototype. We then go through components of the implicit scaffolding design framework and present examples of user interactions that highlight ways the design of the prototype sim with inclusive features supported the screen reader users (or not) to engage in productive interactions.

5.1 User Interview Overview

Though Amy and Leela each chose a unique sequence of interactions there were numerous similarities in the two user interviews. Both users started their interaction by navigating sequentially through the Scene Description (described in Sect. 3.3), and listening to all (Amy) or most (Leela) of the descriptions. While listening to the descriptions, each decided on a specific element they wanted to interact with first. They then started interacting with elements in the circuit group, the battery voltage and the capacitor plates. After interacting with all of the sim’s interactive elements (in Amy’s interview, all except for the voltmeter), both users chose a specific conceptual goal to pursue. Amy wanted to determine how all of the interactive elements in the circuit group (battery voltage, capacitor plate area and separation) impacted the capacitance (as indicated by the capacitance graph readout). Leela chose to use the voltmeter tool to try to determine what interactive elements in the circuit group impacted the voltage of the circuit.

At this point, the two interviews began to differ significantly. Amy encountered no usability challenges while determining the relationships between the circuit elements and capacitance, while Leela had significant usability challenges with the voltmeter. The result is that Amy was able to focus on learning from the relationships represented in the sim, and Leela spent a significant amount of time trying to make productive use of the voltmeter. As Amy was able to explore the concepts presented in the Capacitance screen more deeply, there was not enough interview time for her to explore the Light Bulb screen. In Leela’s interview, she spent the majority of her Capacitance screen time focused on determining if changing the circuit features (battery voltage or capacitor area or distance) impacted the circuit voltage (hindered by usability issues with the voltmeter), and was then prompted by the interviewer to try exploring the second screen of the sim, the Light Bulb screen. On this screen, she set a goal to light the light bulb. To light the light bulb she needed to increase the battery voltage to charge the capacitor, and then connect the capacitor circuit to the light bulb. She was able to successfully light the light bulb in under three minutes after starting on the Light Bulb screen, and was pleased to accomplish this task.

Ultimately, both users were able to access the Scene Description and the interactive elements of the sim. They were able to fluidly navigate to all sim elements, and aside from the voltmeter, make productive use of these elements. Even though neither user was given a specific goal (other than to explore the sim), both were able to meet some of the learning goals (see Sect. 3.1) intended by the designers. Amy was successfully able to meet all of the learning goals of the Capacitance screen. Leela struggled with usability issues in the Capacitance screen, but was able to rapidly accomplish a main goal of the Light Bulb screen.

5.2 Scaffolding Insights from User Interviews

Here, we go through two components of the implicit scaffolding design framework – framing engagement and enabling sense making – and highlight specific interview segments that provide insights into the relationship between implicit scaffolding and inclusive features. To present the interview data, we show interview segments with time points in the min:sec format (starting from first auditory description read aloud by the screen reader), followed by a brief summary of the user’s interactions and verbalizations in that time segment. When brief, full quotes and interaction patterns are provided. Lengthy quotes or interaction patterns are abbreviated or summarized.

Framing Engagement.

Framing can be thought of as the way in which a user determines an answer to the question “what sort of activity is this?” [14]. For PhET sims, a goal of the design is for students to interpret use of the sim as an opportunity to actively explore and make sense of the representations and the relationships found in the sim. In this section, we look at how two users of the sim prototype, utilizing screen readers, interpreted the activity of sim use and how they framed their engagement.

Initial Listening.

Amy and Leela each started their sim use by using the keyboard to navigate through, and listen to, the Scene Description for the Capacitance screen. Here are the users first few minutes with the sim, starting with Amy, when primarily focused on listening to the Scene Description and spatially orienting themselves to the representations on the screen.

0:00:

Amy hears first readout from Capacitance Screen, begins navigating and listening to rest of Scene Description, listening to some descriptions multiple times.

1:10:

Begins using left hand to gesture, indicating locations of objects being described, continues gesturing until has listened to all descriptions.

4:16:

Screen reader stops. Amy says “Okay. I think I more or less know the parts.”

In total, Amy spent just over four minutes listening to the descriptions. As she listened, she paid particular attention to the spatial cues provided, as indicated by the corresponding hand gestures she made as she heard each spatial description. Here are Leela’s first few minutes with the sim.

0:00:

Leela hears first readout from Capacitance Screen, begins navigating and listening to rest of Scene Description, skimming some quickly (without waiting for full description) and listening to some descriptions twice. Quietly repeats portions of descriptions to herself.

0:48:

Repeats out loud from the description “There’s a capacitor to the left.”

2:14:

Screen reader stops reading. Leela changes screen reader from “browse” mode to “forms” mode and begins interacting with the battery’s voltage slider.

Consistent with prior research on the use of screen readers [15], both users engaged in an initial listening phase following a remarkably similar pattern. Interestingly, both users indicated a focus on spatial aspects of the descriptions, though each indicated this in different ways – Amy gestured and Leela verbalized. Later in the interview, both users verbalized some of the information provided in the Scene Description to help them with sim use, and each navigated back to the Scene Description later to look for specific information.

Transition to Interaction.

After listening to the Scene Description, Amy and Leela differed in how seamlessly they transitioned to interacting with the sim’s features. Amy did not realize that the screen reader needed to be switched from the “browse” mode it was currently in, to “forms” mode to allow her to interact with sim elements.

4:36:

Amy says “It’s not clear…” and goes on to describe how she is confused by the descriptions that say “press enter” to interact, but she does not know where keyboard focus should be when pressing enter. She navigates screen reader focus to different headings and tries various keyboard keys, all unsuccessful.

6:00:

Interviewer suggests switching screen reader to “forms” mode. Amy makes this change, and begins navigating around the sim’s elements, passing the circuit group, but not entering the circuit group.

6:52:

Amy indicates she is unclear how to navigate to the interactive capacitor plates by saying she is looking for the “left right slide and the up down slide to kind of change how far apart the capacitance were and how large they were. So I’m looking for that right now, because I figured that’s the first thing I would start playing with.” She continues navigating around the sim, searching for the capacitor plate sliders.

7:59:

Interviewer suggests pressing the “Enter” key when on the circuit group. Amy does this and navigates to the plate separation slider and begins changing the plate separation. She does not require any further prompts from the interviewer through the rest of her sim use.

In contrast, Leela recognized independently that the screen reader needed to change into “forms” mode, and did this without prompting once she was done listening to the Scene Description, as indicated in the description of Leela’s initial listening.

A note regarding screen reader modes – the sim is compatible with different screen readers, each with different modes and unique methods for changing modes. An unmet challenge we face is the design of appropriate descriptions to cue users that they may need to switch screen reader modes when transitioning from listening to the Scene Description to interacting with the sim’s elements. This sim prototype did not contain any descriptions regarding changing modes, and from these interviews it seems that some users may infer that a change of mode is needed, while others will need some description to support them in making this transition.

Aside from the challenges Amy faced transitioning to using the sim’s interactive elements, both users seemed to take similar approaches to sim use and framed the activity of sim use in similar ways. Both listened to the Scene Description, and from this, selected a specific interactive feature they wanted to explore. They then navigated to that feature to begin exploring. For comparison, students utilizing the visual design of the sim (not using a screen reader), typically begin using the sim by first taking a few seconds to visually scan the sim and then begin interacting with sim elements [3, 8, 16]. For Capacitor Lab: Basics, initial interactions are typically with the elements in the circuit group. These particular elements are highlighted visually through their central location, large size, and brightly colored touch or mouse target areas (blue slider knob for the voltage slider and green arrows for capacitor plates).

Amy and Leela utilized a different approach with the use of the screen reader, though obtained a similar outcome. Their initial choice of interactions (Amy choosing the capacitor plate separation, and Leela choosing the battery) is consistent with those of users exploring the sim visually, indicating our auditory descriptions may appropriately cue users to explore, starting with features in the circuit group.

Enabling Sense Making.

We designed the sim’s representations and interactive elements to support students in making sense of the underlying scientific or mathematical relationships. This requires that the users interact with the sim (nothing occurs in PhET sims prior to user interaction), receive feedback from the sim based on their interaction (e.g., the form of changes in the representations) and appropriately interpret this feedback to form their conception of the underlying relationships. In this section, we look to the interviews to understand how the feedback provided was working to support understanding of the underlying relationships.

In Amy’s interview, she articulates the goal of sense making in an interesting way. As described previously, she begins interacting with the sim by changing the capacitor plate separation (7:59). Next, she articulates a need for a larger purpose, and uses the sim to seek one out.

8:24:

Changes capacitor plate separation, plate area, battery voltage, and navigates through all of sim’s interactive elements.

11:05:

Changes from “forms” mode to “browse” mode, and begins listening to the Scene Description.

11:39:

Says “…I’m looking for where it would tell me… so if I can control voltage, I can control size and I can control distance…so that all goes together to… so what’s the output?” Continues listening through description.

13:20:

Listens to description of graph, which reads “Capacitance Graph, measures the capacitance of the capacitor”. Amy says “I guess I’m looking for capacitance.”

In this segment, Amy has explored multiple sim elements that she can change, and recognizes that there must be some larger purpose to changing these elements. She then actively searches out what this could be, and determines that the larger purpose could be related to capacitance.

As designed, the capacitance value changes when the capacitor plate separation or area is changed and the screen reader provides live updates to the user of the new capacitance value. Unfortunately, the screen reader was not providing live updates of the capacitance value as it changed (a technical issue that appeared during the interview), so Amy was only aware of the capacitance from the Scene Description and from navigating by the capacitance graph in her exploration. If this issue had not come up, Amy would likely have recognized sooner that changing the circuit group elements impacted capacitance. Once Amy indicates an interest in capacitance, the interviewer lets Amy know the technical issue with the capacitance readout, and that Amy can access the capacitance information by asking the interviewer as needed. Amy goes on to engage in experimentation with the circuit group elements with her newfound focus on capacitance.

13:27:

The interviewer tells Amy that the capacitance value should be read out by the screen reader each time her actions result in a change in capacitance, but that this is not occurring due to a technical issue with the sim. Any time Amy would like to know the capacitance value, the interviewer can read it aloud to her – the current value of the capacitance was 0.11 pF (where pF is picofarads, a unit of capacitance). Amy indicates she understands, pauses to think about what capacitance is, and decides to experiment and find out. She switches from “browse” mode to “forms” mode.

14:25:

Amy makes an initial prediction before interacting “I assume that if I push the voltage up, the capacitance is also going to go up, but it might stay the same because it might not be dependent on – I don’t know, so I’m going to check.” She navigates to the battery’s voltage slider, increase the voltage and asks “How many picofarads?”. Capacitance was 0.11 pF. Amy says “Okay, so it’s the same. So voltage does not impact – okay.”

15:29:

Amy navigates to plate area slider, currently with a value of 121 mm² (millimeters squared, a unit of area). “Okay, so I had it at 121.” She increases area to 144 mm². “Okay, so if I go up, then what is it?” Capacitance was 0.13 pF. She increases area to 169 mm². “Okay, what is it there?” Capacitance was 0.15 pF. “Okay, the size of the capacitor is increasing, umm, I don’t know if exponentially is the right word.” She goes on to describe to the interviewer, using hand gestures and capacitance values, her ideas about the mathematical relationship between the plate size and capacitance.

18:01:

“I’m going to take the size back down to where the default was.” She decreases the plate area back to 121 mm², and navigates to the plate separation slider. She decreases plate separation, and then increases it back to maximum (10 mm). “Okay, so it’s back to 0.11 there, right?” Interviewer agrees. Amy decreases the plate separation from its maximum (10 mm) to its minimum (5 mm), asking for the capacitance readout with each change. She then describes to the interviewer, using hand gestures and capacitance values, her ideas about the mathematical relationships between the plate separation and the capacitance.

While each PhET sim provides multiple representations and interactive elements to explore, it is ultimately the goal for students to explore these features and attend to specific relationships between them rather than attend only to each in isolation. Amy’s interview provides an example of a user recognizing that there must be more to the sim than simply interacting with the individual elements. It was unfortunate that the capacitance readout was not updating as intended, but it was illuminating to see her process of seeking – and finding – relationships to engage in sense making with.

Student exploration of PhET sims using the visual design typically includes use of all or most of the sim’s interactive elements before selection of specific relationships for deeper sense making. Amy and Leela’s use followed this pattern of exploration before deeper sense making behavior, though required more time. In the example above, Amy spent three minutes exploring the sim’s features before seeking an underlying relationship to explore. In contrast, prior analysis of 22 student groups freely exploring a PhET sim during a college chemistry class showed that students explored 18 out of the 23 interactive elements available across the sim’s three screens – and each screen’s interactive elements were typically explored in less than a minute [3].

While successful in many ways, this work also encountered multiple challenges to be addressed in the future. Some of these challenges include:

• Technical Implementation Challenges. Amy and Leela differed significantly in their overall learning experience with the sim – in large part due to technical difficulties inhibiting Leela from making productive use of the voltmeter.

• Inclusive Design Challenges. Some representations were not used consistently for sense making (e.g., Amy not using voltmeter, Leela not making use of the capacitance readout). Additionally, not all representations were described (i.e., electric field, charges), or were not described usefully (e.g., current).

• Designing for Efficiency Challenges. Some aspects of sim exploration took longer in the user interviews than is typical for those visually exploring the sim. Some of this additional time seemed intrinsic to the screen reader users’ approach (e.g., listening before interacting) while others (e.g., the amount of time required to explore all interactive elements) indicate that the efficiency of the interactions could be improved.

Results of our analysis indicate that the underlying premise of implicit scaffolding is indeed multi-dimensional, and prior work in implicit scaffolding for visual interfaces can be applied to non-visual interfaces. To do this, some components need to be broadened and expanded, and initial assumptions need to be replaced with more inclusive perspectives. For example, in prior work [8] we specify that implicit scaffolding is “neither written nor verbal” to highlight that implicit scaffolding provides guidance without explicitly providing a series of actions for the user to enact. In this work, we show that providing verbal scaffolding can be done implicitly, resulting in productive user actions and learning. This result provides a direct relationship between the implicit scaffolding design framework and inclusive design, connecting science education research and inclusive design through the use of an interactive science simulation.