Shared Mental Models in Support of Adaptive Instruction for Teams Using the GIFT Tutoring Architecture

We are evolving a concept that will enable Intelligent Tutoring Systems (ITS) to support adaptive instruction in collective tasks. These tasks have been explored and discussed in the Artificial Intelligence in Education (AIED) literature primarily as collaboration, collaborative problem solving, and collaborative activities (Adamson et al. 2014; McManus and Aiken 1995, 2016; Soller 2001). Collectives may be defined as two or more individuals who are attempting to accomplish one or more shared objectives. Collectives may be independent, requiring individual members to communicate or interact in order to accomplish their objectives, or interdependent, requiring frequent, coordinated interactions among their members (Salas and Cannon-Bowers 2000; Fiore et al. 2010). Members of independent collectives may be trained primarily as individuals and by using a wide variety of well-known instructional techniques. Our concern here is with interdependent collectives, which we designate as teams (Salas et al. 2015). Training for teams has been widely studied, but research and development efforts to apply ITS techniques in team training are rare.

A critical antecedent for high performing teams is an understanding shared by all team members of team objectives and the processes needed to achieve them (DeChurch and Mesmer-Magnus 2010; Fletcher and Sottilare 2013; Salas et al. 2015). This collective understanding was identified by Rouse and Morris (1986) as a shared mental model – a model that must be taken into account to enhance team training and team performance (e.g., Blickensderfer et al. 1997). We acknowledge that non-cognitive factors (e.g., physiological and affective states) influence team performance and processes just as they do in individual performance (Landon et al. 2016; Van den Bossche et al. 2006). However, here we focus on cognitive factors in small teams (e.g., squads, crews, seminars, classroom teams, or other small groups) and leave the discussion of both non-cognitive factors and higher echelon collectives (teams of teams) for separate consideration.

The primary contribution of this article to artificial intelligence in education research and development is to suggest the synthesis of a process for developing and applying shared mental models of cognition to the adaptive instruction of collective tasks. This process is a necessary precursor to nurturing the development and maintenance of shared mental models within ITSs. We provide a rudimentary discussion of issues and factors involved in the development and application of mental models used in adaptive training for interdependent teams.

We begin with a discussion of teams and teamwork. We then provide an overview of shared mental models in teams that is followed by discussion of team training. After that we turn to the nexus of intelligent tutoring, shared mental models, and team training. Next we describe the Generalized Intelligent Framework for Tutoring (GIFT), an open source architecture for authoring, delivering, managing, and evaluating ITS (Sottilare et al. 2012). This description is followed by a discussion of GIFT and how it can contribute to team learning by capitalizing on the likely effectiveness of ITSs that apply both individual and shared mental models. The article concludes by suggesting next steps for research and development of these capabilities and the value of doing so.

Most activity in commerce, business, and the military is performed by teams (Cannon-Bowers and Bowers 2011). Teams are interdependent collectives consisting of two or more individuals with different responsibilities who must interact with one another to accomplish a common task, objective, or mission (e.g., Kozlowski and Ilgen 2006). In addition to interdependency, teams have well-defined functional roles, share more than one information source, and are hierarchically organized (Salas and Cannon-Bowers 2000). Roles and responsibilities of individual team members may be specifically assigned, or they may arise spontaneously, depending on team size, team leadership, presence of newcomers, or exigent demands (Guimera et al. 2005; Hirst 2009). There are, of course, teams within teams (Mathieu et al. 2001), and most teams are components of a larger enterprise.

Cannon-Bowers and Bowers (2011) suggest that the most complex and taxing performance demands on an individual in a typical operational environment arise from participating as a member of a team. These demands may be primarily due to the competencies an individual must acquire to perform teamwork effectively as a member of a team, sharing knowledge and equipment, in addition to the basic competencies an individual must possess to perform tasks (or ‘taskwork’ in contrast to teamwork) as an individual within a team. Salas and Cannon-Bowers (2000) discuss cue-strategy competency that is an essential function in teamwork and must be acquired by individual team members -- a competency that enables team members to participate in performing essential team activities without overt coordination or communication. Kozlowski and Ilgen (2006) write of the additional, complex, and dynamic nature of teamwork arising from the constant need to adapt to emergent teamwork processes and phenomena. Grand et al. (2016) have described these dynamics by modeling team cognition, shared knowledge of team tasks, and other mechanisms that underlie team knowledge development.

Teams vary in the amount and intensity of interactions they require. Roby and Lanzetta (1958) long ago described this as the “assembly effect” or more colloquially as what Fletcher and Sottilare (2013), Salas and Cannon-Bowers (2000), and others describe as “teamness”. An early study by Jones (1974) examined this issue. He compared baseball, tennis, football, and basketball teams by regressing the capabilities of individual team members onto team effectiveness and success. Jones found success to be positively associated with the effectiveness of individual members of baseball, tennis, and football teams, but not basketball teams, where success depends more heavily on balanced communication, timing, and coordination among team members than in the other three. The greater the need for these functions, the greater the need to deal with the team as a learning unit in its own right.

To coordinate and communicate, team members must learn what specific information other team members need, when they need it, and how well other team members and they, themselves, perform team tasks. Team members must also understand team processes -- how information and activities are shared, communicated, and coordinated. Team processes may be determined by the team itself or obtained from external sources. Marks et al. (2001) define team processes as members’ interdependent actions that convert inputs to outcomes through cognitive, verbal, and behavioral activities directed toward organizing ‘taskwork’ to achieve collective goals. They define teamwork as interdependent acts by team members to convert inputs to outcomes through the cognitive, verbal, and behavioral activities that are used to organize tasks in achieving team goals.

Teamwork competencies were summarized by Salas et al. (2015) and Sottilare et al. (2015) as the following “7 C’s”:

• Cooperation—Attitudes and beliefs about the team and the value of teamwork.

• Coordination—Behavioral mechanisms including mutual performance monitoring, back-up and support, adaptability, and task-related assertiveness.

• Communication—Information exchange protocols including clarity, timeliness, confirmation, and openness.

• Cognition—Shared understanding of roles and responsibilities, knowledge of team objectives, knowledge of other team members, and cue-strategy associations.

• Coaching—Leadership activities promoting teamwork, concern for other team members, and sharing.

• Conflict Resolution—Interpersonal skills, mutual trust, and psychological safety for other team members.

• Conditions (Context, Composition, Culture) — Clear team norms for appropriate and supportive behavior.

This article notes that ITSs use dynamic mental models for training individuals. It suggests that augmenting the mental models of individuals with an analogous application of the mental models of team processes, goals, and activities shared by interdependent members of teams, the increasingly manifest advantages and capabilities of ITSs may be made available for training teams.

The notion of mental models may be found in the primordial origins of scientific psychology. William James (1890/1950) stated his General Law of Perception as the following: “Whilst part of what we perceive comes through our senses from the object before us, another part (and it may be the larger part) always comes out of our mind” (p. 747, 1890/1950). A mental model, then, is a mental representation of the perceived world informed, however imperfectly, by our senses.

Shared mental models may be viewed as the descriptions, explanations, and predictions of team behavior that members of a group, such as a team or collective, hold in common. Rouse and Morris (1986) determined that shared mental models reflect organized knowledge that enables humans to understand the basic functioning of any system and then form predictions and expectations about its future states. They further identified common themes in the use of these models by teams and described these models as “mechanisms whereby humans are able to generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions (or expectations) of future system states” (p 351).

The success of interdependent teams in performing tasks and achieving objectives and missions then, depends on the extent to which this model sharing occurs, and the proficiency and competency of teams in performing their tasks and missions may be keyed to these models. Cannon-Bowers et al. (1993) invoked the concept of shared mental models to explain the performance of expert teams that seamlessly coordinate actions without the need for overt communication. They concluded that these models were the “knowledge structures held by members of a team that enable them to form accurate explanations and expectations for the task, and in turn, to coordinate their actions and adapt their behavior to demands of the task and other team members” (p. 228). More recently Cooke et al. (2012) and Gorman (2014) have emphasized that these mental models should not be viewed as property or products, but as dynamic cognitive activities rooted in team member interactions and a meaningful context. Given this understanding of shared mental models as complex, dynamic processes begins to suggest the need for machine intelligence, such as that found in ITSs, to provide adaptive instruction for teams.

Team members with shared mental models of objectives and processes more easily synchronize and adapt as they plan and execute their tasks whereas teams with inconsistencies in their models of the team suffer from absent and/or ineffective team processes (e.g., Cannon-Bowers et al. 1990; Espevik et al. 2011a, b; Moore et al. 2015; Rouse and Morris 1986).

These shared models involve (1) Orientation functions, such as information exchange regarding member resources and constraints, (2) Resource-distribution tasks that involve balancing the task load across members, (3) Timing functions that influence how activity is paced, (4) Response-coordination such as the timing and synchronization of coordination, (5) Motivational functions, such as balancing attention to individual and team goals, (6) Systems-monitoring functions that enable the adjustment of team and member activities in response to errors and omissions, (7) Procedure maintenance, such as monitoring of general procedures and activities. The models enable team members “to interpret cues in a similar manner, make compatible decisions, and take appropriate action” (Cannon-Bowers and Salas 2001, p. 196). Their identification and application in the design and development of adaptive instruction of teams appears to be a natural and promising approach for consideration.

Team training is the activity of preparing a team and its members with necessary skills and knowledge (Salas and Cannon-Bowers 2000) along with the macrocognition needed for team problem solving (Fiore et al. 2010), in performing designated tasks and/or missions. Team training involves the usual systems engineering principles applied to individual team members (‘task-work’) and to the team itself (‘teamwork’) as an entity:

• Analysis to identify objectives, standards, and where appropriate, conditions of instruction;

• Design of instructional treatment(s) to achieve these objectives;

• Development of the instruction;

• Implementation of the instruction;

• Assessment to determine if the instruction produces the targeted objectives;

• Feedback to identify needed improvements.

An important issue in team training concerns the roles and tasks within its hierarchies. For instance and among other matters, team leaders must induce acceptance (or, better, enthusiasm) for the team, its goals, and the leader’s judgment in guiding the team to achieve them. Much of this is accomplished through pre-briefs (setting the stage for a team activity) and interactive post-briefs (facilitated discussion after the activity is completed). Training team leaders to present these briefs was found by Tannenbaum et al. (1998) to significantly improve not only the capabilities and behavior of team leaders, but team performance itself.

The nature of post-briefs has evolved in military team training. Rather than team critiques, these post-briefs evolved into after action reviews intended to facilitate self-examination and discovery by leaders and team members to determine what happened, why it happened, and how, if necessary, the team could improve its performance. These reviews were integrated with the team learning process and intended to help participants learn about the causes and outcomes of their actions so that they could develop strategies, as needed, for improvement. Morrison and Meliza (1999) point out that these reviews were considerably aided, if not made possible, by the development and application of field technologies -- especially the miniaturization of video and audio recording capabilities that increased the degree to which team actions could be recorded and thereby reduced the likelihood of false recollections. In explicating and clarifying what happened in team training, understanding of the team and its behavior became more readily shared by all team members.

A meta-analysis by Burke et al. (2006) examined the relationship between team leadership and behaviorally oriented team performance outcomes. They concluded that, overall, person-focused and task-focused leadership matters. They found that across 19 studies, task-focused leadership behaviors accounted for about 10% of variance in perceptions of team effectiveness and that empowerment behaviors (leader actions to develop self-management and/or self-leadership skills) accounted for nearly 30% of the variance in team learning. They emphasized that leaders need to be trained in task-focused and empowerment behaviors because they both contribute and are needed for teams to be effective. Their research also found that the importance of leadership in teams rises with the interdependencies of team tasks – indicating the importance of leadership in coordinating actions among team members. They noted that the value of team leadership keys on what team leaders do to facilitate team interactions among team members – which includes themselves as members of the team.

A brief summary of objectives for team training adapted from Cannon-Bowers and Bowers (2011) follows:

• Team members should understand the knowledge, skills, and abilities required by all team members

• Team members should be trained on the intra-positional dependencies required by all team members

• Teams should develop interpersonal trust, consult with others, and resolve conflicts either while practicing teamwork or during down time

• Team members should realize the extent to which other team members are performing competently through cross checking of a shared mental model (team cognition)

• Team members should develop active listening skills and an information exchange with other team members

• Individual technical skills can be refined during team training, but they should be acquired before the team is trained

• When teams are geographically dispersed, there is a limit on the level of training that can be achieved in term of team performance

Landon et al. (2016) list six strategies used to train teams:

• Event-Based Training / Scenario-Based Training—teams learn and practice team skills in a simulated operational context.

• Self-Correction Training / Guided Self-Correction Training—teams review past performance, self-diagnose, and create plans for improving.

• Cross-Training—teams are trained on the tasks of different team members in order to foster a shared understanding of how each contributes to the team objectives.

• Stress Training—Teams are taught to recognize the symptoms of stress, and to practice relaxation and other stress-reducing methods

• Team Adaptation and Coordination Training—a case-study method and team guided discussion.

• Team Building—team activities (not necessarily naturalistic to the team) meant to build trust and cohesion.

A meta-analysis by Salas et al. (2008) found that if team training is delivered within the anticipated operational context (e.g., in the field, on the flight line, or in similarly relevant contexts) and if it uses realistic event-based or scenario-based experiences, what is learned is more likely to transfer to operational performance. However, some team capabilities are better trained by simulation experiences keyed to these objectives. For instance, military maneuver may be better trained in virtual simulation than in live field exercises, where maneuvers have to avoid constraints presented by highways, buildings, or shipping lanes that might be ignored during combat or in simulation. On the other hand, leadership skills may be better developed in the rain, snow, or desert heat of field exercises than in the air-conditioned buildings used for virtual and constructive simulation. Overall, teams should practice as research suggests, applying learning and team skills in a context fitted to training objectives, reviewing their performance in that context, and learning to self-diagnose and create plans for improvement (Kozlowski and Ilgen 2006; Salas et al. 2015).

Training for teams must adapt to or even prepare for the self-organization and self-assembly that occur in all teams (Guimera et al. 2005). This preparation seems especially important for the pick-up teams that are frequently and inevitably assembled to perform military operations. Such teams initially lack the ‘transactive memory’ developed by members of established teams. This memory contains the knowledge and skills of specific team members and an awareness of who can perform team tasks under what conditions of motivation and support (Wegner 1986). It allows for division of cognitive labor within a team, permitting the team’s collective knowledge to exceed that of any individual team member.

The model tracing and knowledge tracing techniques described by Anderson et al. (1995), among others, can be used to guide training of team members as well as of teams themselves as entities. Model tracing overlays the steps taken by an individual or a team as it solves a problem onto those taken by an expert or an experienced team to solve the same problem. Used well, model tracing can be used to guide the learning of team members or teams while incorporating novel, but effective, paths to solution in its own repertoire of expertise. It can be used explicitly in the field for training or implicitly through the use of historical examples.

Knowledge tracing explores the deeper, conceptual issues underlying performance. It can begin with a model, often a concept map to access progress by individual team members in acquiring concepts and the inter-relations among them that underlie successful performance as discussed by Hoffman, Shadbolt, Burton, & Klein (1995), Novak & Cañas (2008), and others. These approaches suggest the promise of ITS techniques applied to both individual and team training.

Salas and Cannon-Bowers (2000) provide extensive lists of principles for teamwork, team training, and guidelines in team training that might be incorporated in ITS for teams through the use of shared mental models. Salas et al. (2015) provide an equally extensive referenced list of research-based objectives for attitudes, behaviors, and cognition in team training.

The definition of intelligent tutoring systems (ITS) varies across researchers, designers, and developers. One view is that ITS development is motivated by the empirically evident benefits of human tutoring (e.g., Bloom 1984; Graesser et al. 2011; VanLehn 2011) and a long-standing desire to make these benefits more widely accessible and affordable than those delivered by individual humans (Fletcher 1992, 2009; Corbett 2001). Within GIFT, intelligent tutoring may be viewed as an effort to capture in computer technology the capabilities and best practices of a human instructor who is expert in both one-to-one tutoring and the subject matter being presented.

Another motivation for the development of ITSs grew from the recognition that although computers could be used to teach effectively, it was difficult, if not demonstrably impossible, to anticipate all possible states of the learner and program appropriate instructional responses to them. Here, the idea was instead to enlist computer technology and machine intelligence to identify these states for instruction (Fletcher 2009).

Response to these motivations requires a generative capability, which is a defining characteristic of ITS. Dynamic information structures and mixed-initiative in computer-based tutorial dialogue were intended to generate tutorial interactions in real time (Carbonell 1970; Feurzeig 1969) thereby relieving much of the burden and cost of authoring adaptive, individualized instruction, a possibility that early motivated considerable research and development investment by the Department of Defense (Fletcher 2009; Fletcher and Rockway 1986). Today, with a capability to access almost all human knowledge through the global information infrastructure, ITS capabilities may make learning affordable, universally accessible, and generated on demand -- anytime and anywhere (Dodds and Fletcher 2004; Fletcher et al. 2007).

Mental models of the learner have been found in ITS from its earliest days (Fletcher 1975). Cooke et al. (2000) identified three types of knowledge that exist in these mental models: declarative knowledge, procedural knowledge, and strategic knowledge. Declarative knowledge is defined as “the facts, figures, rules, relations and concepts in a task domain” (p. 153); procedural knowledge is defined as “the steps, procedures, sequences, and actions required for task performance” (p. 153); strategic knowledge is defined as “the overriding task strategies and knowledge of when they apply” (p. 153). For teams, we may wish to add a model of team characteristics – one that includes shared mental models.

The complexity introduced by increased numbers of team members, their dynamically varying roles, the need for them to develop and share mental models adapted to the dynamics of teamwork and team task-work, all in combination with the adaptive tutoring paradigm itself, serve to increase the workload for team training. To reduce this workload, we must seek and develop new methods to automate as much of the authoring process as possible, and we must improve the usability of authoring tools to enable domain experts without software programming or instructional design knowledge to produce effective ITS.

While there are many challenges in moving forward with team training and the development and application of shared mental models, some appear especially significant. A key to establishing effective collaborative learning is the ability to manage the uncertainty and dynamic nature of team interaction and communication. While shared mental models may be incomplete, the limited information within them must be capable of advising an ITS for team training.

One challenge involves the transience of team members – in some situations it may occur rapidly and unexpectedly. It is not unusual for teams in the military and emergency care facilities to be quickly assembled for specific and perhaps urgent tasks. But even in the normal course of events, few teams work as an established entity with the same members over extended periods of time. The understanding among team members that is necessary for communication, coordination, trust-building, development of inter-positional dependencies, and adequate understanding of roles often suffers from this transience (Brown and Palincsar 1989). Any model of team and team member knowledge, skill, and interpersonal dynamics used to guide adaptive training must be dynamically sensitive to transience and its impact on the models shared by team members.

The probability that all trainees understand training material and progress together increases when each member understands his or her roles and responsibilities, and actively participates in the training process (Soller 2001). Shared mental models may be essential tools for ITSs to: promote active participation; encourage the exchange of ideas, information, and perspectives for interaction; provide real-time monitoring of individual and team participation level (e.g., interaction analysis); and manage low participation levels.

Another challenge for an ITS is to understand and manage the relationship between individual and interactive performance and actions – the assembly effect suggested by Roby and Lanzetta (1958), Jones (1974), Salas and Cannon-Bowers (2000), and recently popularized by the concept of “Moneyball” (Lewis 2004). Roles and responsibilities must be defined so the computer-based tutor can aggregate individual actions in a logical, weighted fashion and adapt to the team performance state. The tutor must also understand when to provide feedback to the individual team members based on positive actions (e.g., goals met) or negative actions (e.g., distracting off-task behavior). In addition to timing the feedback, the ITS must decide whether it should be delivered to the team and/or its individual members.

Peer interactions (and their associated mental models) may change enormously as the training domain changes (Brown and Palincsar 1989). Factors that contribute to this volatility and uncertainty in team interactions are ill-defined roles and objectives along with the agility of team members in switching roles between tasks (Burton 1998). Role identification supports social grounding and can create an environment for collaborative learning, problem solving, and effective communication (Soller et al. 1998). This finding suggests that an ITS should develop and maintain a model of team adaptability. As the tasks and objectives become more complex, effective communication within the team becomes more important, and the ability of the team and its members to adapt may lead to a richer learning experience. An ITS must diagnose and redirect incorrect solution paths, divide complex tasks into sub-tasks associated with assigned individual team members, clusters of team members, and the entire team. The idea of a team tutoring system as an observer, manager (decision-maker), and a director is evolving.

While individual behaviors are observable, another challenge in developing shared mental models for ITS arises from unobservable cues to individual states. Stress and anxiety, which limit cognition, may manifest themselves in outwardly observable behavior by novices, but may be more veiled by experts who have learned to set aside external stressors in order to focus on the task at hand (Inzana et al. 1996). Physiological sensors can help meet this challenge. For instance, electro-dermal activity (EDA) has been shown to indicate stress and anxiety (Scheirer et al. 2002). A mechanism is needed to indicate the source of the stress and help the ITS manage the training through guidance (e.g., scaffolding) and optimize the difficulty level of the training experience -- in accord with Yerkes and Dodson’s (1908) inverted U, Vygotsky’s (1978) Zone of Proximal Development, and similar notions for adapting difficulty to the learner.

A rapidly growing challenge is to provide training for geographically distributed teams and develop their associated mental models at a distance. Local teams have been found to learn more than geographically-distributed teams and distributed teams exchange information less effectively than local teams (Andres 2002; Warkentin et al. 1997). However, with sufficient time to develop strong group relationships and become comfortable with the communication environment, dispersed teams can communicate as effectively as co-located teams (Chidambaram 1996). Compressing the time to do so may be particularly important.

Although the military has developed distributed, networked simulation to make team training affordable and accessible (Alluisi 1991), many teams are assembled on short notice under less than ideal circumstances and with limited access to relevant simulation facilities. Mechanisms to rapidly develop shared mental models adjusted to the traits, qualifications, and experience of individual team members either locally or distributed are at a particular premium when newly assembled members must quickly adjust to the individual differences of its members and learn to interact effectively and efficiently in performing assigned tasks or missions. Such mechanisms are also of considerable value in training longer term teams assembled under less rapid and demanding circumstances. GIFT may do much to provide these mechanisms in both cases, providing a way forward for both team assembly and training.

This discussion highlights how team objectives and processes needed to achieve them either align with current GIFT structures or require modification in the current GIFT architecture. Our approach to implementing shared mental models is based on research by Cannon-Bowers & Salas (2001, p. 228) that describes them as knowledge structures held by team members. These knowledge structures vary with the roles and responsibilities of each team member.

GIFT is composed of modules that interact through standardized messages and support tutoring for individual tasks. The current structure of GIFT (version 2016–1) cannot provide collective tutoring sufficiently without significant modification. Therefore, our discussion focuses on how each GIFT module should be modified to support the generation and use of shared mental models in a collaborative or team training context. The discussion of the sensor and learner modules concerns models of individual learners in a team along with their shared models. Discussion of the pedagogical module focuses on instructional strategies applied to teams. Discussion of the domain module focuses on team concepts to be learned, assessment methods for shared states, and decision mechanisms and tactical actions by the tutor. The collective interactions between modules and with each individual learner is defined by an adaptive learning effect model (LEM; Sottilare 2012; Fletcher and Sottilare 2013, Sottilare et al. 2013) which in turn measures the effect of adaptations by the ITS to individual learning, performance, retention, and ability to transfer skills from training environments to work environments.

GIFT Sensor and Learner Modules in a Collective Tutoring Context

The GIFT Sensor Module provides interfaces for behavioral and physiological sensors. It accepts raw sensor data and processes this data as needed to determine individual states (e.g., workload, engagement), which it transfers to the Learner Module. To realize a collective tutoring capability, GIFT might use these individual sensor feeds and other available data (e.g., historical performance and state data from record stores) to develop shared mental models that represent the collective purpose, behavior, and function of each team during training. Individual values and traits like grit (resilience) and conscientiousness could also be included as measures of collective purpose in a shared mental state.

A model of collective purpose might represent an understanding of individual team member goals and the member’s commitment to their attainment (Pettigrew 1979). In GIFT, learning objectives are represented as “concepts”, which are defined by the ITS author and linked via metadata to the selection and presentation of appropriate course content and feedback by the tutor. Individual learner attributes (e.g., prior knowledge, personality, emotional state) also influence tutor interactions (e.g., feedback or changes in the content difficulty level). GIFT already supports both a non-hierarchical and a hierarchical view of concepts, which allow individual roles and responsibilities to be layered as subordinate or otherwise supporting collective objectives.

What are not currently defined in the GIFT architecture are the roles of team members. A method must be developed to link responsibilities to individual team members or subgroups of teams and to define assessments for each responsibility. Many military training systems use a convention of task/conditions/standards to represent what must be accomplished, under what circumstances, and to what minimum expectation. For individual training, GIFT defines concepts and leaves to the author’s discretion what task(s) must be done under what conditions to demonstrate proficiency or mastery of that concept. Standards are also left to the author to be checked by testing learner knowledge or a performance in a practice scenario. Users of GIFT will expect the same flexibility in the architecture for team tutoring, but the goal and associated tasks, conditions, and measures of success must be specifically defined so the tutor can effectively adapt content and provide feedback.

Currently in GIFT, individuals participate in adaptive instructional experiences where concepts (learning objectives) are set for one person, the experience is tailored for one person, learning and performance are assessed for one person, and the ITS interacts with one person in an effort to optimize learning and performance. To support the tutoring of teams, GIFT will need to assess the decisions and interdependent actions of multiple people along with the concepts that include both individual and team level concepts. Table 1 shows how a single team member -- in this case performing the role of Center on an American football team -- must share knowledge about the actions and states of other team members in order to complete assigned responsibilities.

The activities required to exercise both individual and team skills in interdependent teams differ from the current GIFT concepts. A GIFT-based ITS will require knowledge of what is expected from each relevant team member and the team as a whole. These expectations may be quantified in interdependent responsibilities (expected actions) in GIFT. The ITS must allocate roles and responsibilities to the team so its concept goals can be assessed in terms of learning and performance. For this reason, the GIFT authoring tools will be modified to allow authors to define team concepts, conditions, scenarios, individual roles, responsibilities, and assessments as shown in the Table 1 example of a football team attempting to convert a long yardage play into a first down.

Table 1 provides a breakout of the Center’s responsibilities with associated measures of assessment. Each responsibility requires knowledge and skill related to an individual concept (learning objective) associated with measures of assessment. The table focuses on the team hierarchy assuming that individual skills are sufficient to support team coordination and cohesion.

Table 1 suggests how individual states contribute to collective states (e.g., shared mental models) and what might be trained and assessed in preparing team members to perform team tasks and missions. However, collective learning and performance in a team composed of interdependent members (e.g., centers, guards, quarterbacks) is more than the sum of its parts – differences in individuals may fill gaps in capabilities while similarities in individuals may create synergies that produce capabilities greater than their sum. Models of collective learning and performance and their associated shared mental models should be separate from individual models but influenced by them. The Learner Module uses data received from the Sensor Module, performance and knowledge assessments from the domain module, and demographic data from a record store to determine each learner’s cognitive, non-cognitive, and competency states. A Team Model will also require inputs (e.g., individual behaviors) to assess team states (e.g., performance, cohesion, or coaching/leadership).

Team and individual state information will be sent to the Pedagogical Module where current states are compared to expected team and individual learner states and matched with successful strategies and best practices of human tutors to identify and select an appropriate, or even optimal, instructional strategy.

The changes needed in GIFT to support collective training include an array of team models as antecedents of learning and performance that may be derived from behavioral markers of team members and their interactions such as the “7 C’s” of communication, coordination, conflict, cooperation, cognition, coaching, and conditions (Salas et al. 2015; Sottilare et al. 2015) listed earlier. Over time, GIFT will use data analytic techniques to evolve a multi-scale understanding of how to use individual learner, team, and team of team data to accurately identify team states and then optimize selection and implementation of effective team training strategies.

GIFT Pedagogical Module in a Collective Tutoring Context

The Pedagogical Module is a domain-independent recommender engine within GIFT that selects generalized instructional strategies (e.g., pump, prompt, hint, reflect, direct, or support) that determine next steps in the tutoring process. It uses individual learner state, performance data, and prior knowledge models to determine the content, order, and flow of instruction. It recommends instructional strategies to guide the Domain Module’s selection of domain-dependent tactics. The GIFT strategy selection mechanism is the engine that manages adaptive pedagogy (eMAP) based on an extensive meta-analysis of the training literature that examined the effect size of various interventions across an array of individual task domains (Goldberg et al. 2012).

To support adaptive instruction of collective tasks, the Pedagogical Module might use ITS techniques for knowledge and performance model tracing to recommend a simulation with the type of scenarios intended to develop a number of strategic, general team capabilities such as adaptability, grit, situational awareness, problem solving, communication, or coordination. Initially, the shared mental modeling in GIFT, pedagogical decision trees, and agent-based policies would be extended to use collective behavioral markers and shared states cross-referenced with research literature to select collective strategies. Later, reinforcement learning techniques (e.g., Markov Decision Processes) could be applied to analyze team data linking shared mental models, collective instructional strategies, and learning/performance outcomes in order to optimize the selection of learning strategy.

GIFT Domain Module in a Collective Tutoring Context

The GIFT Domain Module is domain-dependent. It defines and structures the instructional domain’s declarative and procedural knowledge requirements. It translates the Pedagogical Module’s strategic recommendations into domain-specific instructional tactics that determine the content, order, pace, and feedback alternatives for presentation to the learner. For instance, it assesses and predicts learner progress toward instructional objectives (i.e. at, above, or below expectation). Similar to the Pedagogical Module, the Domain Module can also learn and improve upon the selection of effective instructional tactics through reinforcement learning mechanisms.

The largest degree of change for team training is likely to be in the design of the GIFT domain module and specifically in the Domain Knowledge File (DKF). A DKF is an XML file that contains the information needed to execute a single course. The DKF file is read by the Domain module and used by both the Domain and Pedagogical modules. Each module takes specific actions on sections of the file. Below is a sample of a DKF structure for an individual task with subordinate concepts and conditions for playing center in American football (Table 2).

Concepts have a user defined node name, which also appears in scoring and any after-action review, and also has a unique node identifier, that locates this node in the domain. Conditions map to Java classes that are responsible for assessments (e.g., condition 1.1.2 is true). Each condition has one or more associated measures usually tied to learner behaviors/actions or states (e.g., performance). For psychomotor tasks like playing Center on an American Football team, learner states might include physical states (e.g., fatigue or unsteady blocking position) derived from sensor data.

Augmenting GIFT for collective training will require changes in the architecture to allow hierarchical DKFs. Most collective training applications will only require one DKF for each team member and one for the team. More complex team within team, collective training may require one for each team member and one for each group or cluster where multiple teams are training. The basic structure of the DKF may remain the same for individuals, but team DKFs will need to expand to include roles and responsibilities, as shown by the Table 1 football example, and any additional data required by the team model to foster tutoring decisions.

Collective Tutoring in GIFT

In short, GIFT modules and their functions may perform in team training just as they do in individual training. However, there remain issues that are unique to team training and team cognition and that require attention in designing shared mental models for intelligent training of teams. GIFT’s modular, distributed architecture allows for asynchronous interaction and simultaneous tutoring of individuals. This architecture allows for individual feedback in a team context where each learner’s tutor communicates changes in an individual’s state to the other tutoring agents within GIFT, thereby supporting both team level models and individual learner models. In other words, a distributed architecture may be needed to support team tutoring where the tutor for learner ‘A’ continually develops and maintains a model for learner ‘A’ to be shared with other team member models to assist and augment cross training and reciprocal understanding of teamwork. It also shares the team states (e.g., performance) across computers/devices so each learner on the team has a fully informed model of the team as its performance and capabilities are developing.

Team performance models, as noted above, will include observations of team behavior as it progresses toward one or more training objectives, including performance conditions and standards. Team assessment techniques are crucial in developing a clear understanding of team performance. Salas et al. (2009) argued that performance measurement works best when it captures and considers performance from multiple sources, is tightly coupled to the action needed, uses validated expert models to assess the performance, directly supports learning, and provides real-time corrective feedback. Individual and team assessments analyze factors including (1) when each learner is ready to take an action, (2) proper timing for the action, and (3) value of the action to the team task and/or mission.

Team competency models may be used to predict performance within a domain since they are based on previous related experiences and associated levels of success. Cumulative team competency models are needed for the ITS to choose training scenarios and set expectations for performance. Team learning objectives, individual state information, the interdependencies of team members performing the training task, and the interdependence of workflows within and between teams can be used to inform a Team Cognitive State model for assessing mental workload, engagement, and learning progress to determine team and individual strategies and tactics.

While not specifically targeted in this paper, modeling of team affect is necessary for optimizing cognitive performance. Behavioral observations may provide a context for understanding individual and team affect, but more evidence is needed. For instance, affect is a moderator of cognition (Van den Bossche et al. 2006). Problem solving and decision-making become more difficult as affect becomes more extreme. A shared mental model of affect can inform the ITS to take action to guide, mediate, or challenge team members to get back on track.

Much ITS activity consists of problem solving exercises in which the progress of learners toward problem solutions can be explicitly and objectively observed (Kulik and Fletcher 2016). ITS designers employ procedural models that lay out the actions an expert might use to solve a problem in the subject domain. Based on a learner’s actions in solving a problem, an ITS can thereby infer what the learner knows. Bayesian techniques, for example, are currently prominent in such inferences. They turn cause and effect on its head, allowing us to estimate the probability of a given cause (e.g., a component of the knowledge model) that brought about the observed effect. These estimates improve as experience with additional learners build up, allowing the learning system itself to learn.

ITS tracing activities (mapping actions taken onto procedure models and inferred knowledge onto knowledge models) lend themselves well to team training, much of which involves exercises and problem solving, which provide practice with feedback for teams and team tutors. We can apply ITS modeling and model tracing processes to teams in two ways – to identify and assess the declarative (including strategic) knowledge of individual team members and to do the same for the team itself as a collective. Empirical study on both as they apply to the training of teams and accessing their progress toward targeted instructional objectives may do much to develop ITS capabilities for team training and to inform tactical and operational decision making.

One issue concerns what must be shared among the mental models of team members. Given the research on transactive memory and its evident contributions to successful team performance, it appears that not all team members must possess all team knowledge (e.g., Yoo and Kanawattanachai 2001). This is not a surprising conclusion, but research on team and team member cognition (or mental models) may identify more specifically what must be shared, the priorities for sharing it, and, what knowledge can be taught and what can only be done in team training, which is typically given limited time and resources.

Another matter for research concerns the tutorial dialogues that are the eventual target for ITS development. These dialogues seem likely to remain at the individual level, but computer-based tutors could have full access to team exercise instrumentation data, provided by the GIFT Sensor Module, individual history and other team-relevant information provided by the GIFT Learner Model, training objectives held in the Pedagogical Model, and domain specific data obtainable from the Domain Module. These dialogues could provide private, individualized feedback to team members. Capabilities to do this are well within the state of the art. Eventually these dialogues might become genuine facilitated discussions with an individual. A research task with fairly rapid return may be to base ITS dialogue capabilities on team exercise data and use it to provide feedback to individual team members. Doing so will extract more value from training exercises than is now possible because of more accurate and comprehensive access to data and the ability to interact privately with each team member as an individual.

In accord with Olmstead (1992), among others, the amount and type of communication is considered a significant behavioral indicator of team trust and cooperation for our notional shared mental model. Roles (e.g., leader, follower, domain expert) should also be considered a factor in that roles may determine communications and expected communications. For example, a team leader would be expected to communicate mission intent (goals), clarify roles, and direct activities as needed. The ITS managing the shared mental model of cognition would be expected to monitor communication to determine how it met or did not meet expectations.

Finally, Sottilare et al. (2011) discussed specific and separate team state models that may be informed by individual learner models and historical team performance data that might be gathered from relevant operational and training environments. In addition to shared mental models, they note that team social climate and team member affect also determine team cognition and performance and that problem solving and decision-making become more difficult as affect becomes more intense. This article has focused on team cognitive factors, but, as noted, non-cognitive factors, such as morale, confidence, and physiological state, could also influence the development of team models, cognition, and performance. Their effects should also be considered in the development of shared mental models, team capabilities and the development of ITS for team training.

Teams are just one type of collective, which also includes crews, organizational units, and groups of any size. The discussion here might well refer to tutoring of any collective. This discussion has focused on teams because they are most characteristic of the collectives trainers have in mind, and training for teams appears to be the most amenable area for early research into the application of intelligent tutoring in training collectives, but wider applications are possible.

Other questions, as well as other lines of research may well occur to readers. As Salas and Cannon-Bowers (2000) suggest, there may be much in the ITS world of value if it is applied to team training. This chapter has focused on mental models and their sharing in team cognition, but other approaches may also return significant value. Cannon-Bowers and Salas (2001) point out a number of fundamental questions to be addressed by empirical study of shared mental models including determinations of what must be shared, what we mean by sharing, how we should measure sharing, and what outcomes and value can be expected if we are successful.

Finally, our suggestions begin to fill out the GIFT framework with necessary empirical specifics, but we caution that the approach discussed herein is not just plug-and-play. To be effective, our approach requires large amounts of data. This is a multi-scale problem space that requires means to capture data, develop measures for assessments, and apply assessments that drive interactions by the tutor. We hope this is just the beginning of a journey to use ITS in solving a large set of problems. We hope our comments here will encourage others to apply the emerging power of ITSs to the essential and ubiquitous activity of teams.