Keywords

1 Introduction

“The robot would be unable to understand children because it would lack reasoning skills that require cognitive, social, and emotional intelligences. Teaching requires that teachers understand what students already know and use that to help them make new connections. It also requires a relationship between student and teacher, one that allows and encourages risk-taking. A teacher’s job is to balance a push for new knowledge and a stay for students to gain mastery. This takes a lot of intuition and personal judgment.”

-Educator

The above quote was taken from a survey we conducted on educators’ opinions on a robot teaching assistant and mirrors the thoughts of many of the respondents [1]. To address this concern, it is crucial to (1) understand the state of mathematics education in the US, particularly for at risk youth; (2) assess the potential of Human-Robot Interaction (HRI) for education; and (3) consider the Artificial Intelligence (AI) needed to develop a socially adaptive robot tutor. This three-pronged approach (education, HRI, and AI, respectively) lays the foundation of this research.

1.1 Education

Mathematics is among the core academic subjects identified by the US Department of Education [2]. Math competence in early education leads to career and college readiness as it prepares students for undergraduate courses in college [3] and plays a critical part in the competency for workers in the technical workforce and the nation’s economic development [3]. Although math proficiency is extremely important, many students are not excelling in the field [4].

Tutoring.

Tutoring is one approach to help students perform better in mathematics as it is often used to assist students who may show weaknesses in academic areas. Tutoring is a supplemental aid in the learning process that can further enhance a student’s academic ability [5]. Benjamin Bloom found that students who receive one-on-one tutoring outperformed students who receive traditional classroom instruction by two standard deviations (two-sigma problem) [6].

A tutoring interaction is comprised of an academic component and social component [7, 8]. Academically, tutors provide immediate and specific feedback. Socially, tutors provide positive reinforcement and guidance [8]. Together, these components are critical for success in tutoring [5]. Further, this academic and social interaction fosters student engagement [5].

Engagement.

In education, student engagement influences student motivation and progress in learning. The term student engagement encompasses the student’s attention, curiosity, interest, optimism, and passion when learning. There are many facets of engagement as it relates to education including intellectual engagement and emotional engagement [9].

Intellectual engagement focuses on a student’s cognitive state during learning [9]. Teaching strategies are often employed for the maximum benefit of intellectual engagement. Two effective techniques that are encouraged are guided practice and progress monitoring [9]. Emotional engagement describes a student’s affective state during learning. Student emotions impact cognition and positive emotions stimulate attention [10]. It is important to organize emotions in a way that makes emotion groupings meaningful. Scherer et al. labels emotions by valence and control/power [11].

1.2 Human-Robot Interaction

HRI is the field of study that involves understanding, designing, and evaluating robotic systems that communicate with humans [12]. HRI is applied in areas in which it is necessary for the robot to interact with the user [13, 14]. This is exactly the case in a tutoring scenario where a social interaction between the tutor (the robot) and the student is necessary for effective learning to occur [15].

Robots and Education. Several research studies have investigated the use of robots for education. These studies have shown that social robots are useful supplemental tools for education. Yun and colleagues documented a study where students were instructed via a robot tele-operated by a teacher, which led to learning gains for students [16]. Another study investigated the conceptual design of an educational robot that engaged students in a lesson about historical ancient cultures [17]. Though the robot’s sociability has been shown to contribute to student achievement, little has been done to illustrate the specific aspects of the robot that facilitate learning and retention [18, 19].

Social robots have also been widely used to support mathematics education. Brown and Howard used verbal cues to minimize idle time and decrease boredom during a mathematics test [39]. In another study, researchers used personalization to students while playing an adaptive arithmetic game with a robot [20]. Ramachandran and colleagues used a social robot that aided students while practicing fractions [21]. Socially responsive feedback (i.e., task-related feedback, motivational support), was effective in a robot learning companion that helped students practice mathematics problems [22].

Robots have also demonstrated positive trends among student perception and engagement [23]. One study documented how a robot’s perceived sociability increased from the pre- to post-questionnaires during a mathematics tutoring session [24]. Howley et al. documented that students were more willing to ask the robot questions over a human tutor in most situations due to varying perceptions of the robot’s social role during a tutoring session [25]. Kanda et al. concluded that the social behavior of the robot aided in facilitating a better relationship with the student and increased the student’s social acceptance of the robot during a mathematics lesson [26]. The implementation of adaptive robots is an important topic in HRI; however, AI can be applied to develop robots that adapt and respond to a student’s needs.

1.3 Artificial Intelligence

AI is the field of study that involves synthesizing and analyzing computational agents that can act intelligently. An intelligent agent can make decisions about its actions based on factors such as goals/values, prior knowledge, observations, past experiences, and the environment [27].

An effective human tutor adapts to the student (tutee) by gathering information about the student (e.g., capabilities, motivations, etc.) and tailoring real time instruction to meet the learning needs of the student [28]. This adaptability makes AI a probable approach to intelligent tutoring systems. Agents rely on an array of inputs such as student’s prior knowledge, common student errors, or facial expressions which can be used to conduct activities (i.e., assess student knowledge and provide relevant feedback). Figure 1 shows a sample agent system as a tutor.

Fig. 1.
figure 1

Agent system (tutor)

Full size image

Previous work has focused on adaptive tutoring and the robot’s [or computer’s] response once information is inferred. In some cases, social responses are reactions to a student’s state to aid in academic success [29,30,31,32].

Decision Trees.

A decision tree is a model used for classifying data and is one of the most effective methods used for supervised classification learning. A tree is built per its training data, which it uses to make classifications. The internal nodes in a decision tree represent the tree’s features and its classes are represented by the tree’s leaves [33]. Figure 2 shows a sample decision tree that uses four predictors (outlook, temperature, humidity, and wind) to determine a decision (yes, no) to play golf.

Fig. 2.
figure 2

Sample decision tree [34]

Full size image

1.4 Summary

Due to the need for student enrichment in the math, and the benefits of using robots for education, socially adaptive robots are ideal as a teaching tool for mathematics education. Social robots are not only capable of delivering mathematics content, but they are also capable of socially interacting with students to promote an enriching educational experience. However, how do we develop a socially adaptive robot with reasoning skills and an intuition about the student’s emotional state?

2 Impetus of Research

2.1 Problem

Educators have expressed that to best serve students, a robot tutor must possess reasoning skills and the robot must be capable of having an intuition about the student’s emotional state [1]. To date, there is a lack of literature that describes implementation of a socially adaptive robot tutor that uses a decision tree model to predict student emotion and performance for practicing multiplication via effective teaching techniques (i.e., guided practice and progress monitoring).

2.2 Research Goal

The goal of this research was to use a decision tree model to develop a socially adaptive robot tutor that predicted and responded to student emotion and performance to actively engage students in mathematics education. To assess the research goal (i.e., effectiveness of a robot’s ability to educate and engage students), this study addressed the following research questions:

  1. [Q1]

    How well can a decision tree model classify a student’s emotion and performance?

  2. [Q2]

    How well can a socially adaptive robot tutor engage 5th grade students to practice multiplication?

    1. (a)

      How do students perform academically by studying with a socially adaptive robot tutor?

    2. (b)

      How do students respond emotionally by studying with a socially adaptive robot tutor?

  3. [Q3]

    What social perceptions do students have of a socially adaptive robot tutor while practicing multiplication?

To address these research questions, we conducted a study in which students interacted with a robot during multiple tutoring sessions. We recorded information (such as delay in answer) that was needed to help the robot make predictions about the student. We collected information about each student’s mathematics performance before, during, and after the tutoring sessions as well as information about each student’s emotional states throughout the study. We also gathered information about the student’s perceptions and opinions of the robot tutor.

3 Methodology

3.1 Platform

We used the NAO humanoid robot (see Fig. 3) as the robot tutor named Ms. An (Meeting Students’ Academic Needs). The NAO humanoid robot is an ideal platform for delivering education because of its multimodal capabilities such as speech and gesture. The NAO stands 58 cm tall. It has 25 degrees of freedom, 2 cameras, various touch sensors, and 4 microphones. The robot is also capable of voice and vision recognition.

Fig. 3.
figure 3

NAO humanoid robot [35]

Full size image

3.2 Lesson

State Standards.

The multiplication tutoring session covered problems that addressed the South Carolina state standards:

  • (4th grade) 4.NSBT.5 Multiply up to a four-digit number by a one-digit number and multiply a two-digit number by a two-digit number using strategies based on place value and the properties of operations.

  • (5th grade) 5.NSBT.5 Fluently multiply multi-digit whole numbers using strategies to include a standard algorithm [36].

Content.

The content of the lessons spanned across the different ways in which multiplication can be described through equal groups, area arrays, and comparison [37]. Students practiced multiplication with problems that included multiplying whole numbers by up to four digits and one digit and multiplying two-digit numbers by two-digit numbers. To ensure record of a wholistic multiplication experience, students solved problems with different combinations of multiplication question and answer types. For example, Fig. 4 shows a session question that was given as a context question type and pictorial (equal groups) answer type.

Fig. 4.
figure 4

Example session question

Full size image

3.3 HRI Study Design

To analyze the effectiveness of the adaptive robot, we conducted a between-subjects study with two conditions: (A) with a non-adaptive robot (control group); and (B) with a socially adaptive robot (adaptive group). Table 1 shows a comparison of robot traits and behaviors for each condition.

Table 1. Adaptive robot versus non-adaptive robot
Full size table

In the adaptive robot condition, Ms. An predicted the student’s emotion and performance and proactively determined next actions before presenting a question to be solved. For emotion classification, Ms. An used social responses that corresponded to each student’s emotional state (happy, angry, sad, surprised, neutral) before asking a question. For performance, if Ms. An predicted that the student would answer the upcoming question correctly, it proceeded with progress monitoring; however, if Ms. An predicted that the student would answer the upcoming question incorrectly, the robot proceeded with guided practice. In the non-adaptive condition, Ms. An behaved in a neutral emotional state and completed only progress monitoring activities (i.e., asked mathematics questions to be solved without any intervention) despite the student’s affective state or competency.

Student-Robot Interaction.

In addition to gestures and movements, Ms. An communicated with the students verbally using speech and visually through the tablet (see Fig. 5). Ms. An performed actions such as reading each multiplication question, prompting the students at various points during the lesson, and giving feedback on answer choices (verbal communication). These actions corresponded with the question and activity display on the tablet (visual communication). Students could press buttons on the touch screen interface to select answer choices and enter values using a keypad. Ms. An received that data from the tablet and responded accordingly.

Fig. 5.
figure 5

Student-robot communication diagram

Full size image

Participants.

Twenty 4th and 5th grade students were included in the study (10 males, 10 females), ranging from 9–12 years old (M = 9.95, SD = 0.84). Participants were recruited from Blenheim/Elementary Middle School in rural Blenheim, SC. Of those participants, 50% identified themselves as Black/African American; 30%, White/Caucasian, 5%, other; and 5% opted not to report race/ethnicity.

Participants completed a Technology Experience Profile that measured their familiarity with and use of different technologies. While the students rated an overall familiarity (M = 3.59, SD = 1.12) with technology, their experience with robots, specifically, was low (M = 2.95, SD = 1.28). The top technologies which the student reported using on at least a weekly basis (e.g., M = 4.0 or higher) were video games, tablet, smart board, smart phone, music player, and social media. The least used technologies (M = 3.0 or below) were webcam, electronic book reader, LCD projector, student response systems, robots, and a camera. See Fig. 5.

Eleven students were assigned to the control group and nine students were assigned to adaptive group. To ensure the groups were equally split by student performance, we used the pre-test 3 scores to assign students to each condition. Students in the control group had a 32% (SD = 14) average score and students in the adaptive group had a 28% (SD = 9) average score.

3.4 Procedure

Students engaged in one-on-one tutoring sessions to practice multiplication per the South Carolina 4th and 5th grade mathematics state standards. Mainly due to time and resource constraints, most education interventions using robotics are short-term interventions (some being as short as one interaction) [38]. To have a longer intervention and multiple interactions, students worked with Ms. An for two sessions, having one session a week for two weeks. Sessions lasted approximately 30–45 min.

Prior to the study session, students completed a student demographic form, technology experience profile, and multiplication pre-test. Then, each student was asked to sit in a small room and the students worked at a desk with the robot. The student began each session by answering the emotions questionnaire. Next, they interacted with the robot. For both the adaptive and non-adaptive groups, the robot acted as a tutor and completed progress monitoring activities. The robot asked students multiplication questions. Each question was displayed on the tablet and students answered questions via the tablet interface. Contrary to the non-adaptive robot, the adaptive robot employed proactive behaviors and executed those behaviors when needed (see Sects. 3.2 and 3.3). Students completed the emotions questionnaire again, halfway through the study session. Once the tutoring session was complete, students completed the emotions questionnaire a final time.

After all sessions were completed, students completed a final session on solving multiplication using the partial products technique. In this session, students began with guided practice then concluded with progress monitoring.

At the end of all three sessions, students completed a post-test, RPI questionnaire, and interview. Students were given a retention test after all students completed the study. Figure 6 details the study procedure for each student.

Fig. 6.
figure 6

Study procedure

Full size image

4 Results

4.1 Data Analysis

Unless otherwise noted, alpha was set at .05 for all statistical tests. Due to the small sample size in each group, we report this data with guarded generalizations. We indicate all data that are statistically significant with an asterisk (*).

4.2 Decision Tree Model

Research question 1 (How well can a decision tree model classify a student’s emotion and performance?) addressed the accuracy of a decision tree model. To evaluate the robot’s ability to make classifications for emotion and performance and to better understand where misclassifications could occur: (1) for emotion, we compared the robot’s prediction for each student’s emotion to the student’s self-reported emotions throughout the session and (2) for performance, we compared the robot’s prediction for each student’s performance to the student’s actual performance.

We used data from a previous study to train our decision tree, which is a popular technique known as transfer learning. Transfer learning in artificial intelligence is a technique in which the data used for a training set to solve one problem is applied as a training set to solve a similar problem [39]. While using this technique is common, it may have contributed to the low prediction accuracy for both training models to the new models.

These comparisons for emotion classifications are shown in the confusion matrix in Table 2. The values along the diagonal of the matrices are the success rates for predictions.

Table 2. Confusion matrix for each emotion
Full size table

The results show that the model is not accurate for each individual emotion (as expected from results of the training set described in Sect. 3.2). Despite the students exhibiting other emotions, the robot only predicted neutral and sad emotions. Happy was most commonly classified as neutral (51%). Happy was also misclassified as sad 40% out of all sad classifications. Surprise was also misclassified as neutral (21%) and sad (40%).

The comparisons for performance classification are shown in the confusion matrix in Table 3. The values along the diagonal of the matrix are the success rates for predictions.

Table 3. Confusion Matrix for performance
Full size table

Incorrect performance was classified correctly at a higher rate than correct performance. However, the classifications were correct a little over half the time, which is only slightly better than choosing randomly.

4.3 Engagement

Research question 2 (How well can a socially adaptive robot tutor engage 5th grade students to practice multiplication?) emphasized student engagement. To consider two aspects of engagement, research question 2 was comprised of two sub-questions. To address question 2a, how do students perform academically by studying with a socially adaptive robot tutor, we report average learning gains and percent correct by answer type. To address question 2b, how do students respond emotionally by studying with a socially adaptive robot tutor, we report the frequency of emotions exhibited throughout the study sessions.

Learning Gains.

The difference in pre-test and post-test scores is a measure of each participant’s learning gain during the study. We also calculated the difference in session 1 and session 2 scores to measure each participant’s learning gains. To allow for a reliable analysis for our between-subjects design, we calculated the normalized learning gain in each group [40].

Pre-/post-test 1.

Pre-/post-test 1 was a test on students’ ability to identify the different ways to represent multiplication problems. Figure 7 shows the normalized average learning gains for pre-/post-test 1 for each condition. We conducted Wilcoxon Signed Rank tests to compare pre- to post-test scores in each condition. The adaptive conditions did show a statistically significant (z = −2.06, p < .05) improvement from pre- (M = 3.67, SD = 1.32) to post-test (M = 5.44, SD = 2.83) scores. Therefore, the adaptive robot did, in fact, promote learning gains in the students’ ability to identify the different ways to represent multiplication problems. The control condition did not have a significant change (z = −0.239, p = .81) from pre- (M = 4.63, SD = 2.01) to post-test (M = 4.63, SD = 2.06) scores. Therefore, the control condition did not yield learning gains in this skill.

Fig. 7.
figure 7

Pre-/post-test 1 average learning gains

Full size image

Mann-Whitney U test was used to compare the learning gains between conditions. While the adaptive group had higher learning gains from pre- to post-test1 (M = 0.44, SD = 0.22) than the control group (M = −0.15, SD = 0.67), this difference between groups was not statistically significant (z = −1.62, p = .10). It is important to note that although this is a promising trend, the there is no significant difference likely due to the variance in the control group being higher, and due to the small sample size.

Pre-/post-test 2.

Pre-/post-test 2 was a test on students’ ability to correctly solve multiplication problems. Figure 8 shows the normalized average learning gains for pre-/post-test 2 for each condition. We conducted Wilcoxon Signed Rank tests to compare pre- to post-test scores in each condition. The adaptive conditions did not show a statistically significant (z = −1.13, p = .26) improvement from pre- (M = 0.28, SD = 0.09) to post-test (M = 0.39, SD = 0.29) scores. The control condition also did not have a significant change (z = −1.66, p = .10) from pre- (M = 0.32, SD = 0.15) to post-test (M = 0.46, SD = 0.21) scores. Therefore, the control condition also did not yield learning gains in this skill.

Fig. 8.
figure 8

Pre-/post-test 2 average learning gains3

Full size image

Mann-Whitney U test was used to compare the learning gains between conditions. Figure 8 shows the normalized average learning gains for pre-/post-test 2 for each condition. There was not a decrease in learning gains for either group in test 2; thus, the adaptive session did not negatively impact the students. There was not a statistically significant (via Mann-Whitney U test) difference in learning gains between the two conditions for test 2 (z = −.34, p = .73).

Frequency of Emotions.

We assumed that students were equally likely to select any of the 5 emotions (happy, angry, sad, surprised, neutral), and calculated a Pearson’s Goodness of Fit Chi Square. The chi square for both the control (X2 = 37.77) and the adaptive (X2 = 42.62) were significant (p < .001), suggesting that the distribution of reported emotions were not evenly reported. Students significantly reported happiness more often than other emotions. Students were more likely to feel surprised in the control condition. This could be because they had less feedback/coaching on how they were doing. The emotions sadness and anger were not commonly reported.

4.4 Robot Sociability

Research question 3 (What social perceptions do students have of a socially adaptive robot tutor while practicing multiplication?) focuses on students’ perceptions of a robot tutor. To address research question 3, we used the results of the RPI questionnaire. We analyzed the RPI by each individual item.

We then conducted Wilcoxon Signed Rank tests to conduct a within-group comparison for each individual questionnaire item – comparing the mean to 3.00 (neutral).

The significant questionnaire items are listed in Table 4. As depicted in this table, more items from the facilitated learning construct were statistically significant in the adaptive condition. The robot was interesting was statistically significant for both groups. More items from the credible construct were statistically significant in the control condition. The robot seemed knowledgeable and the robot seemed like a teacher were statistically significant for both groups. No items in the human-like construct were statistically significant. Lastly, more items from the engagement construct were statistically significant in the control condition. The robot was motivating was statistically significant for the adaptive group, which directly addressed the engaging persona factor (how well the agent motivated the student).

Table 4. Statistically significant RPI items by condition and agent factors
Full size table

5 Results

This study investigated the use of a socially adaptive robot tutor to engage students in mathematics education. Often, it is difficult to get students to engage in mathematics education [41]. While technology is not a full solution, it can make significant contributions to better engage students in mathematics education [42]. This study was important because it offered strategies to better engage students (emotionally and academically) in mathematics education.

Although our decision tree models were not very predictive, the results gave answers to our current questions and clarity for future directions. Our adaptive strategies to engage students academically were effective. All students enjoyed working with the robot and we did not see a difference in emotional engagement across the two groups. Our adaptive strategies made students think more deeply about their work and focus more. This higher order thinking is preferred in education as it a cognitive process that demonstrates deeper understanding of the academic material [43].

Not only does this study tell us more about education and AI, but it also tells us how to improve the methodology for educational HRI in rural areas. Novelty likely played an important role in this study on a rural population due to lack of exposure for students. Future studies that include urban students may yield different results.

This study offered insight for developing a socially adaptive robot tutor to engage students academically and emotionally while practicing multiplication. Results from this study will inform the human-robot interaction (HRI) and artificial intelligence (AI) communities on best practices and techniques within the scope of this work.