1. Introduction
Chatbots have recently gained attention for being used in education and learning. They are often used as assistant tools for supporting students’ learning activities [
1]. These technologies influence education in several aspects such as advisory, research and development, assessment, administration, and teaching and learning [
2]. For example, providing a personalized learning environment to students according to their performance and behavior, increasing the students’ engagement in learning, using to convey schools’ information, providing universities’ administrative supports such as orientation and recruitment, and research and development to know if a chatbot can assist students to achieve a certain goal. Chatbots are also proven to be convenient tools for the users, i.e., students. They can use chatbots ubiquitously to inquiry without the limitation of office hours. The majority of educational chatbots are designed and created to assist undergraduate students and adult learners [
3,
4,
5,
6]. However, only a few chatbot systems, such as BookBuddy and ScratchThai [
7,
8], are developed to assist young children in their learning. Therefore, there is a need to explore and study a chatbot system for primary and secondary education level.
There are several research gaps and challenges in the development of appropriate chatbots for the primary education level. Firstly, young students have a limitation on chatbot interactions. For example, text-based chatbots are inappropriate for children due to their low ability to correctly type and ask academic questions, causing chatbots to misinterpret and respond inaccurately. Secondly, teachers are often limited in their ability to develop and handle desired tasks with chatbots. For example, teachers may want to make adjustments to a chatbot corresponding to their lesson plans. Adjustments are essential for customizing a chatbot for different schools, students, and teachers; it impacts a chatbot’s adoption in the classroom from the perspectives of both students and teachers.
To address these gaps, this paper proposes a novel ChatBlock framework that helps teachers to create and customize chatbots effortlessly to be suited to targeted classroom environments without computer programming skills. Our proposed authoring tool simplifies the creation of a chatbot by addressing the challenges of the learning curve and preparation time for teachers. Furthermore, to improve young students’ learning with chatbots, we propose to use block-based interactions by dragging and dropping blocks instead of typing. Specifically, this work introduces block-based conversational language to help children formulate sentences. The novelty of our approach lies in the integration of the block-based interaction which is inspired by the block-based programming to facilitate the communication of young learners with a chatbot. Finally, two case studies are exemplified and presented. The following presents three research questions in this work.
RQ1: How can we design and develop a chatbot to assist teachers and young students in a classroom?
RQ2: How can ChatBlock support teachers and young students in a classroom?
RQ3: How can ChatBlock effectively support learning activities of young students?
This paper is organized as follows.
Section 2 reviews chatbot technologies, chatbot-building platforms, and educational chatbots.
Section 3 introduces the ChatBlock framework, its methodology, design and development.
Section 4 demonstrates ChatBlock using two case studies, and evaluates its practicality and usability.
Section 5 discusses the results.
Section 6 concludes and draws future directions.
2. Background and Related Work
Chatbot adoption has expanded to various domains including agriculture, healthcare, e-commerce, education, etc. [
9]. A study in [
10] shows that developing chatbots often requires less effort and time compared to the development of mobile applications for the same purposes. This is one of the reasons that there is an increasing number on the development of chatbots instead of launching new mobile applications to provide services such as querying for product information [
11], querying for COVID-19 information [
12], obtaining user input for healthcare screening [
13], music recommendation [
14], and nursing training courses [
15]. This also emphasizes the benefits of applying chatbots in wide-spread applications. Based on the hype cycle for artificial intelligence 2020 [
16], chatbots are expected to increase in adoption rates in the next two to five years and are currently the most popular AI applications that are widely adopted in many businesses.
2.1. Types of Chatbots
According to [
17], chatbots can be categorized based on four aspects: domain knowledge, services, goals, and response generative methods. The domain knowledge indicates the scope of knowledge in a chatbot. An open domain means that a chatbot can respond to any topic whereas a closed domain chatbot limits the scope to a specific one.
The services provided by chatbots consist of three types. Firstly, an interpersonal chatbot is a bot that focuses on providing a certain service and tasks but does not consider user information. Secondly, an intrapersonal chatbot focuses on providing tasks related to the personal interests and activities of a user. Finally, an inter-agent chatbot is a bot that involves IoT and focuses on providing services by communicating with other agents (bots).
The goals of chatbots can be divided into informative, chat-based, and task-based. The goal of being informative is to provide the requested information, especially from fixed information sources. The chat-based goal is more flexible by providing a correct response to any sentence of users. On the other hand, the task-based goal focuses on performing a certain task that is requested and confirmed by a user.
The response generative methods are methods that chatbots use to process a user’s input and to generate response messages [
18], classified into the following:
Rule-based Method: focuses on answering questions based on a set of predefined rules and predefined question intents. The chatbots only identify keywords from an input sentence and match them with the defined rules to create responses. Thus, the important task of developing rule-based chatbots is to define expected inputs from users and a list of responses for each input pattern [
19]. Artificial Intelligence Markup Language (AIML—
www.aiml.foundation/index.html, accessed on 10 September 2024) is an open standard XML dialect and is the most famous language for rule-based conversational agent development. AIML was introduced by [
20] when A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) was developed. This approach is used for chatbot development such as [
3,
21,
22].
Retrieval-based Method: retrieves a matching response to an input message from its response databases [
23] to generate a correct response. Querying repositories is often carried out using APIs [
24]. However, retrieval-based chatbots have no capability to generate new answers. Therefore, the effectiveness of these chatbots depends on the size of the databases that chatbots retrieve.
Generative Chatbots: can provide better and proper responses compared to other methods by considering current and previous input messages and employing machine learning and deep learning techniques [
18]. Machine translation techniques are typically used in generative models. For example, employing sequence-to-sequence models in order to generate responses during the conversation. Compared to retrieval-based models, generative models can provide responses that do not appear in the corpus [
25].
Based on [
17], our proposed ChatBlock is a closed domain chatbot with the scope of knowledge limited to the teacher’s contents. Regarding the service aspect, ChatBlock is an interpersonal chatbot that focuses on providing requested objects to students. According to the goal aspect, ChatBlock is an informative chatbot. For the response generative method, ChatBlock is designed as a retrieval-based method to provide responses by using APIs. These four aspects of chatbots could be customized according to the desired goals and purposes.
2.2. Chatbot Development Platform
This section introduces four well-known bot building platforms, where non-technical users can create chatbots without handling text input processing tasks, provided by the leading companies.
Google Dialogflow: The key concepts of building chatbots with DialogFlow are Intents and Contexts. Intents are groups of pre-defined user inputs and the action that should be taken by the bot. Contexts are represented by string values that indicate different requests which may have different meanings. DialogFlow classifies a user input to determine whether it is matched with some pre-defined intents. If it does not match with a known intent, they will fall to “Default Fallback intent”.
Language Understanding (LUIS): Microsoft’s LUIS is part of the Microsoft Bot framework. It provides the natural language processing intelligence of the Bot Builder and allows bots that are built with Microsoft’s Bot Builder to draw from a deep library of NLP knowledge.
Wit.ai: Wit.ai of Facebook is an open source chatbot framework. The key concept of building a chatbot with Wit.ai is the use of stories. Each story represents an example of a possible conversation pattern. Since a complicated chatbot requires a lot of pre-defined intents, Wit.ai groups these intents into stories. Wit.ai is part of the Facebook Platform, and hence is suitable for developing chatbots for Facebook Messenger.
Watson Assistant: The Watson Assistant tool, released by IBM in 2016, is a bot-building platform with a web-based interface for defining intents and entities. It provides features to import and export intents and user utterance phrases. We can also list all entities and synonyms for a specific entity-value.
2.3. Chatbots in Education
Several recent studies reported a wider adoption of chatbots in education [
2,
26,
27,
28]. Their adoption mostly focused on language courses, followed by engineering, computing, and science with a few attention on arts and mathematics. Various learning strategies were adopted in the existing chatbot with a guided learning strategy becoming the most widely used technique [
27].
According to the examining of sample types studied research works on the use of AI chatbots in education [
29], the most frequently studied is undergraduate level (35.9%) where the secondary 9–12, the primary 6–8, and the primary 1–5 are 17.9%, 10.3%, and 2.6%, respectively. Most of chatbots for education studied in [
30] are developed for university students aged 18 and above. This indicates the lack of studies at the primary school level (1–5), which is the main focus of our work. There exist a few chatbot systems in educational environments developed to assist young children, e.g., BookBuddy [
7] and ScratchThai [
8,
31]. BookBuddy is a chatbot that requires children to type their messages to communicate the bot where ScratchThai offers predefined chat statements for children, allowing them to select responses instead of typing.
The survey on the educators’ attitudes to chatbots [
32] indicates the unwillingness to adopt chatbots in class since most teachers were influenced by negative experiences with chatbots used in customer support. The study in [
29] also reinforces the hypothesis that educators faced difficulties in developing AI chatbots. Therefore, it is necessary to make chatbots as simple as possible to potentially change this existing perception.
Concerning the benefits of chatbots in education, a study in [
33] shows that learning with chatbots can increase learners’ memory retention and boost learning achievement. In addition, an experiment by [
34] reveals that learners feel more comfortable when chatting with an automatic answering program than when talking with friends and asking teachers. Lastly, a chatbot can immediately provide support and answers to any number of questions. There are several ways that institutions and the education industries can gain benefits from chatbots. We classify their applications in the education domain into two types based on their tasks as follows.
Information Support Chatbot: A chatbot that mainly provides information in some scopes such as course information or institution information. DINA Chatbot [
35] is one of the informative support chatbots that is responsible for providing admission information to applicants. The FIT-EBot [
24] is a chatbot for administrative support in the faculty of Information Technology of Ho Chi Minh City University of Science (FIT-HCMUS). It has responsibilities for providing academic information such as education events, student and staff recruitments, course details, and program announcements. According to [
4], the authors developed a chatbot to assist high school students in learning general knowledge subjects. In addition, MoocBuddy [
36] can assist anyone in discovering news about MOOCs and individual learners to find MOOC courses.
Learning Support Chatbot: A chatbot that assists learners in their learning. For example, it can serve as a tutor, capable of answering questions about lesson contents, and giving some learning materials when students request. Moreover, it can give a quiz to students. These chatbots have potential benefits in improving student learning performance. For example, MathBot [
37] provides math practice questions, feedback, and suggestions to students. In [
38], the chatbot is integrated into an English learning application, and functions as a virtual personal assistant that can remind learners to learn and practice test questions. Its goal is to motivate and increase learner’s engagement.
2.4. Modern Techniques for Supporting Educational Experiences
In addition to chatbots, other emerging technologies are proposed for enhancing engagement and supporting learning experiences. Gamification has used to be a tool for engaging students by incorporating game elements into learning processes, e.g., points, leaderboards, missions, etc. The authors in [
39] provide the study of four aspects used as the gamification solutions in education, i.e., achievement, progress, immersion, social, and non-digital elements. The results indicate that educational gamification typically focuses on aspects that highlight achievement and progress, while social and immersive aspects are less frequently used. Furthermore, the Metaverse in education has also gained attention, especially at the university level [
40]. It enables learners to participate in immersive interactive experiences with low learning costs and risks, unrestricted time and space, and other benefits. This approach allows students to engage in simulations and explorations that transcend the limits of traditional classrooms. For example, they can witness and investigate flight accidents in a virtual aircraft control room. These technologies promote more interactive, student-focused learning, though they come with different challenges for educators.
3. Proposed Framework: ChatBlock
3.1. Overview
Our proposed ChatBlock framework is designed to address the difficulties of constructing a tutoring chatbot as a learning support system for students as well as a learning management system for teachers, especially at the primary education level. Specifically, ChatBlock aims to support teachers to effortlessly build a customized chatbot for their classrooms where the chatbot can support students’ learning with various tasks such as asking questions and taking quizzes. Since ChatBlock can support both students and teachers, the framework consists of two main components: the ChatBlock Tutor module for students, and the ChatBlock Classroom module for teachers. The roles of each component are described as follows:
ChatBlock Tutor is a block-based conversational interface designed for young students as its target users. The interface is simplified for young learners by avoiding typing and speaking. To make a conversation with the chatbot, students can use the provided blocks to generate a sequence of sentences. The following are the functionalities provided to young students in order to use ChatBlock Tutor:
- -
Authentication: Students can authenticate to a classroom by using a username and a password.
- -
Ask a question: Students can ask questions with the bot by using different question blocks.
- -
Request a learning object: Students can request learning objects such as exercises, knowledge sheets, and videos of a lesson (or a topic). Note that the learning objects can be provided in various formats, including PDFs, images, and videos, to support diverse teaching and learning needs.
- -
Take a quiz: ChatBlock system can generate quizzes to students if quiz information is provided in a classroom by a teacher. Students can take a quiz in this module by using quiz blocks.
ChatBlock Classroom is a web-based user interface for managing classroom contents designed for teachers as its target users. ChatBlock Classroom supports teachers to easily create a desired ChatBlock Tutor and manage specific content such as lessons, teaching materials, and quizzes. The following are supporting functionalities of ChatBlock Classroom for teachers:
- -
Create new classrooms: Teachers can register new classrooms and create new ChatBlock Tutors.
- -
Manage lessons: Teachers can add/edit/delete lessons, topics, questions, learning objects, and quizzes for any classroom.
- -
Manage students: Teachers can add/delete students to their classroom.
3.2. System Architecture
Figure 1 depicts the architecture of ChatBlock consisting of three layers: Data and Knowledge, Service, and User Interface layers.
The Data and Knowledge layer maintains a database and a knowledge base for the ChatBlock Tutor and ChatBlock Classroom modules. The ways to retrieve data from this layer depends on an interacting module. That is, given a student’s request from the ChatBlock Tutor, the chatbot can search and retrieve knowledge from this layer to respond to the request (e.g., a question, a learning object, and a quiz). For ChatBlock Classroom, teachers can add/edit/remove classrooms, lessons, and students to/from the database. This layer is implemented using MongoDB, an open source document-oriented database program classified as NoSQL.
The Service layer contains two important services: Conversation Management and Knowledge API. The Conversation Management is responsible for processing an input (a message) received from ChatBlock Tutor by using the intent classification process and the entity (keyword) extraction process. This service is connected to ChatBlock Tutor via Dialogflow Gateway (
https://cloud.google.com/dialogflow/es/docs/basics, accessed on 10 September 2024). Dialogflow is a natural language understanding platform for designing and integrating a conversational user interface into applications, devices, etc. We use Dialogflow for managing conversations with the chatbot, especially identifying a suitable response message to the input intent by querying through the Knowledge API. The Knowledge API enables the Conversation Management to search throughout the Data and Knowledge layer. Additionally, the API has the responsibility to connect with ChatBlock Classroom to handle the required functionalities of teachers such as creating a new classroom and adding new lessons. NodeJS is employed to develop the Knowledge API.
The User Interface layer consists of two user interfaces for students and teachers separately. First, ChatBlock Tutor is an interface that allows students to interact with a block-based chatbot by dragging and dropping blocks. Students can use the provided blocks to generate a sequence of input sentences. Instead of manually typing, they use their block-based sentences to chat with the chatbot. The created sentences represent an input to the Conversation Management service. This interface is built by using NodeJS for the back-end whereas the front-end (the block-based interface) is built by using VueJS (vuejs.org) and Google Blockly (developers.google.com/blockly) package. Note that Google Blockly is an open-source software for creating block-based visual programming languages. Blockly makes writing code easier by connecting visual blocks together for generating code in various languages, e.g., JavaScript, Python, etc. Second, ChatBlock Classroom is an interface for teachers to manage classroom’ lessons and students. ChatBlock Classroom is developed by NodeJS for the back-end and Angular (
https://angular.dev/, accessed on 10 September 2024) for the front end.
3.3. Conceptual Domain Design
Based on the primary education level in Thailand, a subject can consist of many lessons and a lesson can also consist of many topics. Both lesson and topic can have Q&As about specific contents, supportive materials, and quizzes. Thus, we design the structure of subject-related knowledge in ChatBlock as shown in
Figure 2.
3.4. Block-Based Language Design
Considering the structure of subject-related knowledge (i.e., lesson and topic), we aim to define its structure with “block-based language programming concepts”. Block-based programming languages such as Scratch [
41] use draggable jigsaw-like blocks designed to formulate a coded language without syntax errors.
This approach is widely used in education to teach computational thinking and programming skills to beginners and beyond, particularly for young learners [
41,
42]. The blocks are organized, color-coded, and include a graphical presentation to simplify the learning process and enhance engagement. Therefore, our work is inspired by the concept of block-based programming and aims to adopt this approach to introduce a block-based conversational interface.
Our ChatBlock framework employs a block-based concept to support the formulation of a sentence. Each formulated sentence plays a central role in using ChatBlock since target users (i.e., children) have to use such sentences for making a conversation (or a request) with the chatbot. We utilize this concept to develop a block-based conversational interface for primary education level students. Therefore, various types of blocks are defined to represent the necessary elements in subject-related knowledge.
The block-based conversational interface is specified by means of a variant of Extended Backus-Naur Form (EBNF) [
43,
44]. Here, we fix the set of lesson names in classrooms, the set of topics of each lesson, the set of questions for any lesson or topic, the set of learning objects for any lesson or topic, and the set of quizzes for any lesson or topic as terminals. Alternatives are separated by vertical bars (|). Components that can occur one or more are suffixed with a plus symbol. Components that can occur any number of times (including zero) are suffixed with an asterisk (∗). The following shows the proposed production rules used in ChatBlock.
The defined block-based language represents the rules of valid block connections by a user. Students can connect these blocks to create sentences instead of typing and speaking. Basically, they are required to understand the grammar of blocks. However, the block-based conversational interface is designed as jigsaw-like so that they can easily recognize the grammar by just merely playing with the interface. That is, children can drag/drop/delete/connect each provided block. This shows how the grammar is user-friendly and simple enough for children’s understanding. Therefore, this approach answers RQ1 on designing a chatbot to assist young students.
We provide an example of using blocks to create sentences according to the defined syntax.
Figure 3 shows the connected blocks representing sentences for studying the “Natural Phenomenon” lesson. In this scenario, a teacher prepares materials for a Natural Phenomenon lesson which includes Q&As, videos, and quizzes about the Natural Phenomenon. Additionally, the teacher provides a topic called storm which is a type of Natural Phenomenon. The storm topic contains Q&As and videos about the storm. The connected blocks are further divided into two essential parts.
Firstly, the outward block is a Lesson Statement block which is connected to the Lesson block called Natural Phenomenon. This lesson statement consists of one question block, one learning object block, one Topic Statement block, and one quiz block; all occur inside the Natural Phenomenon lesson statement.
Secondly, the
Topic Statement block resides in the
Lesson Statement.
Figure 3 shows an example of using a topic statement block which is connected to the
Topic block called Storm. This topic statement consists of one question block and one learning object block.
There are various possible scenarios for using the aforementioned lesson statement and topic statement. For instance,
Q&A with chatbot: Children can click at a question block to ask a question from the bot; for example, by clicking on the question resided in the Natural Phenomenon lesson (“What is Natural Phenomenon?”) and the Storm topic (“Dangers of the storm”).
Request quizzes from chatbot: Children can click at a quiz block, i.e., Natural Phenomenon Quiz in order to take a quiz.
Request videos from chatbot: Children can request other materials prepared by a teacher. For example, they can click on a learning object block called “video Hurricane” to obtain a video.
The proposed block-based conversational interface is designed to familiarize children with a meta-syntax notation like EBNF by learning to connect blocks in ChatBlock. This can encourage young students to learn and understand block programming such as Scratch Programming Language, as well as implicitly improve their computational thinking skills.
3.5. System Design and Development
Figure 4 depicts the ChatBlock Tutor’s user interface comprising five parts: (1) a send button, (2) a logout button, (3) block categories, (4) a workspace area, and (5) a chat area. The block categories area includes various types of blocks, as illustrated in
Figure 2, such as questions and quizzes. Students can select their desired blocks and assemble them in the workspace area. Once all the blocks are connected, they can click the send message button. The results are then displayed in the chat area on the left.
On the other hand, ChatBlock Classroom’s user interface has three main pages, namely (1) a home page, (2) a student page, and (3) a quiz page. The home page shows a list of lessons for a classroom. The student page displays the table of student data in the classroom. The quiz page shows the collection of quizzes designed for the classroom.
Figure 5 depicts the UI of the ChatBlock Classroom’s home page. Our developed ChatBlock Classroom answers RQ1 on designing a chatbot to assist teachers.
4. Case Studies and Preliminary Evaluation
This section discusses two case studies, ScienceChat and ScratchChat, to exemplify potential applications of our proposed ChatBlock framework related to different type of courses, i.e., a content-based course and a practice-based course. Specifically, the science course (ScienceChat) is a content-based course that focuses on content exploration. Hence, students need to review the lessons to acquire knowledge related to science-related content. On the other hand, the programming course (ScratchChat) is a practice-based course that focuses on practices and exercises. Therefore, teachers usually assign many exercises to students.
4.1. ScienceChat: Applying ChatBlock to Learning Science
ScienceChat is a chatbot built to support the science class of a primary school in Thailand in terms of knowledge reviewing and exercises. At the primary school level, science learning is more focused on exploring the surroundings with curiosity. Therefore, science teachers require a chatbot to have Q&A conversations with students to help them explore and understand the lessons. To support the curiosity of students, a chatbot shall provide supportive learning objects such as infographics and videos. Nonetheless, the science teachers preferred a low-learning-curve chatbot creator that could support them to transform physical materials into a digital platform. To pursue this objective, the teachers adopted ChatBlock to create ScienceChat, which contains the desired online materials according to their science curricula.
To support science learning, ScienceChat provides seven types of blocks in the area of block categories as shown in
Figure 6 (left). Teachers can set up materials for the chatbot based on these blocks. Specifically, they can set up questions, learning objects, and quizzes based on topics and lessons. The figure illustrates how ScienceChat is used to create an interactive lesson on “Natural Phenomena” with the three provided topics, namely tsunami, storm, and earthquake. Each topic contains its related question blocks and learning object blocks.
Using ChatBlock, students can have the flexibility to connect various blocks in order to create sentences. Once they have formulated any statements, they can send them all and receive the relevant contents (responses) from the chatbot. As for teachers, they can group students and assign different topics to each group for self-studying, group projects, homework, etc. ScienceChat can also support studying in the classroom and encourage self-learning by students at home by interacting with the bot in addition to reading books. This case study demonstrates how ChatBlock can support teachers and young students in a classroom, addressing RQ2, and how it can support learning activities, addressing RQ3.
Figure 6 illustrates a case in which a student wants to learn about the Natural Phenomenon lesson from the chatbot. From the figure, there are three goals as follows:
Ask a question “What is Natural Phenomenon?”.
Explore “Tsunami in Thailand”.
Watch a video about tsunamis.
The workspace shows the connected blocks for creating sentences to communicate with ScienceChat. To achieve these goals, the student can follow the following steps for joining the blocks:
Select a lesson: From the area of block category, the student selects “Lesson” to choose a lesson statement block and choose a lesson block called “Natural Phenomenon”; after that, the student joins two blocks together (see
Figure 7).
Select a question in a lesson: From the “Lesson Question” block, the student selects a question block “What is Natural Phenomenon?”.
Select a topic: From the “Topic” block, the student selects a topic statement block, choose a topic block called “Tsunami”, and joins the two blocks together.
Select a question in a topic: From the “Topic Question” category, the student selects a question block about “Tsunami in Thailand”.
Select a learning object in a topic: From the “Topic Learning Object” category, the student selects a learning object, i.e., a video about Tsunami.
When the student clicks on the “Send Message” button, the formulated statements (i.e., lesson statement and topic statement) are sent to the chatbot. After that, the chatbot provides the corresponding responses as depicted in
Figure 6 (right):
The answer to “What is Natural Phenomenon”.
The answer to “Tsunami in Thailand”.
The video about “Tsunami”.
Since ScienceChat is developed to support a content-based course, the teachers can define quizzes about a certain lesson or a topic, and assign them to the students.
Figure 8 shows an example that a student takes a quiz from ScienceChat.
4.2. ScratchChat: Applying ChatBlock to Scratch Coding Practice
By employment of ChatBlock, ScratchChat was built to support a computational thinking and programming course. The target users are grade 5 students in Thailand. Based on Thailand’s Basic Education Core Curriculum (revised 2017—
http://academic.obec.go.th/images/document/1572317446_d_1.pdf, accessed on 10 September 2024), ScratchChat’s content focuses on learning block programming (block-based coding) using Scratch Language [
45]. ScratchChat does not only support knowledge reviewing, but also supports practice-based learning.
Like ScienceChat, ScratchChat provides certain lessons and topics for learning and reviewing the computational thinking course by using Scratch. As shown in
Figure 9, a student can retrieve the lesson entitled “Motion Block Category (Movement)”. From the “Lesson Question” category, the student selects a question block to ask about the motion block category. In the topic statement, the student can further select to learn more about the “Move” block. Also, one question about the Move block is added to the sentence by using the “Topic Question” category.
On the other hand, ScratchChat offers programming exercises to support the practice-based learning style. Teachers can use the “learning object blocks” to provide hyperlinks to the prepared Scratch exercises for each lesson instead of providing image and video objects as used in ScienceChat. This case study demonstrates how ChatBlock can effectively support the learning activities of young students, addressing RQ3.
In
Figure 10, the student requests an exercise entitled “EX101 Ask and Say?”. From the “Lesson Learning Object” category, the student connects EX101 to the lesson statement. This exercise is prepared for students to practice sequence programming skills. Once the student clicks on “Send Message”, the chatbot will provide a link for the exercise instruction and another link to work on this exercise with Scratch in the chat area. Apart from exercises, teachers can also prepare the frequently asked questions (FAQs) for each exercise to support students.
4.3. Evaluation and Results
The evaluations were conducted separately with teachers and students in order to answer the research questions RQ2 and RQ3.
4.3.1. Evaluation with Teacher
We carried out a preliminary evaluation with six primary school teachers (two females and four males with 25 years old in average) using the focus group interview method. Each participant taught different classes and different levels of students. The purpose of this evaluation is to gather feedback on the users’ experience of chatbots built from the ChatBlock framework and determine the usability of bots.
Each participant used ChatBlock Classroom for preparing a lesson and used ChatBlock Tutor for learning with the chatbot. The authors performed the following five steps to collect feedback from the case study of ScratchChat as follows:
Introducing and demonstrating two user interfaces of ScratchChat: ChatBlock Tutor and ChatBlock classroom (30 min).
Requesting the participants to use ChatBlock Classroom to create their classrooms and prepare a lesson (45 min).
Requesting the participants to use ChatBlock Tutor as a student to learn the prepared lesson. Each participant is required to make a conversation with the chatbot by connecting the different types of blocks (30 min).
Using the focus group discussion to interview the participants for collecting feedback based on four criteria: ChatBlock in teacher opinions, student engagement, role of ChatBlock in the classroom, and additional feedback. The interview questions can be found in the
Appendix A section below (60 min).
Requesting the participants to fill in the System Usability Scale (SUS) [
46] based questionnaires to measure the usability of ChatBlock (no time limit). Note that SUS is a simple but reliable tool for measuring the usability and satisfaction of systems, products, and services based on user feedback.
4.3.2. Result of Evaluation with Teachers
Important findings obtained from the interviews are discussed based on four criteria:
ChatBlock from Teacher’s Perspective: Some features in ChatBlock Classroom take time to understand their usage steps, e.g., editing and deleting lessons.
ChatBlock for Student Engagement: The participants agreed that ChatBlock Tutor which is a block-based chatbot was easier to use than a text-based chatbot. From the teachers’ viewpoint, the predefined blocks could help young students to understand the scope of a lesson and a topic to be learned as well as it could support young kids in making a conversation with the chatbot. Additionally, the interaction with jigsaw-like blocks could engage and encourage students to use ChatBlock. They agreed that ChatBlock can enhance students’ focus on learning compared with traditional learning methods such as paper-based quizzes and printed knowledge sheets.
Role of the ChatBlock in the classroom: ChatBlock could play an important role as an additional teaching aid in the classroom and could support kids in self-studying and practicing outside the classroom.
Additional feedback: Apart from supporting learning in any subject, a computational thinking teacher agreed that a block-based chatbot may help primary school students develop computational thinking skills because they have to consider the color, shape, and order of blocks to create block-based sentences. However, chatbots built from the proposed ChatBlock were less flexible since the students could ask only by using pre-defined questions.
Regarding the result of measuring ChatBlock’s usability, the participants answered survey questions based on the System Usability Scale (SUS) [
46]. According to the questionnaires, the average SUS score from six teachers is 69.583, which falls in
OK rating, since the score is above the standard average of SUS score (i.e., 68). We have also learned from the participants that ChatBlock Classroom may require some efforts of teachers to learn and become familiar with beforehand because it offers various types of knowledge in the proposed framework (e.g., questions, learning objects, and quizzes) and knowledge can be further classified into lessons and topics. Therefore, it might take time for a new user to understand these features to prepare suitable lessons and topics. Thus, the demo video of how to use the ChatBlock Classroom’s interface is required to facilitate the teachers in order to mitigate the difficulty.
4.3.3. Evaluation with Students
We also carried out an evaluation by using the case study of ScratchChat with 12 grade 5 students (nine females and three males) as the target users (cf.
Section 4.2). These participating students have experience of learning block-based programming, i.e., Scratch. The purpose of this evaluation is to investigate whether ChatBlock as a block-based chatbot can support young students for their learning purposes, e.g., asking questions, requesting learning materials, and requesting quizzes. Each participant uses ChatBlock Tutor for learning with the chatbot. We have performed the following four steps to collect feedback from the case study of ScratchChat as follows:
Introducing the user interfaces of ScratchChat for students, i.e., covering the interfaces of ChatBlock Tutor only (10 min).
Demonstrating the methods of using block-based interfaces to interact with ScratchChat for certain purposes such as asking questions and requesting learning materials (30 min).
Requesting the participants to use ChatBlock Tutor to learn the prepared lesson. Each participant is required to make a conversation with the chatbot by connecting the different types of blocks (50 min).
Requesting the participants to answer a questionnaire including six questions (30 min).
4.3.4. Result of Evaluation with Students
Table 1 shows the interview questions for 12 students and their corresponding average scores. Each participant was requested to rate each question, ranging from 1 (absolutely disagree) to 5 (absolutely agree). These questions can be divided based on three measurement criteria: (1) student’s attention, (2) comparisons with conventional learning, and (3) computational thinking skill development. Firstly, Questions 1 to 3 are used to measure the student’s attention to the provided chatbot. The total average of student’s attention is 3.41 out of 5. Secondly, Questions 4 to 5 are used to measure the preference for learning with and without the chatbot. The total average score is 3.84. Finally, Question 6 has an average score of 4, indicating the potential for computational thinking skill development while using our chatbot.
5. Discussion
The study results demonstrated improvement in the mutual interactions of the learning environment when compared to conventional methods of learning. ChatBlock could assist both teachers and students in teaching and learning, respectively.
The proposed ChatBlock framework aligns with the principles of inquiry-based learning (IBL) [
47,
48] by fostering active exploration, self-directed learning, and learner-centered education for students. By using ChatBlock to construct sentences, students are engaged in the process of inquiry with the chatbot where the learning process is driven by their curiosity. Additionally, the ChatBlock Classroom empowers teachers to create customizable content in various formats including PDFs, images, and videos.
The feedback and results of questionnaires point out the following takeaways:
ChatBlock helps teachers as a teaching aid to prepare various materials for young students to self-study at their available time without the need for teachers. While the usability of ChatBlock is acceptable, the user interface could be further improved to reduce the effort of users, especially teachers. For example, a participant said that “
most features of ChatBlock are easy to use but there are a few functions that I have to learn how to use them such as edit and delete a lesson”. Compared to existing research that emphasize the difficulties of educators in creating chatbots [
29,
32], our findings highlight the importance of developing a user-friendly chatbot authoring tool for educators.
ChatBlock helps improve students’ engagement, as indicated by high scores in the evaluation criteria: (1) students’ attention and (2) the willingness to continue using it compared with the traditional learning styles. For example, a participant said that “
interacting with these colored blocks can help to engage students”. While students found that ChatBlock is useful (as evidenced by the high scores for Questions 1 and 3–6 in
Table 1), they mentioned the time required to learn how to use the system for the first time use (see the score of 3 for Question 2 in
Table 1). As aforementioned, BookBuddy [
7] and ScratchThai [
8,
31] require typing messages and using predefined chat statements, respectively. The positive feedback highlights the usefulness of our block-based method.
The findings from both the teachers and the questionnaire show that using ChatBlock can keep students practicing and improve their computational thinking skills since ChatBlock is built based on the block programming concept. For example, a participant said that “the block-based chatbot can support primary school students to develop computational thinking skills because they have to consider the color, shape, and order of blocks while constructing sequences of questions”.
The current version of ChatBlock has a limitation of the contents that must be prepared in advance before use (e.g., Q&As, learning materials, quizzes, etc.). For example, students can ask for a limited number of questions only. Therefore, the next version of ChatBlock could also provide additional APIs for free-form querying and answering.
6. Conclusions
This paper proposes the ChatBlock framework which is a block-based chatbot that assists the learning of young students and teaching of teachers at a primary education level. To answer RQ1, we described the design and development of ChatBlock consisting of ChatBlock Tutor and ChatBlock Classroom modules for students and teachers, respectively. For students, ChatBlock Tutor offers a block-based interface to support sentence formulations that enable chatting with the bot without accurate typing and spelling skills. Formulated sentences can be in the form of questions, requests for learning objects, and requests for quizzes. Similarly, teachers can customize the content of the chatbot according to their desired lessons with those three sentence types.
To answer RQ2 and RQ3 and to demonstrate the benefits of employing ChatBlock, we applied it to build ScienceChat (for knowledge reviewing and exercises) and ScratchChat (for practice-based learning). The preliminary evaluation with six primary school teachers shows that involving a chatbot can engage young students to learn and the block-based chatbot is easier for children compared with the traditional chatbot. ChatBlock demonstrates the potential to enhance learning activities and increase the engagement of young students. Survey results indicate that students are also interested in using it across other subjects. They particularly enjoy using ChatBlock for requesting exercises and conducting information searches through inquiry. Additionally, the survey confirms that using the block-based chatbot resembles block programming, enabling students to practice and improve their computational thinking skills while interacting with the chatbot. However, the experimenting teachers faced some difficulties in using ChatBlock Classroom for the first time since they were unfamiliar with the designed interfaces to prepare three types of materials (questions, learning objects, and quizzes) for students. This issue can be mitigated by providing a demo video and a tutoring session. As for future directions, we plan to improve the usability of ChatBlock Classroom and prepare more lessons for conducting a comprehensive evaluation with young students. Additionally, ChatBlock Classroom still requires content preparation in advance which can be time-intensive for educators. To address this issue, we aim to develop a shared library of reusable resources and an option to upload and import existing content. We also plan to apply ChatBlock to further experiment with a larger scale of participants including different age groups and skill levels (e.g., high school students and high school teachers).
Author Contributions
Conceptualization, C.A. and H.C.; methodology, C.A. and H.C.; software, H.C.; validation, W.J. and T.R.; formal analysis, T.R.; investigation, W.J.; writing—original draft preparation, C.A. and H.C.; writing—review and editing, W.J., T.R. and C.A.; visualization, W.J. and T.R. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Data available on request due to privacy.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A
Appendix A.1. Interview Questions for Teachers
As a student/teacher, is the developed system easy to use?
Can the system be your assistant? In which aspects?
How much is its deployment feasibility in the classroom?
In your perspective, can our chatbot stimulate student’s learning motivation?
Can a block-based chatbot improve student’s computational thinking skills?
What are the advantages, disadvantages and recommendations for ChatBlock?
What do you think if we provide open classrooms that students can access and learn without any authentications?
What functions that you want to have for the next version?
Appendix A.2. Interview Questions for Students
The following questions are grouped by measurement criteria. Answers for each question are ranged from 1 (absolutely disagree) to 5 (absolutely agree).
Appendix A.2.1. Student Engagement
Appendix A.2.2. Comparison Between Learning with and Without the Chatbot
I considerably want to learn through this chatbot in other subjects.
I want to use this chatbot to request for exercises more than the conventional practice, e.g., receiving printed exercises.
I want to search the information by using Q&A rather than reading books.
Appendix A.2.3. Computational Thinking Skill Development
References
- Ferrell, O.; Ferrell, L. Technology challenges and opportunities facing marketing education. Mark. Educ. Rev. 2020, 30, 3–14. [Google Scholar] [CrossRef]
- Okonkwo, C.W.; Ade-Ibijola, A. Chatbots applications in education: A systematic review. Comput. Educ. Artif. Intell. 2021, 2, 100033. [Google Scholar] [CrossRef]
- Kazi, H.; Chowdhry, B.S.; Memon, Z. MedChatBot: An UMLS based chatbot for medical students. Int. J. Comput. Appl. 2012, 55, 1–5. [Google Scholar] [CrossRef]
- Dutta, D. Developing an Intelligent Chat-Bot Tool to Assist High School Students for Learning General Knowledge Subjects; Technical Report; Georgia Institute of Technology: Atlanta, GA, USA, 2017. [Google Scholar]
- Karra, R.; Lasfar, A. Enhancing Education System with a Q&A Chatbot: A Case Based on Open edX Platform. In Proceedings of the International Conference on Digital Technologies and Applications, Fez, Morocco, 29–30 January 2021; pp. 655–662. [Google Scholar]
- Chang, C.Y.; Hwang, G.J.; Gau, M.L. Promoting students’ learning achievement and self-efficacy: A mobile chatbot approach for nursing training. Br. J. Educ. Technol. 2022, 53, 171–188. [Google Scholar] [CrossRef]
- Ruan, S.; Willis, A.; Xu, Q.; Davis, G.M.; Jiang, L.; Brunskill, E.; Landay, J.A. Bookbuddy: Turning digital materials into interactive foreign language lessons through a voice chatbot. In Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale, Chicago, IL, USA, 24–25 June 2019; pp. 1–4. [Google Scholar]
- Katchapakirin, K.; Anutariya, C. An architectural design of scratchthai: A conversational agent for computational thinking development using scratch. In Proceedings of the 10th International Conference on Advances in Information Technology, Bangkok, Thailand, 10–13 December 2018; pp. 1–7. [Google Scholar]
- Maroengsit, W.; Piyakulpinyo, T.; Phonyiam, K.; Pongnumkul, S.; Chaovalit, P.; Theeramunkong, T. A survey on evaluation methods for chatbots. In Proceedings of the 2019 7th International Conference on Information and Education Technology, Aizu-Wakamatsu, Japan, 29–31 March 2019; pp. 111–119. [Google Scholar]
- Jearanaiwongkul, W.; Anutariya, C.; Reddy, K.T. Mobile Applications vs. Chat-based Applications: A Comparative Study based on Academic Library Domain. In Proceedings of the 2021 18th International Joint Conference on Computer Science and Software Engineering (JCSSE), Lampang, Thailand, 30 June–2 July 2021; pp. 1–6. [Google Scholar]
- Chen, S.; Li, C.; Ji, F.; Zhou, W.; Chen, H. Driven answer generation for product-related questions in e-commerce. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, Melbourne, Australia, 11–15 February 2019; pp. 411–419. [Google Scholar]
- Herriman, M.; Meer, E.; Rosin, R.; Lee, V.; Washington, V.; Volpp, K.G. Asked and answered: Building a chatbot to address COVID-19-related concerns. NEJM Catal. Innov. Care Deliv. 2020, 1. [Google Scholar]
- Judson, T.J.; Odisho, A.Y.; Young, J.J.; Bigazzi, O.; Steuer, D.; Gonzales, R.; Neinstein, A.B. Implementation of a digital chatbot to screen health system employees during the COVID-19 pandemic. J. Am. Med. Inform. Assoc. 2020, 27, 1450–1455. [Google Scholar] [CrossRef]
- Cai, W.; Jin, Y.; Chen, L. Task-Oriented User Evaluation on Critiquing-Based Recommendation Chatbots. IEEE Trans.-Hum.-Mach. Syst. 2022, 52, 354–366. [Google Scholar] [CrossRef]
- Chang, C.Y.; Kuo, S.Y.; Hwang, G.H. Chatbot-facilitated Nursing Education. Educ. Technol. Soc. 2022, 25, 15–27. [Google Scholar]
- Sicular, S.; Vashisth, S. Hype Cycle for Artificial Intelligence; Gartner: Stamford, CT, USA, 2020. [Google Scholar]
- Nimavat, K.; Champaneria, T. Chatbots: An overview. Types, architecture, tools and future possibilities. Int. J. Sci. Res. Dev. 2017, 5, 1019–1024. [Google Scholar]
- Ramesh, K.; Ravishankaran, S.; Joshi, A.; Chandrasekaran, K. A survey of design techniques for conversational agents. In Proceedings of the International Conference on Information, Communication and Computing Technology, New Delhi, India, 13 May 2017; pp. 336–350. [Google Scholar]
- Singh, J.; Joesph, M.H.; Jabbar, K.B.A. Rule-based chabot for student enquiries. J. Phys. Conf. Ser. 2019, 1228, 012060. [Google Scholar] [CrossRef]
- Wallace, R.S. The anatomy of ALICE. In Parsing the Turing Test; Springer: Dordrecht, The Netherlands, 2009. [Google Scholar]
- Mikic, F.A.; Burguillo, J.C.; Rodríguez, D.A.; Rodríguez, E.; Llamas, M. T-Bot and Q-Bot: A couple of AIML-based bots for tutoring courses and evaluating students. In Proceedings of the 2008 38th Annual Frontiers in Education Conference, Saratoga Springs, NY, USA, 22–25 October 2008; pp. S3A-7–S3A-12. [Google Scholar]
- Memon, Z.; Aghian, H.; Sarfraz, M.S.; Hussain Jalbani, A.; Oskouei, R.J.; Jalbani, K.B.; Hussain Jalbani, G. Framework for Educational Domain-Based Multichatbot Communication System. Sci. Program. 2021, 2021, 5518309. [Google Scholar] [CrossRef]
- Wu, Y.; Wu, W.; Xing, C.; Zhou, M.; Li, Z. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. arXiv 2016, arXiv:1612.01627. [Google Scholar]
- Hien, H.T.; Cuong, P.N.; Nam, L.N.H.; Nhung, H.L.T.K.; Thang, L.D. Intelligent assistants in higher-education environments: The FIT-EBot, a chatbot for administrative and learning support. In Proceedings of the Ninth International Symposium on Information and Communication Technology, Danang City, Vietnam, 6–7 December 2018; pp. 69–76. [Google Scholar]
- Ji, Z.; Lu, Z.; Li, H. An information retrieval approach to short text conversation. arXiv 2014, arXiv:1408.6988. [Google Scholar]
- Zhang, K.; Aslan, A.B. AI technologies for education: Recent research & future directions. Comput. Educ. Artif. Intell. 2021, 2, 100025. [Google Scholar]
- Hwang, G.J.; Chang, C.Y. A review of opportunities and challenges of chatbots in education. Interact. Learn. Environ. 2021, 31, 4099–4112. [Google Scholar] [CrossRef]
- Wang, J.; Hwang, G.H.; Chang, C.Y. Directions of the 100 most cited chatbot-related human behavior research: A review of academic publications. Comput. Educ. Artif. Intell. 2021, 2, 100023. [Google Scholar] [CrossRef]
- Gökçearslan, S.; Tosun, C.; Erdemir, Z.G. Benefits, challenges, and methods of artificial intelligence (AI) chatbots in education: A systematic literature review. Int. J. Technol. Educ. 2024, 7, 19–39. [Google Scholar] [CrossRef]
- Pérez, J.Q.; Daradoumis, T.; Puig, J.M.M. Rediscovering the use of chatbots in education: A systematic literature review. Comput. Appl. Eng. Educ. 2020, 28, 1549–1565. [Google Scholar] [CrossRef]
- Katchapakirin, K.; Anutariya, C.; Supnithi, T. ScratchThAI: A conversation-based learning support framework for computational thinking development. Educ. Inf. Technol. 2022, 27, 8533–8560. [Google Scholar] [CrossRef]
- Merelo, J.J.; Castillo, P.A.; Mora, A.M.; Barranco, F.; Abbas, N.; Guillén, A.; Tsivitanidou, O. Chatbots and messaging platforms in the classroom: An analysis from the teacher’s perspective. Educ. Inf. Technol. 2024, 29, 1903–1938. [Google Scholar] [CrossRef] [PubMed]
- Abbasi, S.; Kazi, H. Measuring effectiveness of learning chatbot systems on student’s learning outcome and memory retention. Asian J. Appl. Sci. Eng. 2014, 3, 251–260. [Google Scholar] [CrossRef]
- Fryer, L.; Carpenter, R. Bots as language learning tools. Lang. Learn. Technol. 2006, 10, 8–14. [Google Scholar]
- Santoso, H.A.; Winarsih, N.A.S.; Mulyanto, E.; Sukmana, S.E.; Rustad, S.; Rohman, M.S.; Nugraha, A.; Firdausillah, F. Dinus intelligent assistance (dina) chatbot for university admission services. In Proceedings of the 2018 International Seminar on Application for Technology of Information and Communication, Semarang, Indonesia, 21–22 September 2018; pp. 417–423. [Google Scholar]
- Holotescu, C. MOOCBuddy: A Chatbot for personalized learning with MOOCs. In Proceedings of the 13th International Conference on Human–Computer Interaction RoCHI, Iasi, Romania, 8–9 September 2016; pp. 91–94. [Google Scholar]
- Grossman, J.; Lin, Z.; Sheng, H.; Wei, J.T.Z.; Williams, J.J.; Goel, S. MathBot: Transforming online resources for learning math into conversational interactions. In Proceedings of the AAAI 2019 Story-Enabled Intelligence, Stanford, CA, USA, 25–27 March 2019. [Google Scholar]
- Pham, X.L.; Pham, T.; Nguyen, Q.M.; Nguyen, T.H.; Cao, T.T.H. Chatbot as an intelligent personal assistant for mobile language learning. In Proceedings of the 2018 2nd International Conference on Education and E-Learning, Bali, Indonesia, 5–7 November 2018; pp. 16–21. [Google Scholar]
- Majuri, J.; Koivisto, J.; Hamari, J. Gamification of education and learning: A review of empirical literature. In Proceedings of the 2nd international GamiFIN Conference, GamiFIN 2018, CEUR-WS, Pori, Finland, 21–23 May 2018. [Google Scholar]
- Lin, H.; Wan, S.; Gan, W.; Chen, J.; Chao, H.C. Metaverse in education: Vision, opportunities, and challenges. In Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan, 17–20 December 2022; pp. 2857–2866. [Google Scholar]
- Resnick, M.; Maloney, J.; Monroy-Hernández, A.; Rusk, N.; Eastmond, E.; Brennan, K.; Millner, A.; Rosenbaum, E.; Silver, J.; Silverman, B.; et al. Scratch: Programming for all. Commun. ACM 2009, 52, 60–67. [Google Scholar] [CrossRef]
- Weintrop, D. Block-based programming in computer science education. Commun. ACM 2019, 62, 22–25. [Google Scholar] [CrossRef]
- McCracken, D.D.; Reilly, E.D. Backus-naur form (bnf). In Encyclopedia of Computer Science; John Wiley and Sons: West Sussex, UK, 2003; pp. 129–131. [Google Scholar]
- Pattis, R.E. Teaching EBNF first in CS 1. ACM SIGCSE Bull. 1994, 26, 300–303. [Google Scholar] [CrossRef]
- Maloney, J.; Resnick, M.; Rusk, N.; Silverman, B.; Eastmond, E. The scratch programming language and environment. ACM Trans. Comput. Educ. (TOCE) 2010, 10, 1–15. [Google Scholar] [CrossRef]
- Brooke, J. Sus: A ‘quick and dirty’ usability scale. In Usability Evaluation in Industry; CRC Press: Boca Raton, FL, USA, 1996; Volume 189. [Google Scholar]
- Friesen, S.; Scott, D. Inquiry-Based Learning: A Review of the Research Literature; Alberta Ministry of Education: Edmonton, AB, Canada, 2013; Volume 32, pp. 1–32. [Google Scholar]
- Pedaste, M.; Mäeots, M.; Siiman, L.A.; De Jong, T.; Van Riesen, S.A.; Kamp, E.T.; Manoli, C.C.; Zacharia, Z.C.; Tsourlidaki, E. Phases of inquiry-based learning: Definitions and the inquiry cycle. Educ. Res. Rev. 2015, 14, 47–61. [Google Scholar] [CrossRef]
| Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).