KR102685644B1 - Learning diagnostic system using explainable artificial intelligence - Google Patents

Abstract

The learning diagnosis system using explainable artificial intelligence according to the present invention includes a processing unit that performs a knowledge diagnosis of a user based on a pre-trained diagnostic model and provides customized learning content to the user, wherein the processing unit is configured to: A knowledge map can be constructed based on the diagnostic model, and a learning path of the knowledge map can be provided according to the user's knowledge state.

Description

Learning diagnosis system using explainable artificial intelligence {LEARNING DIAGNOSTIC SYSTEM USING EXPLAINABLE ARTIFICIAL INTELLIGENCE}

The present invention relates to a learning diagnosis system using explainable artificial intelligence, and more specifically, to a learning diagnosis system using explainable artificial intelligence that performs a learning diagnosis of a learner.

Generally, online education systems provide education to learners through an online environment. However, in the online education system, it is difficult for instructors to diagnose the level of learners and appropriately provide the necessary feedback to each learner.

The conventional online education system has already been disclosed by “Korea Patent Publication No. 2007-0068594 (Online Education System and Method, July 27, 2007).” The disclosed invention is characterized by providing feedback to learners through an online environment.

In this way, the possibility and need for customized learning content tailored to the knowledge level of each learner is emerging. Learners have different knowledge levels and abilities, and each individual has different weaknesses. Accordingly, the currently available online education system focuses on providing content appropriate to the individual's knowledge level and strives to improve learning effectiveness in various ways.

However, conventional online education systems do not provide sophisticated diagnosis of learner learning. Additionally, there is a problem in that efficiency and reliability are not secured as customized learning content is provided based on inaccurate learning diagnosis results.

“Republic of Korea Patent Publication No. 2007-0068594 (Online education system and method, July 27, 2007)”

The purpose of the present invention is to use explainable artificial intelligence to diagnose the learner's knowledge level and optimize the learning path for each learner in the knowledge map for each knowledge area by utilizing the analysis results of knowledge concepts that had a positive and negative impact on the diagnosis result. It is intended to provide a learning diagnosis system.

The learning diagnosis system using explainable artificial intelligence according to the present invention includes a processing unit that performs a knowledge diagnosis of a user based on a pre-trained diagnostic model and provides customized learning content to the user, wherein the processing unit is configured to: A knowledge map can be constructed based on the diagnostic model, and a learning path of the knowledge map can be provided according to the user's knowledge state.

The processing unit may generate the diagnostic model by collecting learning data provided from a plurality of users in constructing the knowledge map.

The processing unit may apply the Dynamic Key-Value Memory Networks for Knowledge Tracing (DKVMN) algorithm in generating the diagnostic model.

The processing unit may calculate a correlation between knowledge concepts using the diagnostic model and construct the knowledge map using the correlation.

The processing unit may calculate the user's level of understanding for each knowledge concept after performing a diagnosis of the user's knowledge.

The processing unit may calculate the user's level of understanding for each knowledge concept after performing a diagnosis of the user's knowledge.

After performing the user's knowledge diagnosis, the processing unit may analyze the contribution of concepts that influenced the learning diagnosis.

In the contribution analysis, the Layer-wise Relevance Propagation (LRP) algorithm may be applied.

After performing a diagnosis of the user's knowledge, the processing unit may provide a learning path of the knowledge map by optimally analyzing the learning path of the knowledge map.

Meanwhile, the learning diagnosis method using explainable artificial intelligence according to the present invention includes the steps of building a knowledge map based on a pre-trained diagnosis model, performing a diagnosis of the user's knowledge based on the diagnosis model, and the steps of diagnosing the user's knowledge based on the diagnosis model. Analysis of the knowledge state is performed to provide customized learning content to the user, and the customized learning content may include a learning path of the knowledge map.

The learning diagnosis system using explainable artificial intelligence according to the present invention has the effect of calculating the level of understanding for each concept for concepts applied in complex to multiple questions, and improving the accuracy of the learner's knowledge level based on this.

In addition, the learning diagnosis system using explainable artificial intelligence according to the present invention can be used to analyze the customized learning path of the knowledge map by constructing a knowledge map using a model and analyzing the characteristic contribution to the diagnosis result, thereby improving academic efficiency. There is an effect that can be improved.

The technical effects of the present invention as described above are not limited to the effects mentioned above, and other technical effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a conceptual diagram schematically showing a learning diagnosis system using explainable artificial intelligence according to this embodiment,
Figure 2 is a flowchart showing a knowledge map construction method and a learning diagnosis method of a learning diagnosis system using explainable artificial intelligence according to this embodiment;
Figure 3 is a conceptual diagram showing the conventional Knowledge Tracing algorithm and the DKVMN algorithm according to this embodiment;
Figure 4 is a conceptual diagram schematically showing the characteristics of the DKVMN algorithm according to this embodiment,
Figure 5 is a conceptual diagram schematically showing contribution analysis using the LRP algorithm according to this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. However, this embodiment is not limited to the embodiment disclosed below and can be implemented in various forms. This embodiment only serves to ensure that the disclosure of the present invention is complete and to convey the scope of the invention to those skilled in the art. It is provided for complete information. The shapes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated with the same symbol in the drawings refer to the same elements.

1 is a conceptual diagram schematically showing a learning diagnosis system using explainable artificial intelligence according to this embodiment.

As shown in FIG. 1, the learning diagnosis system 1000 (hereinafter referred to as the diagnosis system) using explainable artificial intelligence according to this embodiment can collect learning data from learners and perform personalized learning diagnosis. .

This diagnostic system 1000 may include a communication unit 100, a data collection unit 200, and a processing unit 300.

The communication unit 100 can be linked with the personal terminal 10 through a wired or wireless method and provide learning data acquired from the learner to the data collection unit 200. Here, the learning data may include question solution result data. The question solution result data may be the answer to the diagnostic problem provided to the learner. And the personal terminal 10 may include a computer, a smartphone, a tablet PC, etc. for acquiring learning data.

Accordingly, the data collection unit 200 can store and manage learning data provided by learners, and provide the learning data to the processing unit 300 as needed.

Accordingly, the processing unit 300 can perform a learning diagnosis of the learner. Below, we will explain the knowledge map construction method and diagnosis method according to the creation of the diagnosis model.

Figure 2 is a flowchart showing a knowledge map construction method and a learning diagnosis method of a learning diagnosis system using explainable artificial intelligence according to this embodiment.

As shown in FIG. 2, the processing unit 300 of the diagnostic system 1000 according to this embodiment can perform knowledge map construction (S100) and learning diagnosis (S200) according to diagnostic model generation.

First, explaining how to build a knowledge map, the diagnostic system 1000 collects problem-solving result data from all users who possess the personal terminal 10 and can access the diagnostic system 1000 (S1). At this time, the diagnosis system 1000 may provide a diagnostic problem to all users in order to collect question solution result data and then collect the item solution result data.

Afterwards, the diagnosis system 1000 performs a data purification operation on the learning data including the question solution result data (S2). At this time, the diagnostic system 1000 may generate training data for generating a diagnostic model through a process of preprocessing the question solution result data.

Afterwards, the diagnosis system 1000 selects model variables using the training data and creates a diagnosis model for knowledge diagnosis (S3). In general, the Knowledge Tracing algorithm is used to diagnose the learner's knowledge status.

Figure 3 is a conceptual diagram showing the conventional Knowledge Tracing algorithm and the DKVMN algorithm according to this embodiment.

As shown in Figure 3, the Knowledge Tracing algorithm (hereinafter referred to as the KT algorithm) can model the learner's knowledge state that changes while acquiring a certain function. The KT algorithm uses learning data, that is, questions solved by each learner and whether or not they are correct, to quantitatively diagnose the learner's level of proficiency in the knowledge concepts they have learned.

In particular, conventional online education systems use the BKT (Bayesian Knowledge Tracing) algorithm, a statistical-based KT algorithm, to diagnose knowledge.

The BKT algorithm models the learner's knowledge state using the Hidden Markov Model (HMM) and Bayesian inference. However, it is not suitable for processing complex data because it is difficult to model long learning histories. In addition, the relationship between items is assumed to be independent, but the actual learning concept is hierarchically related, so there are limitations in application.

Accordingly, research and development is being conducted on the DKT (Deep Knowledge Tracing) algorithm using a deep learning model that predicts responses according to the user without making separate assumptions about the concept. However, due to the nature of deep learning, the DKT algorithm has a problem in that the learning concept exists as a hidden black box in deep learning, making it difficult to understand the learning concept.

Accordingly, the diagnostic system according to this embodiment generates a diagnostic model by applying the Dynamic Key-Value Memory Networks for Knowledge Tracing (DKVMN) algorithm. The DKVMN algorithm exhibits higher performance than the BKT algorithm, which has limitations, and provides various The DKVMN algorithm is a deep learning approach to consider interactions between concepts and uses a key-value maxrix to calculate the weight of each concept in a given problem and track how each student understands each concept. This calculates the level of knowledge about knowledge concepts applied complexly to multiple questions.

Figure 4 is a conceptual diagram schematically showing the characteristics of the DKVMN algorithm according to this embodiment.

As shown in Figure 4, in the DKVMN algorithm, the correlation weight can be obtained as in S1. At this time, the DKVMN algorithm calculates the similarity for each of n latent concepts through the problem index embedding vector, each column vector of the key matrix, and attention. And the correlation weight can be obtained by performing normalization through Softmax.

에 Value Matrix 의 각 행을 합해 학습자의 문제 에 대한 이해상태 를 계산할 수 있다. 다만, 다른 난이도의 문제일 때 같은 가중치를 가지면 같은 값이 나올 수 있다. 이를 해결하기 위해 레이어를 적용하여 최종 문제를 맞힐 확률 를 산출할 수 있다.Additionally, in the DKVMN algorithm, the correlation weight derived as in S2

In Value Matrix Learner's problem by summing up each row of state of understanding can be calculated. However, when problems of different difficulty levels are given the same weight, the same value may be obtained. Probability of getting the final problem right by applying layers to solve it can be calculated.

And in the DKVMN algorithm, like S3, the user's interaction can be used to update the current user's concept state.

When a diagnostic model is created using the DKVMN algorithm, the diagnostic system 1000 can calculate correlations between knowledge concepts (S4).

Then, the diagnosis system 1000 builds a knowledge map using the correlation between knowledge concepts (S5). A knowledge map may refer to content that can continuously provide knowledge content by classifying and linking knowledge content into fields and concepts. In other words, it may refer to content that links multiple contents to enable continuous learning in the form of a Mysmap, but this is for the purpose of explaining the present embodiment and is not limited thereto.

And when the knowledge map is built, the diagnosis system 1000 can start a service for learning diagnosis.

Hereinafter, the learning diagnosis method will be described. The diagnosis system 1000 can be accessed by a user who wishes to perform a learning diagnosis. Accordingly, the diagnosis system 1000 checks whether information corresponding to the user exists according to the user access (S6).

Afterwards, the diagnosis system 1000 performs learning diagnosis based on the diagnosis model (S7). In other words, learning diagnosis for individual users can be performed using the diagnosis model generated through the DKVMN algorithm.

In addition, the diagnosis system 1000 can perform understanding calculation for each knowledge concept (S8) and contribution analysis (S9) of concepts that had a positive or negative impact on the diagnosis result.

First, the diagnostic system 1000 can calculate the user's level of understanding. At this time, the diagnosis system can calculate the level of understanding for each knowledge concept. That is, the diagnosis system 1000 can calculate the degree of understanding for each knowledge concept using the correlation weight of the DKVMN algorithm.

And the diagnosis system 1000 can analyze the contribution of concepts that influenced the learning diagnosis prediction. In contribution analysis, the Layer-wise Relevance Propagation (LRP) algorithm can be applied.

Figure 5 is a conceptual diagram schematically showing contribution analysis using the LRP algorithm according to this embodiment.

As shown in FIG. 5, the LRP algorithm according to this embodiment is a method that supports understanding the results of a neural network through explanation by decomposition. This LRP algorithm can calculate a relevance score that indicates how much influence the previous layer has on the activation function of deep learning.

The basic assumption is that each neuron has some degree of contribution (certain relevance). Contribution has the characteristic of being redistributed from the output end of each neuron to the input end in a top-down manner. Here, the LRP algorithm has the characteristic of preserving contributions during distribution and redistribution.

에 대한 뉴럴 네트워크의 최종 출력은

이다. 출력단의 뉴런이 가지는 관련성 점수

를

와 같게 둔 다음 이전 레이어들로 번식(propagation)을 진행한다. 이에, 이전 레이어로 계속해서 번식을 진행하면 뉴럴 네트워크 상의 모든 뉴런들의 관련성 점수를 계산할 수 있기 때문에 진단 시스템(1000)은 진단결과에 긍정 및 부정 영향을 미친 개념들에 대한 기여도를 분석할 수 있다.Therefore, in order to calculate the contribution, the LRP algorithm uses a specific input

The final output of the neural network for is

am. Relevance score of the output neuron

cast

Leave it the same as and then proceed with propagation to the previous layers. Accordingly, if propagation continues to the previous layer, the relevance score of all neurons in the neural network can be calculated, so the diagnosis system 1000 can analyze the contribution of concepts that had a positive and negative influence on the diagnosis result.

Meanwhile, referring again to FIG. 2, after computation of understanding (S7) and contribution analysis (S8) are completed, the diagnosis system 1000 may perform optimization analysis on the learning path of the knowledge map created in advance ( S10).

In other words, you can set a user-tailored learning method. And the diagnosis system 1000 can provide customized learning content with a learning path corresponding to each user (S11).

Meanwhile, the diagnosis system 1000 can be accessed by a new user who wishes to perform a learning diagnosis (S12). Accordingly, the diagnosis system 1000 first checks whether information corresponding to the user exists according to the user's access.

If there is no score prediction model for the accessed user, the diagnosis system 1000 provides a diagnosis problem to the user (S13). Then, when the user provides result data for solving the diagnostic problem, the diagnostic system 1000 generates modeling data for the user through a purification process of the learning data (S14).

Afterwards, as described above, the diagnosis system performs knowledge diagnosis based on the learning diagnosis model and optimizes the user's knowledge map learning path through understanding calculation and contribution analysis processes. And the diagnosis system 1000 can provide customized learning content through a learning path corresponding to the user.

In this way, the learning diagnosis system using explainable artificial intelligence according to the present invention has the effect of calculating the understanding of each concept for concepts applied in complex to several questions, and improving the accuracy of the learner's knowledge level based on this. there is.

In addition, the learning diagnosis system using explainable artificial intelligence according to the present invention can be used to analyze the customized learning path of the knowledge map by constructing a knowledge map using a model and analyzing the contribution of characteristics to the diagnosis result, thereby improving academic efficiency. There is an effect that can improve.

An embodiment of the present invention described above and shown in the drawings should not be construed as limiting the technical idea of the present invention. The scope of protection of the present invention is limited only by the matters stated in the claims, and those skilled in the art can improve and change the technical idea of the present invention into various forms. Therefore, such improvements and changes will fall within the scope of protection of the present invention as long as they are obvious to those skilled in the art.

Claims (10)

It includes a processing unit that performs a user's knowledge diagnosis based on a pre-trained diagnosis model and provides customized learning content to the user,
The diagnostic model is
Generated based on the DKVMN (Dynamic Key-Value Memory Networks for Knowledge Tracing) algorithm,
The processing unit is
Construct a knowledge map using the correlation between knowledge concepts calculated using the diagnostic model,
In the diagnosis of the user's knowledge,
Perform knowledge diagnosis by inputting the user's information into the diagnosis model,
After performing the knowledge diagnosis, the user's understanding of each knowledge concept is calculated from the diagnosis model using the correlation weight of each knowledge concept generated in the knowledge diagnosis,
Analyzing the contribution of knowledge concepts that influenced the above knowledge diagnosis,
Explainable artificial intelligence is characterized in that it provides the learning path to the user by analyzing the optimization of the learning path of the knowledge map according to the user's knowledge state determined based on the understanding of each knowledge concept and the contribution. Learning diagnosis system used.
According to claim 1,
The processing unit is
A learning diagnosis system using explainable artificial intelligence, characterized in that the diagnosis model is generated by collecting learning data provided from a plurality of users in constructing the knowledge map.
delete delete delete delete delete According to claim 1,
In the above contribution analysis,
A learning diagnosis system using explainable artificial intelligence, characterized by applying the LRP (Layer-wise Relevance Propagation) algorithm.
delete A processing unit constructing a knowledge map based on a pre-trained diagnostic model;
performing, by the processing unit, a user's knowledge diagnosis based on the diagnosis model; and
A step of the processing unit performing analysis on the user's knowledge state and providing customized learning content to the user,
The diagnostic model is
Generated based on the DKVMN (Dynamic Key-Value Memory Networks for Knowledge Tracing) algorithm,
In the step of building the knowledge map,
The processing unit constructs a knowledge map using correlations between knowledge concepts calculated using the diagnostic model,
The steps for performing the learning diagnosis are
performing knowledge diagnosis by the processing unit inputting the user information into the diagnosis model;
The processing unit calculates the user's understanding of each knowledge concept using the correlation weight of each knowledge concept generated in the knowledge diagnosis from the diagnosis model;
A step of the processing unit analyzing the contribution of knowledge concepts that influenced the knowledge diagnosis,
In the step of providing the customized learning content,
Explainable artificial intelligence is characterized in that the optimization of the learning path of the knowledge map is analyzed according to the user's knowledge state determined based on the understanding of each knowledge concept and the contribution, and the learning path is provided to the user. Learning diagnosis method used.

Images