JP7502831B2 - Task recommendation system - Google Patents

Description

The present invention relates to a system that recommends tasks for learning, and more specifically, to a task recommendation system that provides a high learning effect by recommending tasks that a learner is not only explicitly weak at but also potentially weak at in a self-study system, based on the learning results using information recommendation technology that employs expert knowledge and artificial intelligence technology, etc.

Since the dawn of computer development, various self-study support systems have been developed, and services that provide appropriate tasks according to the learner's level have been in practical use for a long time. These services can be accessed using personal computers, smartphones, tablets, etc., and various services are provided for the purpose of preparing for various exams that require a lot of time, such as preparation for high school entrance exams, university entrance exams, various qualification exams, and language exams.

Because both tests require a certain score or higher, it is necessary to acquire the knowledge and academic ability necessary to answer the questions asked in these tests correctly. Therefore, the purpose of the above service is to support the efficient acquisition of knowledge and academic ability.

Although it is not guaranteed that you will be able to answer a question correctly the next time you answer it correctly, the chances of doing it correctly are higher than if you answered it incorrectly the next time. Therefore, in order to make effective use of your limited study time, it is important to repeatedly work on only the questions you answered incorrectly, so that you can turn your mistakes into correct answers the next time.

For this reason, many services have a function for relearning questions that have been answered incorrectly. Although a certain degree of effectiveness can be achieved by simply re-presenting the incorrect question, if the user re-studies immediately after answering it, it is impossible to eliminate the possibility that the user will choose the correct answer despite not understanding the essence of the question, due to factors such as the relationship between the answer choices. For this reason, it is effective to encourage the user to study the topic by re-presenting a different question that belongs to the same topic as the incorrect question.

Patent No. 6618226 JP 2017-134184 A

Let me consider this point using English language learning as an example. In Japan, when learning English, students are taught: 1. Basic grammar (action verbs and stative verbs, present tense and present progressive tense, past tense and past progressive tense, etc.)
2. Auxiliary verbs: May, can, and must for recommendation, auxiliary verbs and the perfect tense, idiomatic expressions with auxiliary verbs, etc.
3. Conditional mood: past conditional mood, future conditional mood, inversion conditional mood, etc. 4. Gerunds: It is common to learn systems such as gerund usage, negative forms of gerunds, etc. in order.
Therefore, English tests are also focused on checking whether students have mastered grammar. Many English question books and learning support systems are composed of multiple questions that are classified according to which grammar questions they ask. For example, the following question.

Question 1: Choose the appropriate answer for the following ( ).
The wedding party spent a lot of money. There were a lot of guests, and the hotel was the best in the city.
1. Must have cost
2. cannot have cost
3. Could not cost
4. Will cost

In the case of this question, the grammar item "auxiliary verb + perfect tense" related to the correct answer choice "must have cost" that fills in the blank is the theme that indicates the problem, and in the above system, this question belongs to "auxiliary verb and perfect tense" of 2 auxiliary verbs.

However, the key to answering this question lies in the "were" and "was" in the second sentence, and it is essential to understand that these are in the past tense. In other words, a user who does not understand the past tense will not be able to arrive at the correct answer to this question, even if they understand auxiliary verbs.

When conventional learning support systems re-ask questions, they ignore this point and recommend the next question to be answered "only" based on the grammar items related to the answer column, such as auxiliary verbs and perfect tense, and therefore recommend questions that are not essential countermeasures. As a result, it is not possible to improve academic ability by answering questions about the past tense, and it is not efficient in terms of improving academic ability and acquiring knowledge.

Furthermore, when teachers judge a student's achievement level, they usually take into account not only the grammatical theme of the question, but also other factors such as whether or not the student knew the background knowledge of the question beforehand, formal factors such as the question format and the number of blanks, the length of the question and the difficulty of the vocabulary, and psychological factors such as whether the student answered after much deliberation or by sheer luck or natural pencil rolling, but these factors were not taken into consideration in this study.

In recent years, artificial intelligence has been put to practical use on a large scale. One of the areas AI excels at is pattern matching, which is used in learning support systems to determine which pattern a question that a user has answered incorrectly belongs to, in other words, to calculate questions that are similar to the question that was answered incorrectly. AI is a promising technology, as it has the potential to recommend questions that may not be understandable at first glance but actually have important connections and contribute to understanding of a topic. On the other hand, the biggest weakness of information recommendation technology using AI technology is that it cannot clearly state the "reason" why a question was calculated to be similar, and it is difficult to effectively reflect the "knowledge" of experts in the output results by combining the "reason."

Therefore, in the present invention, a problem recommendation providing system that attaches tags to each problem is provided, i.e.,
A question database that manages multiple questions, tags, correct answers to the questions, and answer contents;
an answer status database that manages information regarding the correctness of answers to the questions of each of a plurality of users;
a computing device which performs learning on an information recommendation technology using artificial intelligence technology or the like in advance using information on the correctness of answers of some of the multiple users as an input signal, and is thereby capable of outputting users having information on the correctness of answers similar to that of a certain user when information on the correctness of answers of the certain user is input to the information recommendation technology using artificial intelligence technology or the like;
A task recommendation system comprising:
using the computing device in advance to calculate similarities between the certain user and the plurality of users and create a similarity table;
Extracting from the question database questions tagged with the same tag as a tag expressing a theme for which the certain user requests recommendations and which the certain user has not yet answered;
A problem recommendation system characterized in that, from among the extracted problems, multiple other users who are similar to the one user are selected from the similarity table, and multiple problems that the other users have answered incorrectly are selected and these are used as problems to be recommended to the one user.
I will provide a.

The problem recommendation system of the present invention allows experts to tag each problem multiple times, making it possible to classify problems from various perspectives, and efficiently provide problems that belong to themes truly necessary for the user to acquire knowledge, rather than from a uniform perspective. Furthermore, by recommending problems that similar users determined to have difficulty with using information recommendation technology using artificial intelligence technology, etc., it is possible to recommend only problems that the user is likely to have difficulty with and that the user has not yet answered, making it possible to significantly improve the efficiency of knowledge acquisition. In particular, in language learning, which has often relied solely on the grammar system until now, making recommendations from perspectives other than grammar can greatly increase learning efficiency.

Overview of the system for implementing the present invention Flowchart explaining the operation of the present invention

The problem recommendation system 1 of the present invention will be described with reference to the following diagram. The problem recommendation system 1 is composed of a distribution server 2 and a client terminal 3. FIG. 1 is a diagram showing an overview of the problem recommendation system 1, and FIG. 2 is a flow chart.

The problem recommendation system 1 of this embodiment functions as a system that solves problems on various English grammar in various question formats, such as long passages, short passages, multiple choice, multiple choice, and free description, with knowledge of various fields including languages as the study subject, and recommends problems that enable efficient language learning. The following explanation will be given with a focus on language learning, particularly English.

As shown in FIG. 1, the task recommendation system 1 of this embodiment is composed of a distribution server 2 that is connected to the Internet 1000, stores problem collection content 4 useful for English language learning by level of difficulty, and can distribute the problem collection content 4 via the Internet 1000, and a client terminal 3 that displays the problem collection content 4 distributed by the distribution server 2 to a user 1001 and allows the user 1001 to input answers. The distribution server 2 is installed by a company that provides English learning support services and operates the task recommendation system 1, and the client terminals 3 are each managed by the user 1001, but the client terminals 3 may also be installed and managed collectively by a company in a classroom or the like. Note that an intranet may be used instead of the Internet, and the system may operate as a stand-alone system without a network connection.

The distribution server 2 is hardware that includes a CPU, RAM, storage such as HDD and SDD, a network card, a power supply, and the like, and functions as a problem database 21 that manages problem collection content 4, which is various problems related to English, a web page generating means 22 that dynamically generates problems 41 extracted from the problem database 21 as web pages, a communication means 23 that transmits and receives data to and from the client terminal 3, an answer status database 24 that manages the answer status from the user 1001, and a recommendation system 25 that extracts problems similar to the problem 41 that the user answered incorrectly. In addition, the distribution server 2 may be equipped with a pointing device such as a keyboard, mouse, or trackpad to enable control of the distribution server 2 itself, and a display to display information about the distribution server 2 itself. The database may be an existing DBMS or alternative software having similar functions. The web page generating means 22 generates HTML, but may also use other languages such as PDF and PS, as long as it processes the problems 41 into information that the user can recognize.

When the distribution server 2 receives a question set distribution request from the client terminal 3 operated by the user 1001 via the communication means 23, it extracts questions 41 from the question database 21 that are related to the theme that the user 1001 requests to be distributed, processes the extracted questions 41 into web page display data that can be rendered as a question display screen by the web page generation means 22, and distributes the data to the client terminal 3 via the communication means 23.

The client terminal 3 is a personal computer, a tablet personal computer, a smartphone, etc., and more specifically, is hardware composed of a CPU, RAM, storage such as an HDD or SDD, a network card, a power supply, a display, a speaker, etc., and functions as a display means 31 that displays English questions 41, an input means 32 that accepts various inputs such as accepting answers to the displayed questions 41 in addition to various control instructions such as a question set distribution request and a recommendation request to the distribution server 2, a communication means 33 that can communicate with the distribution server 2, and a storage means 34 that temporarily stores the answer status of the user 1001 by working with a display, a speaker, and software, and presents the contents of the questions 41 to the user 1001 by visually displaying the questions 41 or audibly playing them back.
The input means 32 may be a pointing device such as a keyboard, a mouse, or a trackpad, or may be a touch panel integrated with a display means. In addition to these, the input means 32 has a camera that captures the gaze of the user 1001, a microphone that captures the user's voice, and a timer function that measures the time the user has spent on the answer.

The biggest difference between the task recommendation system 1 and conventional technology is that each problem managed in the problem database 21 is tagged by an educational expert. The tags are extracted elements that are necessary for a solver to determine the correct answer when solving a multiple-choice question and elements that affect the solving activity (hereinafter referred to as "answer elements"). One problem is assigned multiple tags.

Tags are managed by difficulty level as a tag book consisting of three attributes: grammatical attributes, formal attributes, and psychological influence factors. Grammatical attributes are set according to the major categories of English grammar, such as sentence pattern, voice, auxiliary verb, infinitive, gerund, participle, relative clause, comparison, conditional mood, conjunction, and tense. Each of these categories has medium and small categories. For example, the medium category "auxiliary verb" has the medium category "must" and the medium category "can" under it, and the small categories "tense" and "possibility" under the medium category "must". In addition, the medium category "tense" has the medium category "present tense", the medium category "past tense", and the medium category "past tense" has the small categories "basic" and "other". Each small category is assigned an importance level based on the importance of the grammar item. For example, the major category "auxiliary verb", the medium category "must", and the small category "possibility" are assigned an importance level of 3. The tags are expressed in the format "auxiliary verb - must - possibility, importance 3".

For each question, tags are added as primary grammar, which are elements considered to be directly necessary for filling in the answer column, in order of relevance. In addition, tags are added as secondary grammar, which are elements other than primary grammar that are considered to be necessary for the answer, in order of relevance.

The tag book at difficulty level 1, which is the easiest level, does not include tags related to advanced grammar such as the conditional mood or voice. This is because these grammatical elements are not tested in the questions assumed at difficulty level 1. However, the tag books at difficulty level 2, which is medium, and difficulty level 3, which is the most difficult, include tags related to the conditional mood and voice. Even for the same tag, the importance changes depending on the difficulty level. For example, "tense-past-verb to be" is the most important level 3 in the tag book at difficulty level 1, but is the least important level 1 in the tag book at difficulty level 3. This is because it is assumed that users working on difficulty level 3 have already mastered the past tense of the verb to be.

Formal attributes are set according to major classifications that focus on the formal aspects of the question, such as the question format, question format, answer elements, presence or absence of Japanese, number of blanks, number of words in the question, and sentence format, rather than the content of the question. Each of these categories has subcategories. For example, the major category "question format" has subcategories "symbol type", "description type", "rearrangement type", "other", etc.; the question format has "sentence selection type", "blank fill-in type", etc.; the major category "answer element" has subcategories "correct word", "incorrect word", "word fill-in", "word correction", "rearrange (whole sentence)", "rearrange (part of sentence)", etc.; the major category "number of blanks" has subcategories "0", "1", and "2 or more", the major category "number of words in question" has subcategories "less than 20 words", "20 to 40 words", and "40 or more", the major category "sentence format" has subcategories "normal sentence", "dialogue sentence", etc. Tags are expressed in the form of "answer element-word fill-in", "number of words in question-41 or more". All tags that belong to the format attribute will be added to each question.

The psychological influence factors are composed of answer rationale, question genre, word difficulty, special question format, etc. The major category "answer rationale" is divided into sub-categories "Grammar", "Usage", "Vocabulary", "Context", and "English logic". The major category "Question genre" is divided into sub-categories "Abstract sentences", "Social science", "Foreign culture", and "Business". The major category "Word difficulty" is divided into sub-categories "Grade 1 and above", "Pre-Grade 1", "Grade 2", and "Below Grade 2". The major category "Special question format" is divided into the sub-categories "Similar options", "Proper nouns", "No error type", and "Blank at the beginning of a sentence". The major category "special question format" is different from the question format attribute in that it is a question format that is psychologically difficult to get the right answer to, as recognized from experience and research over many years of teaching English conversation. In other words, the question format requires accurate knowledge, makes it difficult to make associations, etc., and is therefore tagged from the perspective that it is difficult and stimulates a sense of inadequacy, leading to a negative psychological state. The tags are expressed in the form of "answer basis - grammar". Tags that belong to the psychological influence element are assigned in order of relevance to tags that are thought to cause a sense of inadequacy in the user.

Below, we will further explain how to tag using some examples.
Question 1: Choose the appropriate answer for the following ( ).
The wedding party spent a lot of money. There were a lot of guests, and the hotel was the best in the city.
1. Must have cost
2. cannot have cost
3. Could not cost
4. Will cost

For question 1, the primary grammar tags are "auxiliary verb -- must -- tense, importance 3" and "auxiliary verb -- must -- possibility, importance 3", and the secondary grammar tags are "tense -- past tense -- basic, importance 2". The formal attributes are "question -- fill in the blank", "answer -- correct phrase", "number of answers -- 1", "number of words in question -- between 21 and 40", "contains Japanese -- no", and "sentence form -- normal sentence". The psychological influence factor is "basis for answer -- context".

Question 2: When rearranging the words in the Japanese sentence to make it have the equivalent meaning, choose the missing word.
It's almost impossible to finish my homework by tomorrow.
It is (impossible, to finish, next) the homework tomorrow.
1 the
2 with
3 of
4 to

In question 2, the primary grammar is tagged as "infinitive - noun usage - special expression, importance level 2", and there is no secondary grammar. The formal attributes are tagged as "question - rearrangement type", "what to answer - correct phrase", "number of answers - 1", "number of words in question - less than 20 words", and "presence of Japanese - yes". The psychological influence factor is tagged as "basis for answer - vocabulary".

Question 3 Muhammad Yunus, an idealist who wanted to ensure that loans are available to the poor, set up Grameen Bank; most of its lending is ( ) such as street vending and farming.
1. For income-generating activities
2. Making significant profits
3. To help excluded but motivated people
4. An alternative to large bank funding

For question 3, the primary grammar tag is "Sentence pattern-2 sentence pattern-complement type, importance 1", and the secondary grammar tag is "Other-Vocabulary-Idiom". The formal attributes are "Question-Fill in the blank", "What to answer-Correct phrase", "Number of answers-1", "Number of words in question-20 to less than 40", "Format-Regular sentence", and "Japanese-None". The psychological influence factors are "Basis for answer-Context", "Basis for answer-English logic", "Sentence content-Foreign culture", and "Special question format-Proper noun".

From the perspective of English grammar knowledge, this question cannot be answered without knowledge of the two sentence patterns, and traditional English learning has emphasized the perspective of grammar. In addition, grammatical knowledge such as "appositive," "relative," and "interpolation" is also required at the same time, so these perspectives have also been taken into consideration.

However, while a sufficient level of English grammar knowledge is likely to ensure that the answer is correct, it is possible that even those without sufficient grammar knowledge can get the answer right. In question 3, if you have knowledge of the microfinance provided by Grameen Bank, you will be able to at least infer the meaning of phrases that may be less familiar to Japanese learners of English, such as "street vending and farming" at the end, and it will be easier to get an idea of the overall meaning of the sentence. The bank's president, Muhammad Yunus, has been awarded the Nobel Peace Prize and has been widely reported in Japan, so the possibility of this being the case is quite high.

The inventors have found that in this way, when there is background knowledge, a certain level of English ability, especially vocabulary ability, can correctly infer the content of the question, which in fact often leads to the correct answer. In teaching English, the inventors have confirmed the existence of a type of learner who uses this background knowledge and then achieves a high rate of correct answers by using partial English knowledge as a foothold to infer and infer the content of the entire sentence.

In this invention, we focus on this point and tag items that are not related to English knowledge such as grammar knowledge or vocabulary, but have strong elements that lead to correct answers. As a result, by further analyzing the tags, we can determine strengths and weaknesses of this type, making it possible to make more efficient recommendations. In other words, even if the answer is correct, if it is thought that the user does not have sufficient grammar knowledge, it is possible to recommend similar grammar questions.

Question 4: If any of the answers (a) to (d) in the following questions contain incorrect English expressions, choose one from (a) to (d). If there are no errors, choose (e) No error.
(a) Having been in disagreement (b) for some time, the two sides made a temporary pact, but as everyone knows (c) all too well, the quarrel is (d) far from over. (e) No error

In question 4, the primary grammar is tagged as "participle-participle construction-tense, importance 2", and there is no secondary grammar. The formal attributes are tagged as "question-underlined type", "what to answer-incorrect word", "number of answers-1", "number of words in question-21 to less than 40", and "contains Japanese-nothing". The psychological influence factors are tagged as "basis for answer-context", "basis for answer-English logic", "sentence content-foreign culture", "special form-proper noun", and "special question format-no error type".

Here, to explain the no error type, the applicant's many years of experience in teaching English and research have revealed that there is a type of learner whose ability to grasp the content of the question and answer the question declines when the answer options include no error, resulting in a clear drop in the correct answer rate. It has been analyzed that the inclusion of no error does not result in a drop in the correct answer rate because more accurate knowledge is required and the difficulty level increases, but rather that the recognition that the difficulty is high takes precedence, and the negative psychological effects associated with a sense of difficulty take precedence, resulting in a drop in the correct answer rate. As with no error, the same applies to factors such as a long question, a large number of answer options, and long answer options, and the present invention is characterized in that these are tagged as psychological influence factors.

For this type of learner, it is extremely beneficial to gain experience in eliminating negative psychological effects other than those related to English, that is, to gain experience in repeatedly and intensively solving questions with the same question format but with a low English level. For this reason, in this invention, psychological influence element tags are introduced in order to efficiently recommend questions that share common psychological influence elements. With the introduction of psychological influence element tags, it is possible to create a set of questions that share common psychological influence elements but have different English-related learning content, such as grammar knowledge and vocabulary knowledge, making it easy to implement a lower English level.

Question 5: Choose the appropriate answer for the following ( ).
After leaving the room, he suddenly remembered that he ( ) the important papers behind.
1 leaf
2 has left
3 left
4 had left

This problem is tagged with "tense-past tense-tense agreement" and "tense-past tense-perfect", but the difficulty level 1 tagbook has it set to different importance depending on the difficulty level: "tense-past tense-tense agreement, importance 4", "tense-past tense-perfect, importance 4", while the difficulty level 3 tagbook has it set to "tense-past tense-tense agreement, importance 1", "tense-past tense-perfect, importance 2".

By changing the importance level according to the level of difficulty, it becomes easier to recommend questions that are optimal for the user's learning situation. To answer question 5 correctly, it is essential to have a complete mastery of the meaning and usage of "tense agreement" and "past perfect". However, because such grammatical usage does not present particularly severe problems in Japanese grammar, it is difficult for beginners to intuitively understand these English grammar items. To understand this, an understanding of English grammar theory and a certain level of experience (proficiency) in English are required.

However, for advanced users, these issues come up frequently, and they can be dealt with almost unconsciously without stopping. In other words, the importance of these issues to advanced users is very low. Therefore, there is little need to solve these problems, and it is reasonable to give them a low priority in the recommendations. This is achieved by setting tagbooks according to difficulty level.

In the question database 21, the above-mentioned tags are attached to each question 41 and managed. That is, the tags are the question ID, the question text, the answer to the question, one or more tags related to primary grammar, one or more tags related to secondary grammar, a tag related to formal attributes, and a tag related to psychological influence factors.

In the answer status database 24, a question ID and whether the question was answered correctly, incorrectly, or not answered are stored for each user. The specific data format can be freely designed using known technology, but for example, for a user's answer status, a component of 0 could represent an unanswered question, 1 a correct answer, and -1 an incorrect answer).

Next, the process flow of the distribution server 2 and the client terminal 3 in the task recommendation system 1 will be explained with reference to FIG. 2. Here, it is assumed that a certain user 1002 is the solver.

The distribution server 2 manages pre-tagged questions in a question database 21, and makes them available for transmission to the client terminal 3 via the web page generation means 22 and communication means 23. At this time, the question database 21 is equipped with an ID unique to the client terminal 3, and an authentication system using a user ID and password. However, this is not the case when security can be ensured, such as in a closed environment not connected to the general Internet.

The user 1002 logs in to the client terminal 3, selects a difficulty level through the input means 32, and requests the delivery of questions belonging to that difficulty level. The client terminal 3 transmits the question delivery request together with the difficulty level to the delivery server 2 through the communication means 33. At this time, a tag may be selected in addition to the difficulty level. For example, a question related to the tag "tense" of difficulty level 2 may be selected.

When a question delivery request is received from the client terminal 3, the delivery server 2 selects a question in response to the question delivery request, reads the selected question from the question database 21, converts it into data suitable for display on the client terminal 3 by the web page generation means 22, and transmits it to the client terminal 3 via the communication means 33. Note that the question delivery request can be freely selected according to the type of tag managed by the question database, such as questions of a certain level of difficulty, limited to a grammar item, limited to multiple grammar items, or limited to a certain question format.

The client terminal 3 displays the problem on the display means 31 using data received from the distribution server 2 via the communication means 33. The answer entered by the user 1002 after considering the problem is received via the input means 32, and the answer is temporarily stored in the storage means 34.

When the user 1002 has entered all the answers, the display means 31 starts displaying the answers to accept an instruction to start grading. When the user 1002 inputs an instruction to start grading through the input means 32, the client terminal 3 grades the answers entered by the user 1002 and displays the grading results on the display means 31. The grading results are also sent to the distribution server 2 through the communication means 33. The distribution server receives the grading results through the communication means 23 and records the question ID and correct or incorrect answer for each question together with the user ID of the user 1002 in the answer status database 24.

At this time, the answers stored in the storage means 34 may be sent to the distribution server 2 via the communication means 33, and the answers received via the communication means 23 may be graded by the distribution server 2 to obtain the grading results, which may then be recorded in the answer status database 24, and may also be sent to the client terminal 2 via the communication means 23, and the grading results obtained via the communication means 33 may be displayed on the display means 31. In this case, there is no need to store data necessary for grading in the client terminal 3.

In conjunction with the display of the scoring results, the display means 31 starts displaying a message for accepting a recommendation request. When the input of a recommendation request by the user 1002 is accepted by the input means 32, the client terminal 3 transmits this to the distribution server 2 via the communication means 33. When the distribution server 2 accepts the recommendation request via the communication means 23, it identifies the tag that was most recently assigned to the incorrect question ID as a recommendation tag candidate 43, and transmits the recommendation tag candidate 43 to the client terminal 3 via the communication means 23.

The client terminal 3 displays the recommendation tag candidates 43 received through the communication means 33 on the display means 31. The user 1002 accesses the recommendation tag candidates 43 displayed on the display means 31 and selects a recommendation tag 44 from among the recommendation tag candidates 43 using the input means 32 based on what problem a recommendation is needed for. The client terminal 3 transmits the recommendation tag 44 to the distribution server 2 through the communication means 33. Alternatively, the user 1002 may be configured to freely select the recommendation tag 44, or to select all of the recommendation tag candidates 43 as recommendation tags 44.

In this case, when the input of a recommendation request is received by the input means 32 in the client terminal 3, the tag that was most recently assigned to the incorrect question ID may be displayed on the display means 31 as a recommendation tag candidate 43 without communicating with the distribution server 2, and the user may select it using the input means 32. In this case, it is necessary to distribute in advance information regarding the question ID and tag to the question to be distributed to the client terminal 3.

When the distribution server 2 receives the recommendation tag 44 via the communication means 23, it extracts questions with the recommendation tag 44 that the user 1002 has not yet answered from the question database 21 as recommended question candidates 44.

The distribution server 2 calculates a set 1003 of users who are considered to have similar incorrect answer situations to the user 1002. This calculation may be performed by applying information recommendation technology using widely known artificial intelligence technology or the like. That is, a network consisting of multiple layers is trained using data on the user's incorrect answers as input signals. The trained network is then used to calculate users who are considered to be similar to the user 1002 using data on the incorrect answers as input. The similarity between users can be expressed by setting a vector for each user with the situation regarding each user's incorrect answers as an element, and by the angle between the vectors.

Since these calculations require a huge amount of calculations, it is necessary for software in the distribution server to automatically perform the calculations using hardware resources such as the CPU and memory. Since even using these hardware resources still requires a large amount of computing power, it is desirable to perform the calculations during times when the distribution server 2 has spare processing power. In this case, the calculation results are stored in the distribution server 2 as a similarity table, and when it becomes necessary to select users similar to a certain user, the similarity is read from the similarity table rather than being calculated each time.

For this calculation, each user needs to solve a certain number of common problems. Otherwise, it is difficult to calculate the similarity and dissimilarity between users. For this reason, each user may be encouraged to solve required problems consisting of multiple fields and multiple levels of difficulty, which may serve as basic material for calculating the similarity between users. Note that even if there are no required problems, similarity can be calculated if a sufficient number of common problems between users have been answered, so it may be left to the user.

Next, for each question in the recommendation question candidates 44, if a user belonging to user set 1003 answers incorrectly, a value multiplied by the similarity with user 1002 is added, and if the answer is correct, no points are added. After performing the calculation for all users belonging to user set 1003, the quotient obtained by dividing the result by user set 1003 is used as the error rate. In other words, a weighted average of the incorrect answering situations for each question is calculated using the user similarity as a weight. Note that if a user answers the same question incorrectly multiple times, the above-mentioned points are added each time.

A certain number of questions with the highest error rate are determined as recommended questions 42. At this time, importance may be taken into account in addition to the error rate. For example, for questions with high importance, the importance may be added to or multiplied by the error rate, or if the user has specified a tag, only the importance assigned to the tag may be added to or multiplied by the error rate. Note that the certain number can be set to a variety of numbers, such as the number of questions suitable for review, a number set by the user based on their own preferences, or a number specified by an expert as appropriate based on the learning situation.

The server 2 uses the question ID determined for the recommended question 42 to read the question from the question database 21, converts it into data suitable for display on the client terminal 3 using the web page generation means 22, and transmits it to the client terminal 3 via the communication means 33.

At that time, questions from completely random fields may be mixed in. This is not only effective from the viewpoint of preventing so-called overlearning, but also significant in terms of providing new material for learning English by recommending questions that the user does not expect. It is extremely useful to mix in questions that are not random but have some kind of correlation, for example, questions that are often answered incorrectly by users belonging to the user set 1003 and do not have the recommendation tag 44 attached, or questions that are often answered incorrectly by users belonging to the user set 1003 and do not have the recommendation tag candidate 43 attached.

As described above, the essence of the present invention is that the distribution server 2 is capable of identifying the field for which the user seeks recommendations, and is also capable of recommending questions that belong to that field, that the user has not yet answered, and that are often mistakenly answered by other users similar to the user. This calculation is too complex for a human to process, but it can be carried out by currently widely used computers, i.e., by using hardware resources in cooperation with software to operate the hardware.

As mentioned above, the recommendation calculation itself requires hardware resources and software, but the remaining parts can be modified without losing the essence of the present invention. For example, it is conceivable to accept direct operation of the distribution server 2 by the user or the person who asks the user questions, obtain information on the tags and difficulty level for which recommendations are sought, and make recommendations based on that information. In this case, since a computer is essential for calculations using a neural network, the use of hardware resources is essential for the distribution server 2, but paper may be used for the client terminal 3.

The embodiments of the present invention have been described above. Some or all of the embodiments of the present invention will be described as follows:
[Appendix 1]
A question database that manages multiple questions, tags, correct answers to the questions, and answer contents;
an answer status database that manages information regarding the correctness of answers to the questions of each of a plurality of users;
a computing device that performs learning on an information recommendation technology using artificial intelligence technology or the like in advance using information on the correctness of answers of some of the multiple users as an input signal, and is thereby capable of outputting users having information on the correctness of answers similar to that of a certain user when information on the correctness of answers of the certain user is input to the information recommendation technology using artificial intelligence technology or the like;
A task recommendation system comprising:
using the computing device in advance to calculate similarities between the certain user and the plurality of users and create a similarity table;
Extracting from the question database questions tagged with the same tag as a tag expressing a theme for which the certain user requests recommendations and which the certain user has not yet answered;
A problem recommendation system characterized in that, from among the extracted problems, multiple other users who are similar to the one user are selected from the similarity table, and multiple problems that the other users have answered incorrectly are selected and these are used as problems to be recommended to the one user.
[Appendix 2]
Computer,
A question database that manages multiple questions, tags, correct answers to the questions, and answer contents;
an answer status database that manages information regarding the correctness of answers to the questions of each of a plurality of users;
a computing device which performs learning on an information recommendation technology using artificial intelligence technology or the like in advance using information on the correctness of answers of some of the multiple users as an input signal, and is thereby capable of outputting users having information on the correctness of answers similar to that of a certain user when information on the correctness of answers of the certain user is input to the information recommendation technology using artificial intelligence technology or the like;
A program that functions as a task recommendation system comprising:
using the computing device in advance to calculate similarities between the certain user and the plurality of users and create a similarity table;
Extracting from the question database questions tagged with the same tag as a tag expressing a theme for which the certain user requests recommendations and which the certain user has not yet answered;
A program characterized by selecting from the similarity table multiple other users who are similar to the one user from the extracted questions, selecting multiple questions from among those incorrectly answered by the other users, and performing a process to recommend these questions to the one user.
[Appendix 3]
A question database that manages multiple tags, questions, correct answers to the questions, and answer contents;
an answer status database that manages information regarding the correctness of answers to the questions of each of a plurality of users;
a computing device that performs learning on an information recommendation technology using artificial intelligence technology or the like in advance using information on the correctness of answers of some of the multiple users as an input signal, and is thereby capable of outputting users having information on the correctness of answers similar to that of a certain user when information on the correctness of answers of the certain user is input to the information recommendation technology using artificial intelligence technology or the like;
A data structure for use in a task recommendation system comprising:
using the computing device in advance to calculate similarities between the certain user and the plurality of users and create a similarity table;
Extracting from the question database questions tagged with the same tag as a tag expressing a theme for which the certain user requests recommendations and which the certain user has not yet answered;
A data structure including tags, questions, correct answers to the questions, and answer contents, which is used in the process of selecting multiple other users who are similar to the one user from the similarity table from the extracted questions, selecting multiple questions from among those incorrectly answered by the other users, and recommending these questions to the one user.
[Appendix 4]
2. The problem recommendation system according to claim 1,
A problem recommendation system, characterized in that each of the problems is assigned a plurality of tags.
[Appendix 5]
5. The problem recommendation system according to claim 4,
The task recommendation system is characterized in that the multiple tags include psychological influence element tags that are assigned from the perspective of any one of the following: the basis for the answer, the genre of the question, the difficulty of the words, and a special question format.

1 Problem recommendation system 2 Distribution server 21 Problem database 22 Web page generation means 23 Communication means 24 Answer status database 25 Recommendation system 3 Client terminal (CPU,
31 Display means 32 Input means 33 Communication means 34 Storage means 4 Question collection content 41 Distributed questions 42 Recommended questions 43 Recommendation tag candidates 44 Recommendation tags 45 Recommended question candidates 1000 Internet 1001 User 1002 User 1003 who is the answerer A set of users with similar incorrect answer situations

Claims (2)

A question database that manages tags managed by difficulty level as a tag book consisting of three attributes , namely, questions , grammatical attributes, format attributes, and psychological influence factors , and multiple correct answers and answer contents for each question;
A problem recommendation system characterized in that, when a problem delivery request consisting of a combination of difficulty level and tag is received, a problem corresponding to the problem delivery request is read out from a problem database and transmitted to a client terminal.
Computer,
A question database that manages tags managed by difficulty level as a tag book consisting of three attributes , namely, questions , grammatical attributes, format attributes, and psychological influence factors , and multiple correct answers and answer contents for each question;
A program that functions as a problem recommendation system, characterized in that when a problem delivery request consisting of a combination of difficulty level and tag is received, a problem corresponding to the problem delivery request is read from a problem database and transmitted to a client terminal.