KR102452100B1 - Method, device and system for providing learning service base on brain wave and blinking eyes - Google Patents

Abstract

According to one embodiment, in a method for providing a learning service based on a brain wave and eye blinking performed by a server, the method comprises: a step of obtaining the brain wave of a first user measured in a first brain wave measuring device, when the brain wave of the first user is measured in the first brain wave measuring device worn by the first user, from a first user terminal connected to the first brain wave measuring device; a step of analyzing whether or not a condition of the first user is good based on the brain wave of the first user; and a step of providing a meditation content to the first user terminal, when confirmed that the condition of the first user is not good, and controlling the meditation content to be reproduced on a screen of the first user terminal. Therefore, the present invention is capable of having an effect of providing a more accurate learning service.

Description

EEG and blink-based learning service method, device and system {METHOD, DEVICE AND SYSTEM FOR PROVIDING LEARNING SERVICE BASE ON BRAIN WAVE AND BLINKING EYES}

The following embodiments relate to a technology for providing a learning service based on brain waves and eye blinking.

As a background technology related to embodiments, Republic of Korea Patent Publication KR 10-1587462 B1 discloses an EEG recognition type learning problem providing system. Specifically, the prior literature includes an input/output unit 23 for outputting a learning problem to a screen, and receiving input of a starting time and an ending time of solving the learning problem; an EEG detecting unit 25 for detecting EEG with the EEG measuring device 10 for a time required for solving a problem from the time when solving one learning problem starts to the time at which solving is finished; a proficiency determination unit 26 for obtaining brain wave energy from the size of the brain wave detected during the problem solving time; Brain wave recognition type comprising a control unit 21 that determines the type or difficulty of a learning problem according to the size of the acquired brain wave energy, selects a learning problem of the determined type or difficulty as the next problem, and outputs it to the input/output unit 23 Disclosed is a system for providing learning problems.

Through this, the prior literature has identified a problem-solving proficiency by brain wave energy when solving a problem to solve a problem that suits the skill of the learner, or to solve a problem to raise the skill for the type of problem to the skill of the top group. An EEG recognition type learning problem provision system is provided.

In addition, Korean Patent Publication No. KR 10-1842750 B1 discloses a real-time simulator for EEG training and an interface device using the same. Specifically, a memory in which at least one program is loaded; and at least one processor, wherein the at least one processor provides, under the control of the program, an imaginary motion for EEG training, and then a process of acquiring measured EEG data from a user through an EEG measuring device; and by extracting feature information corresponding to the imaginary motion from the brainwave data, discloses a system characterized in that the process of learning the brainwave characteristics of the user for the imaginary motion is processed.

However, since the prior literature provides a learning service only with EEG, there is a limit to providing an accurate learning service.

Therefore, it is required to implement a technology that provides a more accurate learning service by comprehensively considering not only the user's brain waves but also the blinking of the eyes.

Republic of Korea Patent Publication KR 10-1587462 B1 Republic of Korea Patent Publication KR 10-1842750 B1 Republic of Korea Patent Publication KR 10-2107337 B1 Republic of Korea Patent Publication KR 10-1955941 B1

According to an embodiment, based on brain wave and eye blinking, which calculates the user's concentration on the learning content by analyzing the time the user performs learning based on the change in the user's brain wave, the eye blinking state, and the gaze state An object of the present invention is to provide a learning service providing method, apparatus and system.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

According to an embodiment, in the method of providing a learning service based on brain waves and eye blinks performed by a server, when the first EEG measurement device worn by the first user measures the EEG of the first user, obtaining, from a first user terminal connected to the first EEG measuring device, the EEG of the first user measured by the first EEG measuring device; analyzing whether the state of the first user is good based on the brain wave of the first user; when it is confirmed that the first user's condition is not good, providing the meditation content to the first user terminal, and controlling the meditation content to be reproduced on the screen of the first user terminal; controlling a camera mounted on the first user terminal to operate and start shooting when it is confirmed that the first user's condition is good; acquiring a face image of the first user from the first user terminal when the first user's face is photographed by the first user terminal; recognizing the eyes of the first user from the face image of the first user and tracking the gaze of the first user; as a result of tracking the gaze of the first user, checking whether the gaze of the first user is gazing at the screen of the first user terminal; When it is confirmed that the gaze of the first user is gazing at the screen of the first user terminal, the learning content is provided to the first user terminal, and the learning content is controlled to be reproduced on the screen of the first user terminal step; While the learning content is being reproduced in the first user terminal, while acquiring the brain wave of the first user from the first user terminal connected to the first EEG measuring device, the face image of the first user is obtained from the first user terminal obtaining from, recognizing the eyes of the first user from the face image of the first user, tracking the gaze of the first user, and measuring the Eye Aspect Ratio (EAR) of the first user; When the reproduction of the learning content is completed, a change in the brain wave of the first user measured while the learning content is reproduced is checked, and a change in the EAR of the first user measured while the learning content is being reproduced checking an eye blinking state through , and checking a gaze state gazing at a screen through a gaze change of the first user tracked while the learning content is being played; and calculating the concentration of the first user on the learning content by analyzing the learning time based on the change in the brain wave of the first user, the blinking state, and the gaze state , EEG and blink-based learning service providing method is provided.

While the learning content is being reproduced in the first user terminal, the step of measuring the EAR of the first user while acquiring the brain wave of the first user may include, when the learning content is reproduced in the first user terminal, the checking whether the first user's EEG and the first user's EAR are being measured; when it is confirmed that both the brain wave of the first user and the EAR of the first user are being measured, adjusting the synch between the brain wave of the first user and the EAR of the first user; when it is confirmed that any one of the brain wave of the first user and the EAR of the first user is not being measured, controlling the reproduction of the learning content to be stopped after a preset waiting time; and when it is confirmed that neither the brain wave of the first user nor the EAR of the first user is being measured, controlling the reproduction of the learning content to immediately stop.

The calculating of the concentration level of the first user includes the concentration level of the first user that changes according to a reproduction time of the learning content based on the change in the brain wave of the first user, the blinking state, and the gaze state. calculating ; classifying and classifying an upper concentration section, a normal concentration section, and a lower concentration section section for the learning content based on the concentration level of the first user; checking whether a length of a first section among the concentration subsections is shorter than a preset first time interval; determining that re-learning of the first section is not necessary when it is confirmed that the length of the first section is shorter than the first time interval; when it is determined that the length of the first section is longer than the first time interval, checking whether the length of the first section is shorter than a preset second time interval; determining that a test for the first section is necessary when it is confirmed that the length of the first section is shorter than the second time interval; determining that re-learning of the first section is necessary when it is confirmed that the length of the first section is longer than the second time interval; If it is determined that the test for the first section is necessary, controlling the test questions for testing the learning contents of the first section to be displayed on the screen of the first user terminal and re-learning for the first section If it is determined that it is necessary, the method may include controlling the first section to be played back on the screen of the first user terminal.

While the learning content is being reproduced in the first user terminal, the step of tracking the gaze of the first user and measuring the EAR of the first user may include tracking the gaze of the first user, as a result of which the first user when it is confirmed that the gaze of the first user does not stare at the screen of the first user terminal, measuring the gaze departure time from the point in time when the gaze of the first user leaves the screen of the first user terminal; measuring the drowsiness time from the time when the eyes of the first user are closed when it is confirmed that the eyes of the first user are closed as a result of measuring the EAR of the first user; When it is confirmed that any one of the gaze departure time and the drowsiness time lasts longer than a preset first reference time, it is determined that the first user is unable to concentrate on learning because the first user is not learning through the learning content to do; when it is determined that the first user is not learning, stopping the reproduction of the learning content, and controlling the first user terminal to output a notification message that requires concentration; determining that a study guide for the first user is necessary when it is confirmed that any one of the gaze departure time and the drowsiness time has been maintained longer than a preset second reference time; and when it is determined that the learning guidance for the first user is necessary, transmitting a learning guidance request message for requesting the learning guidance of the first user to the guardian terminal.

The EEG and blink-based learning service providing method is to obtain 3D data on the surface of the first EEG measurement device through lidar, and obtain 2D data on the surface of the first EEG measurement device through a camera step; separating the union region of the 2D data and the 3D data to extract first data obtained by merging the 2D data and the 3D data; generating a first input signal by encoding the first data; inputting the first input signal to a third artificial intelligence, and obtaining a first output signal based on a result of the input of the third artificial intelligence; generating a first classification result for the surface of the first EEG measuring device based on the first output signal; analyzing the first data to detect cracks occurring on the surface of the first EEG measuring device; identifying cracks generated on the surface of the first EEG measuring device by region, and distinguishing between a normal region in which cracks are detected less than a preset first set value and a damaged region in which cracks are detected above the first preset value; extracting second data from which the damaged area is deleted from the first data; generating a second input signal by encoding the second data; inputting the second input signal to a fourth artificial intelligence, and obtaining a second output signal based on a result of the input of the fourth artificial intelligence; generating a second classification result for the surface of the first EEG measuring device based on the second output signal; and when the first classification result and the second classification result are the same, setting any one of the first classification result and the second classification result as the final classification result for the surface of the first EEG measuring device. may include

According to an embodiment, based on the change in the user's brain wave, the eye blinking state, and the gaze state, the user's learning time is analyzed and the user's concentration on the learning content is calculated, thereby not only the user's brainwave It has the effect of providing a more accurate learning service by comprehensively considering even the blink of an eye.

On the other hand, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is an exemplary diagram of a configuration of a system according to an embodiment.
2 is a flowchart illustrating a process of providing learning content according to an exemplary embodiment.
3 is a flowchart illustrating a process of calculating a user's concentration on learning content according to an exemplary embodiment.
4 is a diagram for explaining indices of learning content according to an embodiment.
5 is a flowchart illustrating a process of checking a measurement state while learning content is reproduced according to an exemplary embodiment.
6 is a flowchart illustrating a process of controlling re-learning to be performed for each section according to a user's concentration level according to an embodiment.
7 is a flowchart illustrating a process of providing a first response method when a user does not concentrate on learning according to an exemplary embodiment.
8 is a flowchart illustrating a process of providing a secondary response method when a user does not concentrate on learning according to an exemplary embodiment.
9 is a flowchart for explaining a process of classifying a surface of an EEG measuring apparatus according to an embodiment.
10 is a diagram for explaining learning of a first artificial intelligence and a second artificial intelligence according to an embodiment.
11 is a diagram for explaining a third artificial intelligence and a fourth artificial intelligence according to an embodiment.
12 is a diagram for explaining a method of learning an artificial neural network according to an embodiment.
13 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

Terms used in the examples are used for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

The embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like.

1 is an exemplary diagram of a configuration of a system according to an embodiment.

The system according to an embodiment may include the server 100 , user terminals 111-113 and wearable EEG measuring devices 131-133. The server 100, the user terminals 111-113, and the wearable EEG measuring devices 131-133 may be connected to a network to exchange data through wired/wireless communication. The system may analyze the brain waves and eye blinks of the user learning the learning content, and may calculate the learning degree by evaluating the user's learning state based on the analysis result.

The server 100 may be a self-server 100 owned by a person or an organization that provides services using the server 100, a cloud server 100, or p2p (peer-) of distributed nodes. to-peer) set. The server 100 includes an arithmetic function that a typical computer has; It may be configured to perform all or part of a store/refer function, an input/output function, and a control function. The server 100 may have at least one or more artificial intelligences that perform an inference function. The server 100 may be configured to communicate with the user terminals 111-113 through wired or wireless communication.

The server 100 may be linked with a web page or an application. The web page or application may provide a function of displaying and playing learning content for a user, such as a learning video. The webpage or the application may provide a function of displaying EEG information measured by the wearable EEG measuring devices 131-133 during user learning. The web page or application may provide a function of displaying the learning achievement information of users collected and stored in the server 100 .

The user terminals 111-113 may be a desktop computer, a notebook computer, a tablet, a smart phone, or the like. For example, as shown in FIG. 1 , the first user terminal 111 may be a desktop, the second user terminal 112 may be a laptop computer, and the third user terminal 113 may be a smartphone. The types of user terminals 111-113 may vary according to embodiments. The user terminals 111-113 may be configured to perform all or part of an arithmetic function, a storage/referencing function, an input/output function, and a control function of a typical computer. The user terminals 111-113 may be configured to communicate with the server 100 by wire or wireless. The user terminals 111-113 may access a web page linked with the server 100 or an application linked with the server 100 may be installed. The user terminals 111-113 may exchange data with the server 100 through a web page or an application.

The accounts 121-123 may log in to the user terminals 111-113 through a web page or an application. For example, the first user account 121 may log in to the first user terminal 111 , the second user account 122 may log in to the second user terminal 112 , and the third user account 123 may log in to the second user terminal 112 . ) may log in to the third user terminal 113 . When the accounts 121-123 are logged into the user terminals 111-113, respectively, the user terminals 111-113 may exchange data with the server 100 through a web page or an application. Each of the user accounts (121, 122, 123) has the right to access the basic information of each user stored in the server 100, EEG information and eye blink information while using each user's learning content.

The wearable brain wave measuring devices 131-133 may be worn by users before they watch learning content. Each of the wearable EEG measuring devices 131-133 may have a corresponding relationship with each of the user terminals 111-113. Each of the wearable EEG measuring devices 131 to 133 may communicate with each of the user terminals 111-113 by wire or wireless. That is, the first user terminal 111 and the first EEG measuring device 131 are connected, the second user terminal 112 and the second EEG measuring device 132 are connected, and the third user terminal 113 is connected. ) and the third EEG measuring device 133 may be in a connected state.

Each of the wearable EEG measuring devices 131-133 may have the same configuration and function as the bone conduction headset 130 of FIG. 1 . A person or organization that provides a service using the server 100 may rent, pay, or sell the bone conduction headset 130 to users who view learning content.

The bone conduction headset 130 may be a two-channel frontal sensor-based product. The bone conduction headset 130 may include an EEG sensor 141 , a bone conduction phone 142 , a communication module 143 , and the like. The EEG sensor 141 may measure the frequency, amplitude, wavelength, phase difference, etc. of the user's brain waves, specifically, alpha waves, beta waves, and delta waves. In addition, the bone conduction headset 130 may include an EOG sensor. Through this, it is possible to measure whether or not the user's eyes are opened or closed.

In addition, each of the wearable EEG measuring devices 131-133 may have a configuration, a shape, and a design of a headphone. In this case, the EEG sensor attached to the headphone may come into contact with the user's ear and the crown when worn, and may measure EEG information of the user. Alternatively, each of the wearable EEG measuring devices 131-133 may have an in-ear configuration, shape, and design that can be inserted into the user's ear, such as a wireless earphone. In this case, the EEG sensor attached to the in-ear type EEG measurement device may contact the user's ear part when worn, and may measure the user's EEG information. Each of the wearable EEG measuring devices 131-133 may be borrowed without limitation, as long as it can analyze the user's EEG through the attached EEG sensor, its configuration, shape, design, size, additional functions, and the like.

The server 100 may provide learning content for a user, such as a learning video, through a web page or an application. The server 100 may collect, analyze, and store EEG information measured through the wearable EEG measuring devices 131-133 while using the learning content of the users. The server 100 may measure the user's Eye Aspect Ratio (EAR) based on the user's image information captured through the user terminals 111-113. Here, the EAR is an aspect ratio of the eyes, and whether the eyes are open or closed may be checked through the EAR.

The server 100 may reconfigure and provide learning content for each user based on EEG information, eye blink information, and user's basic information.

Meanwhile, for convenience of explanation, only three user terminals 111-113 and three wearable EEG measurement devices 131-133 are illustrated in FIG. 1 , but the number of terminals may vary. As long as the processing capacity of the server 100 allows, the number of terminals is not particularly limited.

Hereinafter, embodiments will be described focusing on the operation of the server 100, and the embodiments are not limited by the subject or aspect of communication, and various application examples may be employed.

On the other hand, the server 100 may provide the learning content only when the brain wave enters a stable state through meditation by providing the meditation content to improve concentration.

When it is confirmed that the user's gaze deviates from the screen for a certain period of time (for example, 3 seconds) or more while the user is watching the learning content being reproduced on the screen of the user terminals 111-113, the server 100 learns You can control the content to stop playing. In this case, the server 100 may control the reproduction of the learning content to stop even through the time when the eyes are closed, in addition to the time when the gaze departs from the screen.

The server 100 checks the brain wave state of users while the users perform learning by watching the learning content being played on the screens of the user terminals 111-113, and the user's eye blinking state through the EAR. have.

The server 100 may calculate the concentration on the learning content through the brain wave state, the blinking state (drowsiness), the degree of gaze deviation from the screen, etc., when the learning of the learning content is completed and the viewing of the learning content is finished, and the learning time , information on learning time, etc. can be analyzed and provided to users.

The server 100 analyzes the brain wave state, the blinking state, the degree of gaze deviance from the screen, etc., while the users watch the learning content, and can select a section where the concentration is lowered, and the concentration is lower than a certain level. A learning service may be provided so that re-learning is performed for the section to be used.

According to an embodiment, the server 100 may provide a learning service for students, parents, and teachers. That is, the server 100 may provide a learning service that processes users who perform learning through a student application to proceed with learning, and may provide a learning service that processes so that guardians check the learning status through the parent application. In addition, it is possible to provide a learning service that processes to recommend a learning process through the teacher application. In this case, a posting space may be provided for each student so that comments left between students, parents, and teachers can be checked.

2 is a flowchart illustrating a process of providing learning content according to an exemplary embodiment.

Referring to FIG. 2 , first, in step S201 , the server 100 measures the EEG of the first user in the first EEG measurement device 131 worn by the first user, the first EEG measurement device 131 measures it. The EEG of the first user may be obtained from the first user terminal 111 .

Specifically, when the first user wears the first EEG measuring device 131, the first EEG measuring device 131 may measure the EEG of the first user and provide the measured EEG information of the first user. 1 may be transmitted to the user terminal 111 . At this time, the first user terminal 111 transmits the EEG information of the first user received from the first EEG measurement device 131 to the server 100 , and the server 100 receives the first EEG information from the first user terminal 111 . 1 It is possible to obtain the user's brain wave information.

According to an embodiment, the first EEG measuring apparatus 131 may measure a beta wave, an alpha wave, a delta wave, etc. of the first user. The first EEG measuring device 131 may measure the frequency, amplitude, wavelength, and phase difference of the EEG. The type and measurement item of EEG to be measured may vary depending on the embodiment.

In step S202 , the server 100 may analyze whether the state of the first user is good, based on the brain wave of the first user.

Specifically, the server 100 compares the brain wave of the first user registered in advance in the database with the brain wave of the first user obtained in real time, and sets the state of the first user to any one of a bad state, a good state, and a good state. can be classified.

The server 100 may analyze the state of the first user as good when the state of the first user is confirmed as good and good, and when the state of the first user is confirmed as bad, the first user can be analyzed to be in poor condition.

In step S203 , the server 100 may check whether the state of the first user is good.

If it is confirmed in step S203 that the state of the first user is not good, in step S204 , the server 100 provides the meditation content to the first user terminal 111 and meditates on the screen of the first user terminal 111 . You can control the content to be played. In this case, the server 100 may select the meditation content according to the state of the first user, and provide the meditation content with a longer reproduction length as the state of the first user becomes more serious.

After step S204, when the reproduction of the meditation content is completed, the process returns to step S201, and the server 100 may acquire the EEG of the first user stable through meditation again.

On the other hand, if it is confirmed in step S203 that the state of the first user is good, in step S205 , the server 100 may control the camera mounted on the first user terminal 111 to operate and start shooting. That is, the server 100 may transmit a control signal for camera operation to the first user terminal 111 , and the first user terminal 111 may start shooting through the camera.

In step S206 , when the first user's face is photographed by the first user terminal 111 , the server 100 may obtain an image of the first user's face from the first user terminal 111 . In this case, the first user terminal 111 may generate a face image of the first user by photographing the face of the first user through the camera, and may transmit the generated face image of the first user to the server 100 . .

In step S207 , the server 100 may recognize the eyes of the first user from the face image of the first user and track the gaze of the first user.

Specifically, the server 100 may recognize the left and right eye regions of the first user from the face image of the first user, respectively. That is, the server 100 may analyze the face image of the first user to determine the region where the left and right eyes of the first user are located, and through this, the region where the left eye of the first user is located and the first Regions in which the user's right eye is located may be recognized, respectively.

The server 100 may check the positions of the pupils in each of the recognized eye regions. That is, the server 100 may determine where the left pupil is located in the left eye region and may determine where the right pupil is located in the right eye region.

The server 100 may calculate a separation distance between the first user and the first user terminal 111 based on the size of each eye region and the size of the pupils.

Specifically, the server 100 may determine whether the first user is looking directly at the screen of the first user terminal 111 by comparing the size of the left eye area and the size of the right eye area, and If it is confirmed that the difference between the size and the size of the right eye region is within the error range, it may be confirmed that the first user is looking directly at the screen of the first user terminal 111 . In a state where the first user looks directly at the screen of the first user terminal 111, the server 100 checks the size occupied by the left pupil in the left eye region or the size occupied by the right pupil in the right eye region. , a separation distance between the first user and the first user terminal 111 may be calculated. To this end, in a state in which the pupil size of the first user is registered in advance for each distance, when the pupil is identified as the first size, the server 100 determines the distance corresponding to the first size between the first user and the first user terminal 111 . It can be calculated as the distance between them. For example, when the size of the pupil of the first user is determined to be 1 cm, the separation distance between the first user and the first user terminal 111 is calculated to be 30 cm, and when the pupil of the first user is confirmed to be 2 cm, the second The separation distance between the first user and the first user terminal 111 may be calculated as 20 cm. That is, as the size of the pupil increases, the separation distance between the first user and the first user terminal 111 may be calculated as a smaller value.

The server 100 may determine a position to which the first user's gaze is directed based on the position and separation distance of the pupils, and when it is confirmed that the gaze of the first user has moved according to the change in the position of the pupil, the moving first The user's gaze can be tracked.

Specifically, the server 100 sets the vector direction by checking which direction the pupils are located in each eye region, and sets the vector size through the separation distance between the first user and the first user terminal 111 , , a vector direction and a vector size can be used to predict a position to which the first user's gaze is directed to determine which part is gazing, and when the position of the pupils is changed, the gaze of the first user that moves according to the change in the position of the pupils can be continuously tracked.

In step S208 , as a result of tracking the gaze of the first user, the server 100 may determine whether the gaze of the first user is gazing at the screen of the first user terminal 111 . That is, the server 100 confirms whether the gaze of the first user is located within the screen of the first user terminal 111 through the position to which the gaze of the first user is directed, so that the first user ), you can check whether you are staring at the screen.

If it is confirmed in step S208 that the gaze of the first user is gazing at the screen of the first user terminal 111, in step S209, the server 100 provides the learning content to the first user terminal 111, 1 It is possible to control so that learning content is reproduced on the screen of the user terminal 111 .

Before providing the learning content to the first user terminal 111 , the server 100 may generate learning content suitable for the first user to take, based on the basic information of the first user. Here, the basic information of the first user is the user's age, gender, region, learning achievement, subjects taken, subjects not taken, subjects to be taken, preferred subjects, non-preferred subjects, and EEG information of the user measured when learning each learning content and the like. EEG information of the user measured when learning each learning content may be obtained from the first EEG measuring device 131 .

The learning content may be a content unit having one learning topic. The learning content may be a learning video or the like. Each learning content may include one or more learning content modules that can be changed in order or type. For example, when the learning content is a moving picture, the learning content may include a basic frame and one or more detailed image parts (learning content modules) that can be changed in order or type. The learning content modules may be inserted in a section in which the learning content modules may be inserted in the basic frame.

Learning content modules may be classified by category. For example, there may be categories such as a core concept explanation module, a background knowledge explanation module, a case application module, a problem solving explanation module, and a deep learning module. The learning content modules may have versions with different levels of difficulty for each category. For example, there are learning content module versions with different levels of difficulty within one learning content module category, such as core concept explanation module-low (bottom), core concept explanation module-medium (middle), and core concept explanation module-upper (top). there may be At least one or more categories of learning content modules may be included in the learning content. In this case, if the user has a high learning achievement, the user's learning content may include a learning content module of the same category of higher-difficulty level Modules may be included.

The server 100 may select each learning content suitable for each user to take, based on the basic information of each user. Subsequently, the server 100 may select the order and type of learning content modules in each learning content based on the basic information of each user. The server 100 may select the types of learning content modules by selecting the difficulty level of the learning content modules in the same category based on the user's learning achievement. The server 100 may change the order of learning content modules between different categories in the learning content so that the user's concentration duration is increased and the user's concentration subsection is minimized. Through this, the server 100 may generate learning content for each user.

A series of operations in which the server 100 generates each user's learning content may be performed based on the first artificial intelligence. To this end, the server 100 may include a pre-learned first artificial intelligence. The first artificial intelligence may be trained to receive basic information of each account 121 , 122 , and 123 to generate learning content for each account 121 , 122 , 123 . The first artificial intelligence receives as inputs the subjects that the account user needs to learn, the account user's learning achievement to date, the change in the account user's learning achievement change, and the EEG information when the account user uses the previous learning content, Learning content for (121, 122, 123) can be created. A specific learning operation of the first artificial intelligence will be described later with reference to FIG. 10 .

On the other hand, if it is confirmed that the first user's gaze is not gazing at the screen of the first user terminal 111 in step S208, it is determined that the learning preparation is not completed, and the process returns to step S201, and the server 100 may be performed again from the process of acquiring the brain wave of the first user.

3 is a flowchart illustrating a process of calculating a user's concentration on learning content according to an exemplary embodiment.

Referring to FIG. 3 , first, in step S301 , the server 100 may provide the learning content to the first user terminal 111 and control the learning content to be reproduced on the screen of the first user terminal 111 . . That is, the server 100 obtains the brain wave of the first user from the first user terminal 111 , and when it is confirmed that the gaze of the first user is gazing at the screen of the first user terminal 111 , preparation for learning By determining that the is completed, the learning content may be provided to the first user terminal 111 .

In step S302 , the server 100 compares the first user's EEG measured by the first EEG measuring device 131 with the first EEG measuring device 131 while the learning content is being reproduced in the first user terminal 111 . It may be obtained from the connected first user terminal 111 . In this case, the server 100 may index each user's EEG according to a predefined criterion. Here, the predefined indicators may include a high concentration section for each learning content, a normal concentration section, a low concentration section, a concentration duration, a drowsiness section, and the like. A detailed description of the predefined indicators will be described later with reference to FIG. 4 .

In step S303 , the server 100 acquires the face image of the first user from the first user terminal 111 that is photographing the face of the first user while the learning content is played in the first user terminal 111 . can

In step S304 , the server 100 may recognize the eyes of the first user from the face image of the first user.

In step S305 , the server 100 may check the movement of the eyes of the first user based on the eyes of the first user recognized from the face image of the first user, and track the gaze of the first user according to the movement of the eyes.

In step S306 , the server 100 may measure an EAR (Eye Aspect Ratio) of the first user based on the eyes of the first user recognized from the face image of the first user.

In step S307, the server 100 may check whether the reproduction of the learning content is completed.

If it is confirmed in step S307 that the reproduction of the learning content has not been completed, the process returns to step S301, and the server 100 reproduces the learning content on the screen of the first user terminal 111 until the reproduction of the learning content is completed. can be controlled as much as possible.

If it is confirmed in step S307 that the reproduction of the learning content has been completed, in step S308, the server 100 may determine a change in the brainwave of the first user based on the brainwave of the first user measured while the learning content is being reproduced. have.

If it is confirmed in step S307 that the reproduction of the learning content is completed, in step S309, the server 100 based on the gaze of the first user tracked while the learning content is being reproduced, through the change of the gaze of the first user, the first user You can check the status of the staring at the screen for each time period.

If it is confirmed that the reproduction of the learning content is completed in step S307, in step S310, the server 100 is based on the EAR of the first user measured while the learning content is being reproduced, through a change to the EAR of the first user. You can check the blinking status by time period. At this time, the server 100 measures the aspect ratio of the eyes of the first user through the EAR, and can check whether the first user has their eyes open or closed for each time period, and the number of eye blinks, the interval You can also check more.

In step S311, the server 100 may calculate the concentration of the first user on the learning content by analyzing the learning time based on the change in the brain wave of the first user, the blinking state, and the gaze state. have.

Specifically, the server 100 may analyze how intensively the learning was performed for each time period based on the change in the brain wave of the first user, and based on the blinking state of the first user, the number of blinks, the interval It is possible to analyze how intensively the learning was performed for each time zone through the time when the eyes were closed, etc., and based on the gaze state of the first user, it is possible to analyze how intensively the learning was performed for each time zone through whether or not the screen was stared. can do. That is, the server 100 may calculate the concentration of the first user on the learning content by comprehensively considering the first user's brain waves, blinking eyes, staring at the screen, and the like.

4 is a diagram for explaining indices of learning content according to an embodiment.

The learning content 400 may be a content unit having one learning topic. 4 , the learning content 400 may be a learning video.

Each of the wearable brain wave measuring devices 131-133 may measure a beta wave 401, an alpha wave 402, a delta wave 403, and the like of a user who is using the learning content. Each of the wearable EEG measuring devices 131-133 may measure the frequency, amplitude, wavelength, phase difference, etc. of each user's EEG. The type and measurement item of EEG to be measured may vary depending on the embodiment. The user's EEG information measured by each of the wearable EEG measuring devices 131-133 is transmitted to the server 100 through the respective user terminals 111-113 connected to the EEG measuring devices 131-133. can be

The server 100 may index each user's brain waves according to a predefined criterion. To this end, the server 100 determines the types of brain waves—beta wave 401, alpha wave 402, delta wave 403, etc.—and measurement items—frequency, while the user watches the learning content 400; Based on amplitude, wavelength, phase difference, etc., the concentration 410 of the user on the learning content 400 may be calculated. The user's concentration 410 may have time-series that changes according to the progress time of the learning content 400 . In a section in which the user's beta wave 401 is activated, the user's concentration 410 may increase or have a high value. In a section in which the user's alpha wave 402 or delta wave 403 is activated, the user's concentration 410 may decrease or have a low value.

Then, the server 100 may calculate each user's indices based on each user's concentration 410 . The index of each user may include a high concentration section for each learning content, a normal concentration section for each learning content, a low concentration section for each learning content, and a duration of concentration of each user.

The high concentration section 411 may be a section in which the user's concentration level 410 is activated by more than a predefined criterion. For example, in this section, the user's beta wave 401 may be activated above a predefined level. The concentration subsection 413 may be a section in which the user's concentration level 410 is lowered to less than or equal to a predefined standard. For example, in this section, the user's alpha wave 402 or delta wave 403 may be a section in which more than a predefined level is activated. The concentration normal section 412 may be a section in which the concentration level 410 of the user falls within a predefined reference range.

In the user's concentration level 410 , a predefined reference level of concentration (upper-threshold level of concentration and lower-level level of concentration) that separates the upper level section 411 and the lower level section 413 of the concentration level may be different for each learning content. The predefined criteria for classifying the high concentration section 411 and the concentration low section 413 may vary depending on the content of the learning content, difficulty level, target grade, and the like.

A predefined reference degree of concentration for classifying the upper concentration section 411 and the lower concentration section 413 for each learning content may be made based on the pre-learned inference of the second artificial intelligence. To this end, the server 100 may include a pre-learned second artificial intelligence. The second artificial intelligence receives the learning topic of each learning content, learning difficulty, average age of users taking the learning content, brain wave information of users who have taken the learning content, etc. A predefined reference degree of concentration (higher-threshold concentration and lower-threshold concentration) that distinguishes the section 411 and the concentration subsection 413 may be output for each learning content. A specific learning operation of the second artificial intelligence will be described later with reference to FIG. 10 .

The concentration duration 414 is a time interval from the start of the learning content 400 to the start point of the first subsection of the concentration level, the time from the end point of the n (n is a natural number)-th subsection to the start point of the n+1th subsection of the concentration level The interval and the time interval from the end point of the last concentration subsection to the end of the learning content 400 may be composed of a list as elements. That is, the concentration duration 414 may be configured using, as an element, a time period belonging to the high concentration section 411 and the concentration normal section 412 .

The server 100 may check the total concentration time of the user while the learning content is in progress by summing each element of the concentration duration 414 . In addition, there is a tendency for students to concentrate in the beginning of a lecture, but to become distracted after the middle of the lecture. In order to monitor this point, the server 100 checks the length of each element of the concentration duration 414 to see how well the user maintains concentration in the beginning of the learning content (lecture, etc.) You can see how much concentration you have lost. In addition, since the time zone that does not belong to each element is a time zone that belongs to the concentration subsection 413, the server 100 determines which module is the learning content module that the user is not interested in or has given up due to the high difficulty through the time zone that does not belong to each element. can be checked Through this, the server 100 may analyze the concentration and academic achievement of the user who uses the learning content 400 .

The server 100 reconfigures the learning content so that the concentration of the user can be improved based on the concentration high section 411, the concentration normal section 412, the concentration low section 413, and the concentration duration 414. have. Specifically, the server 100 increases the high concentration time for each user's learning content, reduces the concentration low time, reduces the drowsiness section confirmed through eye blinking, or the inability to concentrate section confirmed through the gaze It is possible to provide each user by reorganizing the learning content for the same learning topic by reducing the number of . The operation of the server 100 to reconstruct each user's learning content may include the following.

First, the server 100 determines the order or type of at least one or more learning content modules in the learning content 400 based on each user's concentration upper section of the learning content 400 and the concentration lower section of the learning content 400 . can be changed

For example, there may be a learning content module in which the first concentration sub-section of the learning content 400 and the time period overlap the most. The corresponding learning content module may be too difficult for the user or, conversely, too easy to attract the user's attention. By inputting the EEG information of the user measured when the corresponding learning content module is played back to the artificial intelligence included in the server 100, it is possible to determine whether the user thinks the learning content module is too difficult or easy. Based on this, the server 100 may change the learning content module to a version of the learning content module in which the difficulty of the corresponding learning content module is adjusted upward or downward.

Alternatively, there may be a learning content module in which the time period most overlaps with the first high-intensity section of the learning content 400 . The corresponding learning content module may be an area that the user is interested in, or simply because it is the beginning of the learning content, the user's concentration may be high. By inputting the user's brain wave information measured when the corresponding learning content module is played back to the artificial intelligence included in the server 100, the user's concentration is reduced whether the user finds the corresponding learning content module interesting or simply because it is the beginning of the learning content. can determine if it is good. If the user finds the corresponding learning content module interesting, the server 100 may increase the difficulty level of the corresponding learning content module. If the concentration is high simply because it is the beginning of the learning content, the server 100 may arrange the order of the learning content module to the later part when the user reviews the learning content.

In addition, the server 100 may distribute the break time to the learning content 400 provided to each user based on the concentration duration 414 of each user.

For example, when the user uses 60-minute learning content 400, the first element of the concentration duration 414 is “Lecture start ~ 30 minutes”, the second element is “35 minutes ~ 45 minutes”, and the third element is It may be “55 minutes ~ the end of the lecture”. The server 100 indicates that the user has good concentration for up to half the time of the learning content (30-minute concentration), then loses concentration for a short time (low concentration for 5 minutes), and the concentration time becomes shorter (10-minute concentration), It is possible to analyze the point that the time to miss concentration becomes longer (concentration subsection for 10 minutes) and the concentration time becomes shorter (concentration for 5 minutes).

The server 100 may distribute the break time based on the time the user's concentration is disturbed, and set the length of each break time. For example, based on the elapsed time of the learning content excluding the break time, the server 100 may distribute the break time such as “After 30 minutes: 5 minutes break”, “After 20 minutes: 10 minutes break”, etc. .

5 is a flowchart illustrating a process of checking a measurement state while learning content is reproduced according to an exemplary embodiment.

Referring to FIG. 5 , first, in step S501 , the server 100 may provide the learning content to the first user terminal 111 to control the learning content to be reproduced on the screen of the first user terminal 111 . .

When the learning content is reproduced in the first user terminal 111, the server 100 measures the brain wave of the first user according to whether the brain wave information of the first user is being acquired from the first user terminal 111. It can be checked whether the state is in the state, and whether the EAR of the first user is being measured through the face image obtained from the first user terminal 111 can be confirmed.

In step S502 , the server 100 may determine whether the EEG of the first user is being measured according to whether EEG information of the first user is being obtained from the first user terminal 111 .

If it is confirmed in step S502 that the brain wave of the first user is being measured, in step S503 , the server 100 determines whether the EAR of the first user is being measured through the face image obtained from the first user terminal 111 . can check whether

If it is confirmed in step S503 that the EAR of the first user is being measured, in step S504, the server 100 may adjust the synchronization between the brainwave of the first user and the EAR of the first user. That is, if it is confirmed that both the brainwave of the first user and the EAR of the first user are being measured, the server 100 may adjust the synchronization between the brainwave of the first user and the EAR of the first user.

In step S505, the server 100 may check whether the reproduction of the learning content is completed.

If it is confirmed in step S505 that the reproduction of the learning content has been completed, in step S508 , the server 100 may control the reproduction of the learning content to be stopped by completing the reproduction of the learning content.

If it is confirmed in step S505 that the reproduction of the learning content is not completed, the process returns to step S501, and the server 100 may control the learning content to be continuously reproduced.

On the other hand, if it is confirmed in step S502 that the brain wave of the first user is not being measured, in step S506 , the server 100 measures the EAR of the first user through the face image obtained from the first user terminal 111 and You can check whether it exists or not.

If it is confirmed that the EAR of the first user is being measured in step S506, or it is confirmed that the EAR of the first user is not being measured in step S503, in step S507, the server 100 performs EEG or EAR within a preset waiting time. It can be checked whether this is additionally measured.

That is, when it is confirmed that the EEG of the first user is measured, but the EAR of the first user is not measured, the server 100 may check whether the EAR of the first user is additionally measured within the waiting time, When it is confirmed that the first user's EAR is measured but the first user's EEG is not measured, it may be checked whether the first user's EEG is additionally measured within the waiting time.

If it is confirmed that the brainwave of the first user or the EAR of the first user is additionally measured in step S507, in step S504, the server 100 may adjust to synchronize the brainwave of the first user and the EAR of the first user. have.

If it is confirmed that the brain wave of the first user or the EAR of the first user is not additionally measured in step S507, in step S508, the server 100 may control the reproduction of the learning content to be stopped. That is, when it is confirmed that any one of the brain wave of the first user and the EAR of the first user is not being measured, the server 100 may control the reproduction of the learning content to be stopped after the waiting time.

On the other hand, if it is confirmed in step S506 that the EAR of the first user is not being measured, in step S508 , the server 100 may control the reproduction of the learning content to be stopped. That is, when it is confirmed that neither the brain wave of the first user nor the EAR of the first user is being measured, the server 100 may control the reproduction of the learning content to immediately stop.

6 is a flowchart illustrating a process of controlling re-learning to be performed for each section according to a user's concentration level according to an embodiment.

Referring to FIG. 6 , first, in step S601 , the server 100 changes the concentration of the first user according to the reproduction time of the learning content based on the change of the first user's brain wave, the blinking state, and the gaze state. can be calculated. In this case, the concentration of the first user may have a time-series characteristic that changes according to the progress time of the learning content, like the concentration 410 of the user shown in FIG. 4 .

In step S602 , the server 100 may classify and classify an upper concentration section, a normal concentration level section, and a lower concentration level section for the learning content based on the concentration level of the first user. At this time, the concentration high section, the concentration normal section, and the concentration low section classified in the concentration of the first user are the high concentration section 411, the concentration normal section 412 and the concentration sub section 413 shown in FIG. It may be classified and classified according to a predefined reference concentration level (higher threshold concentration level and lower threshold concentration level).

In step S603, the server 100 may check the length of the first section among one or more sections classified into the concentration subsection. In this case, the server 100 may select a section having the longest time length among sections classified into the concentration subsections as the first section, thereby confirming the time length of the first section.

In step S604 , the server 100 may check whether the length of the first section is shorter than the first time interval. Here, the first time interval may be set differently according to the content, difficulty, target grade, etc. of the learning content.

If it is determined in step S604 that the length of the first section is shorter than the first time interval, in step S605, the server 100 may determine that re-learning for the first section is not required.

If it is determined in step S604 that the length of the first section is longer than the first time interval, in step S606, the server 100 may check whether the length of the first section is shorter than the second time interval. Here, the second time interval may be set to a larger value than the first time interval.

If it is determined in step S606 that the length of the first section is shorter than the second time interval, in step S607, the server 100 may determine that the test for the first section is necessary.

In step S608 , if it is determined that the test for the first section is necessary, the server 100 may control so that a test question for testing the learning contents of the first section is displayed on the screen of the first user terminal.

According to one embodiment, when it is confirmed that the server 100 has passed the test as a result of performing the test for the first section through the test questions for testing the learning contents of the first section, by performing step S605 , it can be determined that re-learning for the first section is not necessary, and when it is confirmed that the test has not been passed, step S609 is performed to determine that re-learning for the first section is necessary.

On the other hand, if it is confirmed in step S606 that the length of the first section is longer than the second time interval, in step S609, the server 100 may determine that re-learning of the first section is necessary.

In step S610 , if it is determined that re-learning for the first section is necessary, the server 100 may control the first bestiality to be re-played on the screen of the first user terminal 111 .

According to one embodiment, when it is confirmed that there are more sections classified into the concentration subsection in addition to the first section, the server 100 selects a section having the longest time length after the first section among sections classified as the concentration subsection. By selecting 2 sections, the time length of the second section can be checked. Thereafter, the server 100 may determine whether re-learning is necessary for the second section through the length of the second section. That is, the server 100 may determine whether re-learning is required for each of the sections classified into the concentration subsections, and select only sections requiring re-learning to be reproduced on the screen of the first user terminal 111 . can be controlled

7 is a flowchart illustrating a process of providing a first response method when a user does not concentrate on learning according to an exemplary embodiment.

Referring to FIG. 7 , first, in step S701 , the server 100 determines which point the first user's gaze is gazing at based on the image information obtained from the first user terminal 111 , and the first user can be tracked.

In step S702 , as a result of tracking the gaze of the first user, the server 100 may determine whether the gaze of the first user is gazing at the screen of the first user terminal 111 . In this case, the first user terminal 111 is playing the learning content on the screen.

When it is confirmed that the gaze of the first user is gazing at the screen of the first user terminal 111 in step S702, the first user performs learning through the learning content being reproduced on the screen of the first user terminal 111 Determining that the user is doing, returning to step S701, the server 100 may continue to track the gaze of the first user.

If it is confirmed in step S702 that the gaze of the first user is not gazing at the screen of the first user terminal 111 , in step S703 , the server 100 determines that the gaze of the first user is that of the first user terminal 111 . It is possible to measure the gaze departure time from the point of time out of the screen. That is, when the first user's gaze is gazing at the screen of the first user terminal 111 and then leaves the screen of the first user terminal 111 and gazes at another point, the first user terminal ( 111), the time spent gazing at another point outside the screen can be measured as the gaze departure time.

In step S704 , the server 100 may determine whether the gaze departure time continues longer than the first reference time. Here, the first reference time may be set differently according to embodiments.

If it is confirmed that the gaze departure time is shorter than the first reference time in step S704, it is determined that the first user temporarily leaves the screen of the first user terminal 111 and gazes at another point, and returns to step S701 , the server 100 may continue to track the gaze of the first user.

If it is confirmed in step S704 that the gaze departure time is longer than the first reference time, in step S709, the server 100 performs learning through the learning content that the first user is playing in the first user terminal 111 and It can be judged as a state of inability to concentrate on learning.

Meanwhile, in step S705 , the server 100 may measure the EAR of the first user by recognizing the eyes of the first user based on the image information obtained from the first user terminal 111 .

In step S706 , the server 100 may determine whether the eyes of the first user are closed as a result of measuring the EAR of the first user. In this case, the first user terminal 111 is playing the learning content on the screen.

If it is determined in step S706 that the eyes of the first user are not closed, it is determined that the first user is learning through the learning content being reproduced on the screen of the first user terminal 111, and the process returns to step S705. A, the server 100 may continue to measure the EAR of the first user.

If it is determined in step S706 that the eyes of the first user are closed, in step S707 , the server 100 may measure the drowsiness time from the time when the eyes of the first user are closed. That is, when the first user's eyes are changed from an open state to a closed state, the server 100 may measure the time maintained in the closed state as drowsiness time.

In step S708 , the server 100 may determine whether the drowsiness time lasts longer than the first reference time.

If it is confirmed in step S708 that the drowsiness time is shorter than the first reference time, it is determined that the first user has temporarily closed and opened their eyes, and the process returns to step S705, and the server 100 retrieves the EAR of the first user. You can continue to measure.

If it is confirmed in step S708 that the drowsiness time lasts longer than the first reference time, in step S709, the server 100 is learning through the learning content that the first user is playing in the first user terminal 111. Therefore, it can be judged as a state of inability to concentrate on learning.

In step S710 , the server 100 may control to stop the reproduction of the learning content being reproduced in the first user terminal 111 , and may control the first user terminal 111 to output a notification message that requires concentration. In this case, the first user terminal 111 may output a notification message requiring concentration as a sound to induce the user to focus on learning again.

8 is a flowchart illustrating a process of providing a secondary response method when a user does not concentrate on learning according to an exemplary embodiment.

In the description of FIG. 8 , the contents overlapping those of FIG. 7 will be omitted.

Referring to FIG. 8 , in step S801 , the server 100 may re-track the gaze of the first user after the first user terminal 111 outputs a concentration need notification message.

In step S802 , as a result of tracking the gaze of the first user, the server 100 may determine whether the gaze of the first user is gazing at the screen of the first user terminal 111 . At this time, the first user terminal 111 is in a standby state because reproduction of the learning content is stopped.

When it is confirmed that the gaze of the first user is gazing at the screen of the first user terminal 111 in step S802, it is determined that the first user is ready to focus on learning, and the server 100 sends the first user terminal ( 111), it is possible to control the learning content to be played again. After that, returning to step S801, the server 100 may continue to track the gaze of the first user.

If it is confirmed in step S802 that the gaze of the first user is not gazing at the screen of the first user terminal 111, in step S803, the server 100 follows the gaze departure time measured in step S703, followed by the gaze departure time can be further measured.

In step S804, the server 100 may determine whether the gaze departure time continues longer than the second reference time.

When it is confirmed that the gaze departure time is shorter than the second reference time in step S804, it is determined that the first user is ready to focus on learning, and the server 100 returns the learning content from the first user terminal 111 to the first user terminal 111. You can control it to play. After that, returning to step S801, the server 100 may continue to track the gaze of the first user.

If it is confirmed in step S804 that the gaze departure time continues longer than the second reference time, in step S809, the server 100 determines that the first user cannot concentrate on learning and thus learning guidance for the first user is required. have.

Meanwhile, in step S805 , the server 100 may measure the EAR of the first user again after the concentration required notification message is output from the first user terminal 111 .

In step S806 , as a result of measuring the EAR of the first user, the server 100 may determine whether the eyes of the first user are closed. At this time, the first user terminal 111 is in a standby state because reproduction of the learning content is stopped.

If it is determined that the eyes of the first user are not closed in step S806 , it is determined that the first user is ready to concentrate on learning, and the server 100 controls the learning content to be played back in the first user terminal 111 . can do. After that, returning to step S805, the server 100 may continue to measure the EAR of the first user.

If it is determined in step S806 that the eyes of the first user are closed, in step S807 , the server 100 may further measure the drowsiness time following the drowsiness time measured in step S707 .

In step S808 , the server 100 may determine whether the drowsiness time lasts longer than the second reference time.

If it is determined that the drowsiness time is shorter than the second reference time in step S808 , it is determined that the first user is ready to focus on learning, and the server 100 replays the learning content in the first user terminal 111 . can be controlled as much as possible. After that, returning to step S805, the server 100 may continue to measure the EAR of the first user.

If it is confirmed in step S808 that the drowsiness time lasts longer than the second reference time, in step S809, the server 100 determines that the first user is unable to concentrate on learning, so that learning guidance for the first user is required. .

In step S810 , if it is determined that the learning guidance for the first user is required, the server 100 may transmit a learning guidance request message requesting the learning guidance of the first user to the guardian terminal. In this case, the server 100 may check the guardian contact information of the first user and transmit a learning guidance request message to the guardian terminal.

That is, when the first user cannot concentrate on learning, the server 100 takes the first response method. The first user terminal 111 can control to output a notification message that requires concentration, and when the first user still cannot concentrate on learning even after the output of the concentration notification message, as a secondary countermeasure, a learning guidance request message is sent to the guardian It can be transmitted to the terminal.

In step S710 , the server 100 may control to stop the reproduction of the learning content being reproduced in the first user terminal 111 , and may control the first user terminal 111 to output a notification message that requires concentration. In this case, the first user terminal 111 may output a notification message requiring concentration as a sound to induce the user to focus on learning again.

9 is a flowchart for explaining a process of classifying a surface of an EEG measuring apparatus according to an embodiment.

Referring to FIG. 9 , first, in step S901 , the server 100 may acquire 3D data on the surface of the first EEG measuring device 131 through the lidar. Here, the 3D data is a 3D image of the surface of the first EEG measuring device 131 . To this end, the server 100 may be connected to the first user terminal 111 in which the lidar is mounted through wired or wireless.

In step S902, the server 100 may acquire 2D data on the surface of the first EEG measuring device 131 through the camera. Here, the 2D data is a 2D image of the surface of the first EEG measuring device 131 . To this end, the server 100 may be connected to the first user terminal 111 equipped with a camera through wired or wireless.

In step S903 , the server 100 may separate the union region of the 2D data and the 3D data to extract the first data obtained by merging the 2D data and the 3D data.

Specifically, the server 100 can compare the 2D data and the 3D data to identify overlapping union regions, separate the union regions from the 2D data and separate the union regions from the 3D data, and merge the separated union regions. Thus, the first data may be extracted. Here, the first data may consist of 4 channels, 3 channels may be 2D data representing an RGB value, and 1 channel may be data representing a 3D depth value.

The server 100 may generate a first input signal by encoding the first data.

Specifically, the server 100 may generate a first input signal by encoding pixels of the first data with color information. The color information may include, but is not limited to, RGB color information, brightness information, saturation information, and depth information. The server 100 may convert the color information into a numerical value, and may encode the first data in the form of a data sheet including this value.

The server 100 may input the first input signal to the third artificial intelligence learned in advance in the server 100 .

The third artificial intelligence according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network stacks an input signal with a convolutional layer and a pooling layer sequentially. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

According to an embodiment, the classification neural network has a hidden layer and an output layer. The third artificial intelligence classification neural network for classifying the roughness level of the surface of the EEG measuring device consists of 5 or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of one output layer node of the classification neural network, and an output value for the classification of the surface of the EEG measuring device may be output to the output layer node. A detailed description of the third artificial intelligence will be described later with reference to FIG. 11 .

The server 100 may obtain the first output signal based on the result of the input of the third artificial intelligence.

In step S904 , the server 100 may generate a first classification result for the surface of the first EEG measuring device 131 based on the first output signal. Here, the first classification result may include information on which stage the surface of the first EEG measuring device 131 is classified.

For example, the server 100 checks the output value of the first output signal, and when the output value is 1, the surface of the first EEG measuring device 131 corresponds to step 1 and generates a first classification result, When the output value is 2, the first classification result may be generated as the surface of the first EEG measuring device 131 corresponds to the second stage. It can be seen that the surface of the first EEG measuring device 131 becomes rougher as the step increases.

In step S905 , the server 100 may analyze the first data to detect a crack generated on the surface of the first EEG measuring device 131 . When the crack is detected, only a portion confirmed to be larger than a certain size through image analysis may be detected as a crack generated on the surface of the first EEG measuring device 131 .

In step S906 , the server 100 may identify the cracks generated on the surface of the first EEG measuring device 131 for each region, and distinguish the normal region from the damaged region.

Specifically, the server 100 divides the first data into a plurality of areas such as a first area and a second area, and can check how many cracks are detected for each area, and the number of cracks is less than the first set value. The detected area may be divided into a normal area, and an area in which cracks greater than or equal to the first set value are detected may be classified as a damaged area. In this case, the first set value may be set differently depending on the embodiment.

In step S907 , the server 100 may extract second data from which the damaged area is deleted from the first data.

For example, the image in the first data consists of a first region, a second region, and a third region, wherein the first region is divided into a damaged region, and the second region and the third region are divided into a normal region. In this case, the server 100 may extract an image including only the second area and the third area as the second data.

The server 100 may generate a second input signal by encoding the second data.

Specifically, the server 100 may generate a second input signal by encoding pixels of the second data with color information. The color information may include, but is not limited to, RGB color information, brightness information, saturation information, and depth information. The server 100 may convert the color information into a numerical value, and may encode the second data in the form of a data sheet including this value.

The server 100 may input the second input signal to the fourth artificial intelligence learned in advance in the server 100 .

The fourth artificial intelligence according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks a convolutional layer and a pooling layer on an input signal. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

According to an embodiment, the classification neural network has a hidden layer and an output layer. The fourth artificial intelligence classification neural network for classifying the roughness level of the surface of the EEG measurement device consists of 5 or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of one output layer node of the classification neural network, and an output value for the classification of the surface of the EEG measuring device may be output to the output layer node. A detailed description of the fourth artificial intelligence will be described later with reference to FIG. 11 .

The server 100 may obtain a second output signal based on a result of the input of the fourth artificial intelligence.

In step S908 , the server 100 may generate a second classification result for the surface of the first EEG measuring apparatus 131 based on the second output signal. Here, the second classification result may include information on which stage the surface of the first EEG measuring device 131 is classified.

For example, the server 100 checks the output value of the second output signal, and when the output value is 1, the surface of the first EEG measuring device 131 corresponds to step 1 and generates a second classification result, When the output value is 2, the second classification result may be generated as the surface of the first EEG measuring device 131 corresponds to the second stage.

In step S909 , the server 100 may set a final classification result for the surface of the first EEG measuring device 131 based on the first classification result and the second classification result.

Specifically, when the first classification result and the second classification result are the same, the server 100 uses any one of the first classification result and the second classification result as the final classification result for the surface of the first EEG measuring device 131 . can be set.

The server 100 may determine whether a scratch has occurred on the surface of the first EEG measuring device 131 based on the final classification result for the surface of the first EEG measuring device 131 .

10 is a diagram for explaining learning of a first artificial intelligence and a second artificial intelligence according to an embodiment.

The first artificial intelligence and the second artificial intelligence may be components included in the server 100 and may be learned through the server 100 or a separate learning device.

The first artificial intelligence may be trained to generate learning content for each account user.

The second artificial intelligence may be trained to output a reference degree of concentration for classifying an upper section and a lower section of concentration of the learning content for each learning content.

Hereinafter, a process in which the first artificial intelligence and the second artificial intelligence are learned through the learning device will be described.

First, in step S1001, the learning apparatus may obtain training data and a label.

For the learning of the first artificial intelligence, the learning device determines each user's age, gender, region, learning achievement, subjects to be taken, subjects not taken, subjects to be taken, preferred subjects, non-preferred subjects, and the user when learning each learning content. It is possible to obtain a data set including the brain wave information of each of the training data. In addition, the learning apparatus may acquire, as a label corresponding to each training data, learning content for each user recommended by professional education consultants based on the basic information of each user.

For the learning of the second artificial intelligence, the learning device is a data set including the learning topic of each learning content, the learning difficulty, the average age of users who take the learning content, brain wave information of users who have taken the learning content, etc. can be obtained as each training data. In addition, the learning device sets the concentration out of the predefined +σ standard deviation from the average of the concentration level of the learning content of the high-ranking user group specified for each learning content as the upper threshold concentration level (a criterion for classifying the upper level of concentration), and sets the pre-defined -σ standard The concentration deviating from the deviation may be obtained as a label corresponding to each training data using the lower threshold concentration (a criterion for classifying the concentration subsection).

In step S1002, the learning apparatus may generate an input of artificial intelligence from the training data.

The learning apparatus may use the training data as it is as an input of artificial intelligence, or may generate an input of artificial intelligence after a normal process of removing unnecessary information from each training data.

In step S1003, the learning device may apply the input to the artificial intelligence.

The artificial intelligence included in the server 100 may be an artificial intelligence that is learned according to supervised learning. Artificial intelligence may be a convolutional neural network (CNN) or recurrent neural network (RNN) structure suitable for learning through supervised learning.

In step S1004, the learning apparatus may obtain an output from artificial intelligence.

The output of the first artificial intelligence may be a learning content recommendation for each account user. The output of the second artificial intelligence may be an inference of a threshold upper concentration level and a threshold lower concentration level that distinguishes an upper concentration section and a lower concentration level section for each learning content.

In step S1005, the learning apparatus may compare the output with the label. The process of comparing the output of the artificial intelligence corresponding to the inference and the label corresponding to the correct answer may be performed by calculating a loss function. As the loss function, a known mean squared error (MSE), cross entropy error (CEE), etc. may be used. However, the present invention is not limited thereto, and if it is possible to measure the deviation, error, or difference between the output of the artificial intelligence and the label, loss functions used in various artificial intelligence models may be used.

In step S1006, the learning apparatus may optimize the artificial intelligence based on the comparison value. The learning device can gradually match the AI output corresponding to inference and the label corresponding to the correct answer by updating the weight of the artificial intelligence nodes so that the comparison value becomes smaller and smaller, and through this, the artificial intelligence can be optimized to output an inference close to the correct answer. Specifically, the learning apparatus may optimize the artificial intelligence by repeating the process of resetting the weight of the artificial intelligence so that the loss function corresponding to the comparison value approaches the estimated value of the minimum value. For the optimization of AI, a known backpropagation algorithm, stochastic gradient descent, etc. may be used. However, the present invention is not limited thereto, and a weight optimization algorithm used in various neural network models may be used.

The learning device can learn artificial intelligence by repeating this process.

Through this, based on the basic information of the user, it is possible to learn the first artificial intelligence that outputs the learning content for the user.

In addition, based on the learning topic of the learning content, the learning difficulty, the average age of users who take the learning content, and the EEG information of users who have taken the learning content during that time, the concentration upper section and the lower concentration section of the learning content concentration for each learning content It is possible to train the second artificial intelligence that outputs the standard concentration level for classifying the .

11 is a diagram for explaining a third artificial intelligence and a fourth artificial intelligence according to an embodiment.

The artificial neural network 1100 according to an embodiment may be any one of a third artificial intelligence and a fourth artificial intelligence. In the case of the third artificial intelligence, by inputting the first input signal generated by the encoding of the first data, information on which stage of the roughness stage of the EEG measuring device is classified may be output. In the case of the fourth artificial intelligence, by receiving the second input signal generated by encoding the second data as an input, information on which stage of the roughness stage of the EEG measuring device is classified may be output.

Encoding according to an embodiment may be performed by storing color information for each pixel of an image in the form of a digitized data sheet, and the color information includes RGB color, brightness information, saturation information, and depth information of one pixel. can, but is not limited to.

According to an embodiment, the artificial neural network 1100 is composed of a feature extraction neural network 1110 and a classification neural network 1120, and the feature extraction neural network 1110 separates an EEG region and a background region from an image. may be performed, and the classification neural network 1120 may perform a task of determining whether the surface of the EEG measuring device is classified into which stage of the roughness stage in the image.

In the method for the feature extraction neural network 1110 to distinguish the region and the background region of the EEG measurement device, the change in each value of color information from the data sheet of the input signal encoding the image is at least 6 out of 8 pixels including one pixel. A bundle of pixels that are detected as having a change of 30% or more in , may be used as a boundary between the area of the EEG measurement device and the background area, but is not limited thereto.

The feature extraction neural network 1110 proceeds by sequentially stacking a convolutional layer and a pooling layer on the input signal. The convolution layer includes a convolution operation, a convolution filter, and an activation function. The calculation of the convolution filter is adjusted according to the matrix size of the target input, but a 9X9 matrix is generally used. The activation function generally uses, but is not limited to, a ReLU function, a sigmoid function, and a tanh function. The pooling layer is a layer that reduces the size of the input matrix, and uses a method of extracting representative values by tying pixels in a specific area. In general, the average value or the maximum value is often used for the calculation of the pooling layer, but is not limited thereto. The operation is performed using a square matrix, usually a 9x9 matrix. The convolutional layer and the pooling layer are repeated alternately until the corresponding input becomes small enough while maintaining the difference.

The classification neural network 1120 checks the surface of the region of the EEG measurement device separated from the background through the feature extraction neural network 1110, and checks whether it is similar to the predefined roughness step surface state, It is possible to determine whether the surface is classified into which level of roughness level. In order to compare the roughness step-by-step surface state, information stored in the database of the server 100 may be utilized.

The classification neural network 1120 has a hidden layer and an output layer, and consists of five or less hidden layers, including a total of 50 or less hidden layer nodes, and the activation function of the hidden layer is a ReLU function and a sigmoid function. and tanh functions, but is not limited thereto.

The classification neural network 1120 may include only one output layer node in total.

The output of the classification neural network 1120 is an output value of which stage of the roughness stage the surface of the EEG measuring apparatus is classified, and may indicate which stage of the roughness stage it corresponds to. For example, when the output value is 1, it may indicate that the surface of the EEG measuring device corresponds to the first step, and when the output value is 2, it may indicate that the surface of the EEG measuring device corresponds to the second step.

According to an embodiment, the artificial neural network 1100 may learn by receiving the first learning signal generated by the corrected correct answer input by the user when the user discovers a problem in the output according to the artificial neural network 1100 . The problem of output according to the artificial neural network 1100 may mean a case in which an output value classified into another stage among the roughness stages is output with respect to the surface of the EEG measurement apparatus.

The first learning signal according to an embodiment is created based on the error between the correct answer and the output value, and in some cases, SGD using delta, a batch method, or a method following a backpropagation algorithm may be used. The artificial neural network 1100 performs learning by modifying an existing weight according to the first learning signal, and may use momentum in some cases. A cost function can be used to calculate the error, and a cross entropy function can be used as the cost function. Hereinafter, the learning contents of the artificial neural network 1100 will be described with reference to FIG. 12 .

12 is a diagram for explaining a method of learning an artificial neural network according to an embodiment.

According to an embodiment, the learning apparatus may train the artificial neural network 1100 . The learning device may be a separate entity different from the server 100 , but is not limited thereto.

According to an embodiment, the artificial neural network 1100 includes an input layer to which training samples are input and an output layer to output training outputs, and may be learned based on a difference between the training outputs and the first labels. Here, the first labels may be defined based on a representative image registered for each roughness level. The artificial neural network 1100 is connected as a group of a plurality of nodes, and is defined by weights between the connected nodes and an activation function that activates the nodes.

The learning apparatus may train the artificial neural network 1100 using a Gradient Decent (GD) technique or a Stochastic Gradient Descent (SGD) technique. The learning apparatus may use a loss function designed by the outputs and labels of the artificial neural network 1100 .

The learning apparatus may calculate a training error using a predefined loss function. The loss function may be predefined with a label, an output, and a parameter as input variables, where the parameter may be set by weights in the artificial neural network 1100 . For example, the loss function may be designed in a Mean Square Error (MSE) form, an entropy form, or the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning apparatus may find weights affecting the training error by using a backpropagation technique. Here, the weights are relationships between nodes in the artificial neural network 1100 . The learning apparatus may use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning apparatus may update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning apparatus may obtain representative images 1201 of labeled training roughness steps from the database of the server 100 . The learning apparatus may obtain pre-labeled information on each of the representative images 1201 for each roughness stage, and the representative images 1201 for each roughness stage may be labeled according to the pre-classified roughness stage.

According to an embodiment, the learning apparatus may obtain 1000 labeled training roughness step-by-step representative images 1201, and based on the labeled training roughness step-by-step representative images 1201, first training roughness step-by-step vectors ( 1202) can be created. Various methods may be employed to extract the first training roughness step vectors 1202 .

According to an embodiment, the learning apparatus may obtain the first training outputs 1203 by applying the first training roughness step-by-step vectors 1202 to the artificial neural network 1100 . The learning apparatus may train the artificial neural network 1100 based on the first training outputs 1203 and the first labels 1204 . The learning apparatus may train the artificial neural network 1100 by calculating the training errors corresponding to the first training outputs 1203 and optimizing the connection relationship of nodes in the artificial neural network 1100 to minimize the training errors. .

13 is an exemplary diagram of a configuration of an apparatus according to an embodiment.

The device 1300 according to an embodiment includes a processor 1310 and a memory 1320 . The processor 1310 may include at least one of the devices described above with reference to FIGS. 1 to 12 , or perform at least one method described above with reference to FIGS. 1 to 12 . A person or organization using the apparatus 1300 may provide a service related to some or all of the methods described above with reference to FIGS. 1 to 12 .

The memory 1320 may store information related to the above-described methods or a program in which methods to be described below are implemented. The memory 1320 may be a volatile memory or a non-volatile memory.

The processor 1310 may execute a program and control the device 1300 . The code of the program executed by the processor 1310 may be stored in the memory 1320 . The device 1300 may be connected to an external device (eg, a personal computer or a network) through an input/output device (not shown), and may exchange data through wired/wireless communication.

The device 1300 may be used to train an artificial neural network or to use a learned artificial neural network. The memory 1320 may include a learning or learned artificial neural network. The processor 1310 may learn or execute an artificial neural network algorithm stored in the memory 1320 . The apparatus 1300 for learning the artificial neural network and the apparatus 1300 for using the learned artificial neural network may be the same or may be separate.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (3)

In a method of providing a learning service based on brain waves and blinking, performed by a server,
When the EEG of the first user is measured by the first EEG measurement device worn by the first user, from the first user terminal connected to the first EEG measurement device, the first EEG measured by the first EEG measurement device acquiring an EEG;
analyzing whether the state of the first user is good based on the brain wave of the first user;
when it is confirmed that the first user's condition is not good, providing the meditation content to the first user terminal, and controlling the meditation content to be reproduced on the screen of the first user terminal;
controlling a camera mounted on the first user terminal to operate and start shooting when it is confirmed that the first user's condition is good;
acquiring a face image of the first user from the first user terminal when the first user's face is photographed by the first user terminal;
recognizing the eyes of the first user from the face image of the first user and tracking the gaze of the first user;
as a result of tracking the gaze of the first user, checking whether the gaze of the first user is gazing at the screen of the first user terminal;
When it is confirmed that the gaze of the first user is gazing at the screen of the first user terminal, the learning content is provided to the first user terminal, and the learning content is controlled to be reproduced on the screen of the first user terminal step;
While the learning content is being reproduced in the first user terminal, while acquiring the brain wave of the first user from the first user terminal connected to the first EEG measuring device, the face image of the first user is obtained from the first user terminal obtaining from, recognizing the eyes of the first user from the face image of the first user, tracking the gaze of the first user, and measuring the Eye Aspect Ratio (EAR) of the first user;
When the reproduction of the learning content is completed, a change in the brain wave of the first user measured while the learning content is reproduced is checked, and a change in the EAR of the first user measured while the learning content is being reproduced checking an eye blinking state through , and checking a gaze state gazing at a screen through a gaze change of the first user tracked while the learning content is being played; and
Calculating the concentration of the first user on the learning content by analyzing the learning time based on the change in the brain wave of the first user, the blinking state, and the gaze state,
While the learning content is reproduced in the first user terminal, tracking the gaze of the first user and measuring the EAR of the first user comprises:
As a result of tracking the gaze of the first user, if it is confirmed that the gaze of the first user is not gazing at the screen of the first user terminal, the gaze of the first user deviates from the screen of the first user terminal measuring a gaze departure time from the time point;
measuring the drowsiness time from the time when the eyes of the first user are closed when it is confirmed that the eyes of the first user are closed as a result of measuring the EAR of the first user;
When it is confirmed that any one of the gaze departure time and the drowsiness time lasts longer than a preset first reference time, it is determined that the first user is unable to concentrate on learning because the first user is not learning through the learning content to do;
when it is determined that the first user is not learning, stopping the reproduction of the learning content, and controlling the first user terminal to output a notification message that requires concentration;
determining that a study guide for the first user is necessary when it is confirmed that any one of the gaze departure time and the drowsiness time has been maintained longer than a preset second reference time; and
When it is determined that the learning guidance for the first user is necessary, transmitting a learning guidance request message for requesting the learning guidance of the first user to the guardian terminal,
Calculating the concentration of the first user includes:
calculating a concentration level of the first user that changes according to a reproduction time of the learning content based on the change in the brain wave of the first user, the blinking state, and the gaze state;
classifying and classifying an upper concentration section, a normal concentration section, and a lower concentration section section for the learning content based on the concentration level of the first user;
checking whether a length of a first section among the concentration subsections is shorter than a preset first time interval;
determining that re-learning of the first section is not necessary when it is confirmed that the length of the first section is shorter than the first time interval;
when it is determined that the length of the first section is longer than the first time interval, checking whether the length of the first section is shorter than a preset second time interval;
determining that a test for the first section is necessary when it is confirmed that the length of the first section is shorter than the second time interval;
determining that re-learning of the first section is necessary when it is confirmed that the length of the first section is longer than the second time interval;
when it is determined that the test for the first section is necessary, controlling a test question for testing the learning content of the first section to be displayed on the screen of the first user terminal; and
When it is determined that re-learning of the first section is necessary, controlling the first section to be reproduced again on the screen of the first user terminal,
The learning content is generated based on the first artificial intelligence learned in advance,
The predefined reference concentration level for dividing the concentration upper section and the concentration lower section is set based on the pre-learned inference of the second artificial intelligence,
The first artificial intelligence receives as inputs the subjects to be learned for each account user, learning achievement so far, learning achievement change trend, and EEG information when using previous learning content, learning content for each account user create,
The second artificial intelligence receives as inputs a learning topic, learning difficulty, average age of users who receive learning content, and EEG information of users who receive learning content for each learning content, and a high threshold for classifying the upper section of concentration Outputting the reference concentration level including the concentration level and the sub-threshold concentration level for dividing the concentration sub-section for each of the learning contents,
A method of providing EEG and blink-based learning services. The method of claim 1,
Measuring the EAR of the first user while acquiring the brainwave of the first user while the learning content is being reproduced in the first user terminal comprises:
when the learning content is reproduced in the first user terminal, checking whether the brain wave of the first user and the EAR of the first user are being measured;
when it is confirmed that both the brain wave of the first user and the EAR of the first user are being measured, adjusting the synch between the brain wave of the first user and the EAR of the first user;
when it is confirmed that any one of the brain wave of the first user and the EAR of the first user is not being measured, controlling the reproduction of the learning content to be stopped after a preset waiting time; and
When it is confirmed that neither the brain wave of the first user nor the EAR of the first user is being measured, including controlling the reproduction of the learning content to immediately stop,
A method of providing EEG and blink-based learning services. delete

Images