KR102545747B1 - Method and apparatus for determining degree of attention of user - Google Patents

Abstract

In order to determine the level of attention of a user, content is output through a user terminal, reactions of a user observing the content are continuously received, biomarkers are generated based on the reaction, and the level of attention of the user is determined through a plurality of pre-trained models based on the biomarkers.

Description

Method and apparatus for determining the degree of user's attention

The technical field relates to technology for determining a user's attentional level, and more particularly, to an apparatus and method for providing content to a user and determining a user's attentional level based on a user's reaction to the provided content.

ADHD (attention deficit/hyperactivity disorder) is a disease in which symptoms of inattention, hyperactivity, and impulsivity are mainly manifested, and it is the most common and serious neurodevelopmental disorder that induces negative experiences in children and adolescents. The prevalence in primary and secondary school students is 6 to 8 percent. Recently, adult ADHD patients are rapidly increasing, and it is estimated that about 3% (about 1.2 million people) of the adult population are potential patients. It is analyzed that the rapid increase in the number of adult ADHD patients is due to failure to treat them appropriately due to failure to detect them early in childhood and adolescence.
Prior Document Number: Korean Patent Publication No. 10-2019-0140864 (2019.12.20. Publication date)

An embodiment may provide a device and method for determining a user's level of attention.

An embodiment may provide an apparatus and method for determining a user's attentional level based on the user's eye movements.

According to an embodiment, a method for determining a user's attentional level, performed by an electronic device, includes outputting pre-produced target content to determine the user's attentional level through a display of a user terminal - the target content is content in which at least one of the shape, color, and position of at least one object of the content changes over time; continuously receiving coordinate values on the display at which eyes of the user observing the target content gaze; generating one or more target biomarkers for predefined eye movements of the user with respect to the target content based on the coordinate values; and determining the level of attention of the user based on the target biomarkers, and the determined level of attention may be output through the user terminal.

The target content may include an animation in which at least one of the shape, color, and position of at least one object changes, and a voice or text instruction indicating an action to the user.

The continuously received coordinates are determined by the user terminal, and detecting, by the user terminal, a first eye of the user in a first user image; determining a first gaze direction of the user based on the first eye; determining a first coordinate value on the display based on the first gaze direction; and associating the first coordinate value with a first time stamp of the first user image, thereby continuously determining the coordinate value.

The target content is content for detecting pro-saccades of the user, and the target biomarkers predefined for the target content include a fixation time max, an average fixation time. One of (fixation mean), average of times taken to transition from previous saccade to next saccade (saccade latency mean), fixation count, and average fixation to saccade velocity (saccade velocity mean). may contain more than

The target content is content for detecting anti-saccades of the user, and the target biomarkers predefined for the target content include a standard deviation (saccadeStd) of times when saccades lasted, the At least one of the total time the user viewed the target content (screen duration total), the standard deviation of gaze fixation times (fixationStd), the number of fixation times (fixation count), and the mean of x coordinate values (coordX mean) among the coordinate values. can include

The target content is content for detecting memory guided saccades of the user, and the target biomarkers predefined for the target content include fixation time max, fixation time It may include one or more of a fixation duration, a fixation mean, an average of y coordinate values among the coordinate values (coordY mean), and a total time of viewing the target content (screen duration total). can

The target content is content for detecting change detection of the user, and the target biomarkers predefined for the target content include total elapsed times, The standard deviation of the times it took to switch from the previous saccade to the next saccade (saccade latencyStd), the average of the times it took to switch from the previous saccade to the next saccade (saccade latency mean), the average of the times spent looking at the target content (screen duration mean), gaze fixation average time (fixation mean), total time of looking at the target content (screen duration total), and one or more of the total of the times taken to switch from the previous saccade to the next saccade (saccade latency total) can include

The target content is content for a stroop test of the user, and the target biomarkers predefined for the target content include screen duration total, number of saccades (saccade count), the sum of the times saccades lasted (saccade duration), the average of x coordinate values among the coordinate values (coordX mean), the standard deviation of the times saccades lasted (saccadeStd), the target content viewed It may include one or more of a screen duration mean, and a fixation mean.

The electronic device sequentially outputs a plurality of different target contents to a user through the user terminal - the plurality of target contents include first target contents and second target contents - and generates the target biomarkers. The doing may include generating first sub-biomarkers based on first coordinate values received for the first target content; generating second sub biomarkers based on second coordinate values of the second target content; and generating a target biomarker set based on the first sub biomarkers and the second sub biomarkers.

The generating of the target biomarkers may include generating original target biomarkers for eye movements of the user predefined for the target content based on the coordinate values; and generating the target biomarkers corresponding to the original target biomarkers by applying a preset normalization scheme to each of the original target biomarkers.

Determining the level of attention of the user based on the target biomarkers may include inputting the target biomarkers to each of a plurality of models trained in different ways to determine the level of attention of the user - the plurality of models. the models of include a first model and a second model; and determining the degree of attention of the user based on a first result of the first model for the target biomarkers and a second result of the second model for the target biomarkers. .

The first model is K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, Random Forest Classifier, Quadratic Discriminant Analysis, Light Gradient Boosting Machine, Ada Boost Classifier, Ridge Classifier, SVM-Linear Kernel, Decision Tree Classifier, Gradient Boosting Classifier, Linear A model trained based on any one of Discriminant Analysis and Naive Bayes, and the second model may be a different model from the first model.

The first model is a model trained based on any one of K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, and Random Forest Classifier, and the second model is K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, and Random Forest Classifier. Among the classifiers, it may be a model different from the first model.

According to an embodiment, an electronic device for determining a level of attention of a user may include a memory in which a program for determining a level of attention of a user is recorded; and a processor that executes the program, wherein the program outputs pre-produced target content to determine a user's attention level through a display of a user terminal - the target content is the content as time passes. At least one of the shape, color, and position of at least one object of is content that changes -; continuously receiving coordinate values on the display at which eyes of the user observing the target content gaze; generating one or more target biomarkers for predefined eye movements of the user with respect to the target content based on the coordinate values; and determining the level of attention of the user based on the target biomarkers, and the determined level of attention may be output through the user terminal.

An apparatus and method for determining a user's level of attention may be provided.

An apparatus and method for determining a user's attentional level based on a user image including the user's eyes may be provided.

1 is a configuration diagram of a system for determining a user's attentional level according to an example.
2 illustrates images output to a user terminal to determine a user's attentional level according to an example.
3 is a configuration diagram of an electronic device for determining a user's attentional level according to an exemplary embodiment.
4 is a flowchart of a method for determining a user's attentional level according to an exemplary embodiment.
5 illustrates content for adjusting a user's gaze according to an example.
6 illustrates content for detecting a user's pro saccade according to an example.
7 illustrates content for detecting a user's anti-saccade according to an example.
8 illustrates content for detecting a user's memory-induced saccade according to an example.
9 illustrates content for detecting a change detection of a user according to an example.
10 illustrates content for a stroop test of a user according to an example.
11 is a flowchart of a method of continuously determining coordinate values on a display at which eyes of a user gaze, according to an example.
12 is a flowchart of a method of generating target biomarkers based on raw target biomarkers according to an example.
13 is a flowchart of a method of generating a target biomarker set based on a plurality of sub-biomarkers according to an example.
14 is a flowchart of a method of determining a user's attentional level using a plurality of models according to an example.
15 illustrates a method of determining a user's attentional level using a plurality of models according to an example.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these examples. Like reference numerals in each figure indicate like elements.

Various changes may be made to the embodiments described below. The embodiments described below are not intended to be limiting on the embodiments, and should be understood to include all modifications, equivalents or substitutes thereto.

Terms used in the examples are used only to describe specific examples, and are not intended to limit the examples. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

In addition, in the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

1 is a configuration diagram of a system for determining a user's attentional level according to an example.

According to one aspect, the system for determining the level of attention of the user may include an electronic device 110 for determining the level of attention of the user, a user terminal 120 outputting content, and a monitoring terminal 130 of a medical institution. there is. For example, the electronic device 110 may be a server.

The electronic device 110 may provide pre-produced content to the user terminal 120 to determine the user's level of attention. For example, the content may be content for obtaining user images including the user's eyes. Content for obtaining user images will be described in detail with reference to FIGS. 5 to 10 below.

According to an embodiment, the user terminal 120 may communicate with the electronic device 110 by being connected offline or online. The electronic device 110 provides content to the user terminal 120, and the user terminal 120 outputs the content to the user through a display. For example, the user terminal 120 may capture user images as reactions to content through a camera, continuously determine coordinate values on a display that the user's eyes gaze at based on the captured user images, and determine the determined Coordinate values are transmitted to the electronic device 110 .

According to an embodiment, the electronic device 110 may determine the level of attention of the user based on the coordinate values that the user gazes at, and transmit the determined level of attention to the user terminal 120 .

According to an embodiment, the user terminal 120 may be a mobile terminal such as a tablet or a smart phone. When the user terminal 120 is a mobile terminal, the user is not limited by time and place and can measure the level of attention at low cost.

Below, a method for determining the level of attention of the user will be described in detail with reference to FIGS. 2 to 15 .

2 illustrates images output to a user terminal to determine a user's attentional level according to an example.

The images 210 to 240 below may be images of an application for determining a user's level of attention. For example, a user or manager of the electronic device 110 may create and distribute an application, and the user may execute the application through the user terminal 120 .

According to one embodiment, the first image 210 is a start screen of an application.

According to an embodiment, the second image 220 represents functions supported by applications.

According to one embodiment, the third image 230 is an example of content provided to the user. A plurality of contents may be sequentially provided to the user.

According to an embodiment, the fourth image 240 represents the determined level of attention of the user. For example, the first ADHD level, the second ADHD level, and the third ADHD level determined as the degree of attention of the user may be output. In the fourth image 240, a comprehensive judgment may be output together with the degree of attention to individual diseases.

3 is a configuration diagram of an electronic device for determining a user's attentional level according to an exemplary embodiment.

The electronic device 300 includes a communication unit 310 , a processor 320 and a memory 330 . For example, the electronic device 300 may be the electronic device 110 described above with reference to FIG. 1 .

The communication unit 310 is connected to the processor 320 and the memory 330 to transmit and receive data. The communication unit 310 may transmit/receive data by being connected to another external device. Hereinafter, the expression “transmitting and receiving “A” may indicate transmitting and receiving “information or data indicating A”.

The communication unit 310 may be implemented as circuitry within the electronic device 300 . For example, the communication unit 310 may include an internal bus and an external bus. As another example, the communication unit 310 may be an element that connects the electronic device 300 and an external device. The communication unit 310 may be an interface. The communication unit 310 may receive data from an external device and transmit the data to the processor 320 and the memory 330 .

The processor 320 processes data received by the communication unit 310 and data stored in the memory 330 . A “processor” may be a data processing device implemented in hardware having circuitry having a physical structure for executing desired operations. For example, desired operations may include codes or instructions included in a program. For example, a data processing unit implemented in hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , Application-Specific Integrated Circuit (ASIC), and Field Programmable Gate Array (FPGA).

Processor 320 executes computer readable code (eg, software) stored in memory (eg, memory 330 ) and instructions invoked by processor 320 .

The memory 330 stores data received by the communication unit 310 and data processed by the processor 320 . For example, the memory 330 may store a program (or application or software). The stored program may be a set of syntaxes that are coded and executable by the processor 320 to determine the level of attention of the user.

According to one aspect, the memory 330 may include one or more of volatile memory, non-volatile memory and random access memory (RAM), flash memory, a hard disk drive, and an optical disk drive.

The memory 330 stores a command set (eg, software) for operating the electronic device 300 . A set of instructions for operating the electronic device 300 is executed by the processor 320 .

The communication unit 310, the processor 320, and the memory 330 will be described in detail with reference to FIGS. 4 to 15 below.

4 is a flowchart of a method for determining a user's attentional level according to an exemplary embodiment.

Steps 410 to 440 below are performed by the electronic device 300 described above with reference to FIG. 3 .

In step 410, the electronic device 300 outputs pre-produced content to determine the level of attention of the user through the display of the user terminal (eg, the user terminal 120). Content is output to the user terminal, and the user reacts to the content.

The user terminal may generate user images as the reaction using a camera. For example, the camera may generate user images including the user's eyes at preset intervals. The generated user images may be in the form of data files.

According to an embodiment, a plurality of contents may be provided to a user, and user images for each of the plurality of contents may be created.

According to an embodiment, contents for generating user images may be contents prepared in advance to measure saccade and anti-saccade as eye movements of the user. Each type of eye movement can be named as a biomarker, and the types of eye movement are shown in [Table 1] below.

type of eye movement explanation fixation count number of fixations saccade count number of saccades screen duration mean Average time spent looking at content screen duration total Total time spent looking at content fixation duration Total sum of gaze fixation times fixation mean Average gaze fixation time fixationStd Standard deviation of fixation times fixation time max. maximum fixation time saccade duration The total amount of time the saccades lasted saccade mean Average number of times the saccade lasted saccadeStd Standard deviation of times the saccades lasted saccade time max Maximum length of time saccade lasted saccade latency total The sum of the times it took to transition from the previous scade to the next scade saccade latencyStd Standard deviation of times taken to transition from previous saccade to next saccade saccade latency mean Average time taken to transition from previous scade to next scade saccade velocity mean Average rate of transition from gaze fixation to saccade coordX mean Average of x coordinate values among gaze coordinate values coordY mean Average of y coordinate values among gaze coordinate values hit count number of correct answers total elapsed times Total amount of time spent solving the problem

For example, a first user image set may be generated at a preset cycle while the first content is output.

The user terminal continuously determines coordinate values on the display at which the eyes of the user gaze based on the user images. For example, coordinate values on the display may be determined in units of pixels or in units of regions. A method of determining a coordinate value on a display will be described in detail with reference to FIG. 11 below.

According to an embodiment, the electronic device 300 may control the user terminal so that an additional application for determining coordinate values is executed in the user terminal. For example, the additional application may generate user images of a user observing content using a camera of a user terminal, and determine coordinate values on a display at which eyes of the user gaze in the user images.

In step 420, the electronic device 300 continuously receives a user's reaction observing the content from the user terminal. For example, the electronic device 300 may receive, from the user terminal, continuous coordinate values on the display where the eyes of the user observing the content gaze.

According to one embodiment, when a plurality of contents are produced, steps 410 and 420 may be repeatedly performed. For example, as steps 410 and 420 are repeatedly performed, user coordinate value sets for each of a plurality of contents are received.

According to one embodiment, each of a plurality of contents may be prepared in advance to observe eye movements of different users. For example, the first content may be content for detecting pro-saccades of the user. For example, the second content may be content for detecting user's anti-saccades. For example, the third content may be content for detecting memory guided saccades of the user. For example, the fourth content may be content for detecting user's change detection. For example, the fifth content may be content for a stroop test of the user.

According to an embodiment, first to fifth user coordinate value sets for the first to fifth contents may be received.

In step 430, the electronic device 300 generates biomarker information based on the received reaction. For example, biomarker information may include one or more biomarkers.

According to an embodiment, the electronic device 300 may generate one or more biomarkers for eye movements of the user based on the received coordinate values. For example, different biomarkers may be generated for each content. For example, different biomarkers may be predefined for each content.

The types of target biomarkers and specific ratios of the target biomarkers generated for each content disclosed below are derived by experiments based on preset conditions, and the results may vary depending on the conditions set for the experiments. can The present invention should not be construed as being limited to the disclosed types of target biomarkers and specific ratios of the target biomarkers.

According to an embodiment, for the first content for detecting the user's pro-saccades, fixation time max, fixation average time, fixation mean, previous saccade to next One or more of the average of times taken to switch to saccade (saccade latency mean), fixation count (fixation count), and average speed of conversion from fixation to saccade (saccade velocity mean) may be predefined as target biomarkers. can For example, among various types of eye movements, a specific type of eye movement that meets the purpose of detecting a user's pro saccade may be defined as a target biomarker.

According to an embodiment, a ratio of each of the target biomarkers that can well represent the purpose of the first content may be determined in advance. For example, 20.8092% for fixation time max, 20.6771% for fixation mean, saccade latency mean Ratios of 19.9670% for fixation count, 19.4040% for fixation count, and 19.1427% for saccade velocity mean can be determined beforehand.

According to an embodiment, with respect to the second content for detecting the user's anti-saccades, the standard deviation of the times the saccades were maintained (saccadeStd), the total time the user viewed the second content (screen) duration total), standard deviation of gaze fixation times (fixationStd), fixation count, and average of x coordinate values among gaze coordinate values (coordX mean). One or more may be predefined as target biomarkers. . Target biomarkers predefined for the second content may be derived as a result of research through data verification. For example, among various types of eye movements, a specific type of eye movement matching the purpose of detecting the user's anti-saccade may be defined as a target biomarker.

According to an embodiment, a ratio of each of the target biomarkers that can well indicate the purpose of the second content may be determined in advance. For example, 21.2365% for the standard deviation of times the saccades lasted (saccadeStd), 19.9869% for the total time the user looked at the second content (screen duration total), and the standard deviation of gaze fixation times (fixationStd) Ratios of 19.9060% for the fixation count, 19.4984% for the fixation count, and 19.3721% for the coordX mean of the gaze coordinate values may be predetermined.

According to an embodiment, an error rate may be further determined as a target biomarker for the second content. For example, when the user is requested to move the gaze to a position different from that of the target object displayed on the screen of the second content (eg, a position opposite to the reference point), but the user actually moves the gaze to the target object, the gaze error can be determined. When the second content is composed of a plurality of repetitions of tasks (eg, 48 times), a ratio of tasks for which gaze errors are determined among total tasks performed may be determined as an error rate.

According to an embodiment, for the third content for detecting the user's memory guided saccades, a fixation time max, a total sum of fixation times (fixation duration), and fixation One or more of a fixation mean, an average of y-coordinate values among gaze coordinate values (coordY mean), and a total time for viewing the third content (screen duration total) may be predefined as target biomarkers. For example, among various types of eye movements, a specific type of eye movement that meets the purpose of detecting a user's memory inducing saccade may be defined as a target biomarker. For example, the time between when the reference point displayed on the screen of the third content disappears and when the user starts to move his gaze to the memorized location of the target object may be determined as the delay time, and the target biomarker related to the delay time may be can be defined

According to an embodiment, a ratio of each of the target biomarkers that can well indicate the purpose of the third content may be determined in advance. For example, 20.7343% for the fixation time max, 20.3700% for the total fixation duration, 20.1529% for the fixation mean, among the gaze coordinate values. Ratios of 19.9199% for the average of y-coordinate values (coordY mean) and 18.8229 for the total time of viewing the third content (screen duration total) may be determined in advance.

According to one embodiment, for the fourth content for detecting the change detection of the user, the total elapsed times spent to solve the problem, the previous scade to the next scade is switched Standard deviation of times taken (saccade latencyStd), average of times it took to switch from the previous saccade to the next saccade (saccade latency mean), average of times spent looking at the fourth content (screen duration mean), average gaze fixation time (fixation One or more of the mean), the total time of viewing the fourth content (screen duration total), and the total of times taken to switch from the previous scade to the next saccade (saccade latency total) may be predefined as target biomarkers. . For example, among various types of eye movements, a specific type of eye movement that meets the purpose of detecting a user's change detection may be defined as a target biomarker.

According to an embodiment, a ratio of each of the target biomarkers that can well represent the purpose of the fourth content may be determined in advance. For example, 24.0280% for the total elapsed times spent solving a problem, 13.2601% for the standard deviation of the times elapsed from one saccade to the next (saccade latencyStd), and 13.2601% for the total elapsed times 13.2462% for the average of times it took to switch from one cade to the next saccade (saccade latency mean), 12.7526% for the average of times spent looking at the fourth content (screen duration mean), and for the average fixation time (fixation mean) The ratios of 12.5573%, 12.1167% for the screen duration total, and 12.0392% for the total time taken to switch from the previous saccade to the next saccade (saccade latency total) will be predetermined. can

According to an embodiment, with respect to the fifth content for the user's stroop test, the screen duration total of viewing the fifth content, the number of saccades (saccade count), and the duration of the saccades The sum of the times (saccade duration), the average of the x-coordinate values among the gaze coordinate values (coordX mean), the standard deviation of the times that the saccades lasted (saccadeStd), the average of the times of viewing the fifth content (screen duration mean), and One or more of fixation mean times may be predefined as target biomarkers. For example, among various types of eye movements, a specific type of eye movement that meets the purpose of the user's Stroop test may be defined as a target biomarker.

According to an embodiment, a ratio of each of the target biomarkers that can well represent the purpose of the fifth content may be determined in advance. For example, 21.7487% for screen duration total, 14.7708% for saccade count, and 14.3486% for saccade duration total. , 12.8140% of the average of the x coordinate values (coordX mean) of the gaze coordinate values, 12.4422% of the standard deviation (saccadeStd) of the times that the saccades lasted, and the average of the times of viewing the fifth content (screen duration mean) Ratios of 12.1322% for fixation mean and 11.7435% for fixation mean may be predetermined.

In step 440, the electronic device 300 may determine the level of attention of the user based on the biomarkers.

According to an embodiment, the electronic device 300 may determine the user's attentional level by inputting the biomarkers into a previously trained attentional level classification model.

According to an embodiment, the attention level classification model may output a target attention level among a plurality of attention levels based on input biomarkers. For example, the plurality of attentional degrees may include a normal stage and an ADHD stage. As another example, the plurality of levels of attention may further include an attention step.

According to an embodiment, the attention level classification model may output a probability of the target attention level together with the target attention level.

According to an embodiment, the electronic device 300 may input biomarkers to each of a plurality of models trained in different ways to determine the level of attention of the user. For example, the plurality of models may include a first model, a second model, a third model, and a fourth model. Each of the plurality of models is K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, Random Forest Classifier, Quadratic Discriminant Analysis, Light Gradient Boosting Machine, Ada Boost Classifier, Ridge Classifier, SVM-Linear Kernel, Decision Tree Classifier, Gradient Boosting Classifier, It may be a model trained based on any one of Linear Discriminant Analysis and Naive Bayes, and each of a plurality of models may be different from each other. A plurality of models are described in detail with reference to FIGS. 14 and 15 below.

For example, an attention level classification model may be pre-trained based on a neural network, but the structure and training method of the model are not limited to the described embodiments.

According to an embodiment, the electronic device 300 may determine a final user's level of attention based on output results of a plurality of models. For example, the user's level of attention may be determined based on an average value of output results of a plurality of models. A method of determining the level of attention of the user based on the results output by the plurality of models will be described in detail below with reference to FIGS. 14 and 15 .

According to an embodiment, after step 440 is performed, the electronic device 300 may output the determined level of attention through the user terminal.

5 illustrates content for adjusting a user's gaze according to an example.

According to an embodiment, the content 500 provided to the user may be content for calibrating coordinate values on the display at which the user's eyes gaze.

In the first step 510, the purpose of the content 500 to adjust the gaze and instructions for the content 500 may be delivered to the user. For example, it may be instructed to fix the position of the user's face and to move only the user's pupils to gaze at the output object.

In each of the second step 520, the third step 530, the fourth step 540, the fifth step 550, and the sixth step 560, objects 521, 531, 541, 551, 561) may be output. Coordinate values on the display where the user's eyes gaze at the objects 521 , 531 , 541 , 551 , and 561 may be transmitted from the user terminal to the electronic device 300 . The electronic device 300 may adjust the user's line of sight by calculating a transformation relationship between the received coordinate values and coordinate values preset for the objects 521 , 531 , 541 , 551 , and 561 .

According to an embodiment, the entire area of the display may be pre-divided into a plurality of areas. The plurality of areas may be set differently according to the type of content to be provided. For example, a plurality of areas may be set differently depending on whether the content is provided on a horizontal screen or a vertical screen. The objects 521, 531, 541, 551, and 561 may be output to any one of a plurality of regions.

In a seventh step 570, it may be output that the line of sight has been adjusted.

6 illustrates content for detecting a user's pro saccade according to an example.

According to an embodiment, the content 600 provided to the user may be content for measuring pro-saccade as eye movements of the user. The content 600 may include text or voice as instructions for eye movement of the user with respect to the content 600 . The content 600 can be largely composed of four steps.

In the first step 610, an object 611 for inducing a user's gaze 615 may be output within the content 600. For example, the object 611 may be located at the center of the display.

In the second step 620, the object 611 located in the content 600 may disappear.

In the third step 630, the object 631 may be output at a position different from the position at which the object 611 was output within the content 600. The user may move the gaze from the gaze 615 to the gaze 635 to observe the object 631 .

In the fourth step 640, the object 631 located in the content 600 may disappear.

While the content 600 is output on the display, user images of a user observing the content 600 may be generated, and coordinate values on the display at which the user's eyes gaze may be determined based on the user images.

Steps 610 to 640 may be repeatedly performed a preset number of times, and each time the steps 610 to 640 are repeated, the position of the object 631 output in step 630 may be changed. One-time performance of steps 610 to 640 can be understood as a one-time task. The content 600 may be prepared in advance so that the positions of the object 631 change.

7 illustrates content for detecting a user's anti-saccade according to an example.

According to an embodiment, the content 700 provided to the user may be content for detecting anti-saccade as the user's eye movement. The content 700 may include text or voice as instructions for eye movement of the user with respect to the content 700 . The content 700 can be largely composed of five steps.

In the first step 710, an object 711 for inducing a user's gaze 715 may be output within the content 700. The object 711 may be positioned at the center of the display.

In the second step 720, the object 711 located in the content 700 may disappear.

In the third step 730, the object 731 may be output at a position different from the position at which the object 711 was output within the content 700. For example, the user may move the gaze from the gaze 735 of the object 731 to the gaze 736 in order to observe the opposite position 732 of the object 731 . As another example, the user may move the gaze from the gaze 715 to the gaze 736 to observe the opposite position 732 of the object 731 .

In the fourth step 740, the object 742 may be output to the opposite position 732. The object 742 may be a correct answer for informing the user of the opposite location 732 .

In the fifth step 750, the object 742 located in the content 700 may disappear.

While the content 700 is output on the display, user images of a user observing the content 700 may be generated, and based on the user images, coordinate values on the display at which the user's eyes gaze may be determined.

Steps 710 to 750 may be repeatedly performed a preset number of times, and the positions of the objects 731 and 742 output in steps 730 and 740 may be changed each time they are repeated. there is. Performing steps 710 to 750 once may be understood as a single task. The content 700 may be prepared in advance so that the positions of the object 731 and the object 742 change.

8 illustrates content for detecting a user's memory-induced saccade according to an example.

According to an embodiment, the content 800 provided to the user may be content for measuring gaze memory as eye movements of the user. The content 800 may include text or voice as instructions for eye movement of the user with respect to the content 800 . The content 800 can be largely composed of five steps.

In the first step 810, an object 811 for inducing a user's gaze may be output within the content 800. For example, the object 811 may be located at the center of the display.

In the second step 820, the object 822 may be additionally output while the output of the object 811 is maintained.

In the third step 830, the object 822 may disappear while the output of the object 811 is maintained.

In the fourth step 840, the object 811 located in the content 800 may disappear. In advance, the user may be instructed to gaze at the position 841 where the object 822 was output.

In a fifth step 850 , an object 851 may be output at the location 841 . The object 851 may be a correct answer for informing the user of the location 841 .

While the content 800 is output on the display, user images of a user observing the content 800 may be created, and based on the user images, coordinate values on the display at which the user's eyes gaze may be determined.

Steps 810 to 850 may be repeatedly performed a preset number of times, and the locations of the objects 822 and 851 output in steps 820 and 850 may be changed each time they are repeated. there is. One-time performance of steps 810 to 850 can be understood as a one-time task. The content 800 may be prepared in advance so that the positions of the object 822 and the object 851 are changed.

9 illustrates content for detecting a change detection of a user according to an example.

According to an embodiment, the content 900 provided to the user may be content for detecting change detection as the user's eye movement. The content 900 may include text or voice as instructions for eye movement of the user with respect to the content 900 . The content 900 can be largely composed of three steps.

In the first step 910, an image 920 for inducing a user's gaze may be output within the content 900. For example, image 920 may include object 922 . After the image 920 is output for a preset time, the blinking image 930 may be output. The flickering image 930 may be an image for arousing the user's gaze or may be a blank image. After the blinking image 930 is output for a preset time, the image 940 may be output. The image 940 may be an image obtained by modifying a part of the image 920 . For example, the image 940 may include another object 942 instead of the object 922 .

According to an embodiment, although the blinking image 930 is shown to be output between the output of the image 920 and the output of the image 940, the blinking image 930 is not output, and the output of the image 920 and Output of the image 940 may be performed alternately.

The output of the images 920, 930, and 940 may be repeated for a preset period of time.

When the user finds a difference between the image 920 and the image 940, the user may select (eg, touch input) the difference between the image 920 and the image 940. If the user's selection is correct, a second step 950 may be performed.

The second step 950 may be performed even if the user does not find a difference between the image 920 and the image 940 after the elapse of the preset time.

In the second step 950 , the image 920 or the object 922 of the image 940 or a correct answer object 952 for the object 942 may be output in the content 900 . The answer object 952 may be a correct answer for informing the user of the difference between the image 920 and the image 940 .

In the third step 960, a notification that an image different from the image 920 is output in the content 900 may be output.

While the content 900 is output on the display, user images of a user observing the content 900 may be generated, and based on the user images, coordinate values on the display at which the user's eyes gaze may be determined.

Steps 910 to 930 may be repeatedly performed a preset number of times, and objects 920 and 940 output in step 910 may be changed each time they are repeated. One-time performance of steps 910 to 930 can be understood as a one-time task.

10 illustrates content for a stroop test of a user according to an example.

According to an embodiment, the content 900 provided to the user may be content for providing an environment in which it is difficult for the user to concentrate and to perform suggested instructions. The content 1000 may include text or voice as instructions for eye movement of the user with respect to the content 1000 . The content 1000 can be largely composed of two stages.

In the first step 1010, an object 1011 for inducing a user's gaze may be output within the content 1000. The object 1011 may be positioned at the center of the display.

In the second step 1020, an object 1021 having a specific color and text 1022 representing the color may be output within the content 1000. The color of the object 1021 and the color indicated by the text 1022 may be different. For example, the color of the object 1021 may be red.

Additionally, texts 1023 to 1026 representing colors may be output within the content 1000 . A user may select text 1024 representing the color of the object 1021 .

While the content 1000 is being output to the display, user images of a user observing the content 1000 may be generated, and based on the user images, coordinate values of the user's eyes on the display may be determined.

Steps 1010 and 1020 may be repeatedly performed a preset number of times, and whenever repeated, the object 1021 having a specific color output in step 1020 and the text 1022 representing the color may be changed. there is. Performing steps 1010 to 1020 once may be understood as a single task.

11 is a flowchart of a method of continuously determining coordinate values on a display at which eyes of a user gaze, according to an example.

According to an embodiment, after the step 410 described above with reference to FIG. 4 is performed, the following steps 1110 to 1130 may be performed by a user terminal (eg, the user terminal 120 of FIG. 1). there is. For example, the electronic device 300 may control the user terminal so that the user terminal performs steps 1110 to 1130. Steps 1110 to 1130 may be performed by an additional application installed on the user terminal.

In operation 1110, the user terminal may continuously generate user images including eyes of a user observing content through a camera of the user terminal. For example, a user image set (eg, a first user image set) may be generated at a predetermined cycle while content (eg, first content) is output.

According to one embodiment, step 1120 may include steps 1122 to 1128.

In operation 1122, the user terminal may detect a first eye of a user in a first user image among user images received for content.

In operation 1124, the user terminal may determine a first gaze direction of the user based on the first eye. For example, the position of the user's eyes in the 3D space may be determined based on the coordinate values of the first eyes in the first user image, and the first gaze direction may be determined based on the position of the eyes and the direction of the pupil. Software or the like that determines the user's first gaze direction based on the user's image may be used, and is not limited to the described embodiment.

In operation 1126, the user terminal may determine a first coordinate value on the display based on the first gaze direction. For example, the first coordinate value may be a pixel coordinate value on the display. As another example, the first coordinate value may correspond to one of areas obtained by dividing the entire area of the display into a plurality of areas.

In step 1128, the user terminal may associate the first coordinate value and the first time stamp of the first user image. The first time stamp may indicate a time when the first user image was created.

Steps 1122-1128 may be performed iteratively for each of the user images. As steps 1122 to 1128 are repeatedly performed, coordinate values for content may be generated, and a time stamp may be associated with each coordinate value.

In step 1230, the user terminal may transmit the determined coordinate values to the electronic device 300.

12 is a flowchart of a method of generating target biomarkers based on raw target biomarkers according to an example.

According to one embodiment, step 430 described above with reference to FIG. 4 may include steps 1210 and 1220 below.

In step 1210, the electronic device 300 may generate one or more raw target biomarkers for predefined eye movements of the user with respect to the target content based on the coordinate values received from the user terminal and gazed at by the user. .

In operation 1220, the electronic device 300 may generate target biomarkers corresponding to the original target biomarkers by applying a preset normalization scheme to each of the one or more original target biomarkers. For example, a scaler can be used to normalize raw target biomarkers. For example, the scaler may include a min-max scaler, a max-abs scaler, a standard scaler, and a robust scaler. Through normalization, even when the raw target biomarker exhibits an abnormal value, it can be input into the model.

According to an embodiment, the scaler generates a first target biomarker corresponding to the first original target biomarkers by applying a first normalization method to the first original target biomarker, and generates a first target biomarker corresponding to the second original target biomarker. A second target biomarker corresponding to the second original target biomarkers may be generated by applying the second normalization method.

13 is a flowchart of a method of generating a target biomarker set based on a plurality of sub-biomarkers according to an example.

According to one embodiment, step 430 described above with reference to FIG. 4 may include steps 1310 to 1330 below.

In step 1310, the electronic device 300 may generate first sub biomarkers based on the first coordinate values received for the first target content. For example, when the first target content is content for detecting pro-saccades of the user, the first sub biomarkers are fixation time max, fixation mean ), the average of the times taken to switch from the previous saccade to the next saccade (saccade latency mean), the number of fixation counts, and the average speed of transition from fixation to saccade (saccade velocity mean). . According to one example, steps 1210 and 1220 may be performed to perform step 1310.

In step 1320, the electronic device 300 may generate second sub biomarkers based on the second coordinate values received for the second target content. For example, when the second target content is content for detecting anti-saccades, the second sub biomarkers are the standard deviation (saccadeStd) of the durations of the saccades, and the user selects the second content. It may include a screen duration total, a standard deviation of gaze fixation times (fixationStd), a fixation count, and an average (coordX mean) of x coordinate values among gaze coordinate values.

For example, although the configuration in which the first sub biomarkers and the second sub biomarkers are generated through steps 1310 and 1320 has been described, an additional third sub biomarker for the third target content and the fourth target content has been described. Biomarkers and fourth biomarkers may be further generated.

In step 1330, the electronic device 300 may generate a target biomarker set based on the first sub biomarkers and the second sub biomarkers. For example, a target biomarker set may be input to each of a plurality of trained models.

14 is a flowchart of a method of determining a user's attentional level using a plurality of models according to an example.

According to one embodiment, step 440 described above with reference to FIG. 4 may include steps 1410 and 1420 below.

In step 1410, the electronic device 300 may input target biomarkers (or target biomarker sets) to each of a plurality of previously trained models. For example, the plurality of models may include a first model, a second model, a third model, and a fourth model. Each of the first model, the second model, the third model, and the fourth model may be a K Neighbors Classifier, an Extra Trees Classifier, a Logistic Regression, and a Random Forest Classifier.

According to an embodiment, an algorithm or classifier that can be used as a model for classifying the degree of attention may be selected through an experiment. For example, the first model, the second model, the third model, and the fourth model may be selected based on the above experiments. [Table 2] below shows the above experimental results.

The model named 'dummy' in [Table 2] may not be implemented algorithmically accurately, but may be a model that derives only one result, which is simply abnormal. Accordingly, a model exhibiting higher accuracy/reliability than the result of the dummy model may be selected as a model for classifying the degree of attention.

According to [Table 2], models showing higher accuracy/reliability than the results of the dummy model appear as K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, Random Forest Classifier, and Quadratic Discriminant Analysis. Among these, Quadratic Discriminant Analysis can be excluded from the final use model because other indicators (eg AUC, Recall, F1, etc.) other than accuracy are almost similar to the dummy model, and K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, and A Random Forest Classifier can be determined with end-use models.

According to an embodiment, the model based on the Extra Trees Classifier does not use the same data repeatedly during training, but the model based on the random forest classifier allows the same data to be used repeatedly during the training process. A model based on a random forest classifier can catch characteristics of input data that cannot be captured by models based on other methods by allowing data redundancy.

According to an embodiment, each of the plurality of models may output the degree of attention for the input target biomarkers as a result. For example, a first model may output a first result, a second model may output a second result, a third model may output a third result, and a fourth model may output a fourth result.

In operation 1420, the electronic device 300 may determine the level of attention of the user based on the first result of the first model and the second result of the second model.

According to an embodiment, the electronic device 300 determines, for example, that a first result by a first model is a normal stage, a second result by a second model is a normal stage, and a third result by a third model is ADHD. If the step and the fourth result by the fourth model are normal, the user's attention level may be determined to be normal by a voting method.

According to another embodiment, the electronic device 300 has, for example, a probability that a first result from the first model is a normal stage is 90% and a probability that a second result from the second model is a normal stage is 80%. %, if the probability that the third result by the third model is the normal stage is 50%, and the probability that the fourth result by the fourth model is the normal stage is 40%, the average value of the probability is calculated as 65%, The degree of attention of the user may be calculated based on the average value of 65%. For example, if the average value of the normal level is higher than a predetermined threshold value, the user may be determined to be in the normal level, and if the average value is not higher than the threshold value, the user may be determined to be in the ADHD level.

15 illustrates a method of determining a user's attentional level using a plurality of models according to an example.

According to an embodiment, the target biomarkers 1510 (or target biomarker set) include a first model 1522, a second model 1524, a third model 1526, and a fourth model 1528. can be entered into each. The first model 1522, the second model 1524, the third model 1526, and the fourth model 1528 are a first result 1532, a second result 1534, and a third result 1536, respectively. A fourth result 1538 can be output.

According to an embodiment, the electronic device 300 applies a voting method (eg, majority vote) to the first result 1532, the second result 1534, the third result 1536, and the fourth result 1538. By doing so, the level of attention of the user can be determined.

According to an embodiment, the electronic device 300 has a probability (or , Probability of ADHD stage) is calculated, and based on the average value, the degree of attention of the user may be determined.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. may be permanently or temporarily embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (15)

A method for determining the level of attention of a user, performed by an electronic device,
outputting pre-produced first target content to determine a level of attention of a user through a display of a user terminal;
continuously receiving first coordinate values on the display at which eyes of the user observing the first target content gaze;
generating first sub-biomarkers corresponding to the first target content for eye movement of the user based on the first coordinate values;
outputting previously produced second target content to determine the level of attention of the user through the display of the user terminal;
continuously receiving second coordinate values on the display at which eyes of the user observing the second target content gaze;
generating second sub-biomarkers corresponding to the second target content for eye movements of the user based on the second coordinate values;
generating a target biomarker set based on the first sub-biomarkers and the second sub-biomarkers; and
Determining the level of attention of the user based on the target biomarker set
including,
Determining the level of attention of the user based on the target biomarker set,
inputting the target biomarker set to each of a plurality of models including a first model and a second model trained in different ways to determine a user's attentional level; and
A voting method is applied to a plurality of results of the plurality of models including a first result of the first model for the target biomarker set and a second result of the second model for the target biomarker set; or determining the degree of attention of the user based on an average value of the plurality of results;
including,
The first target content is content in which at least one of the shape, color, and position of at least one object of the first target content changes over time to detect pro-saccades of the user, and ,
The second target content is content in which at least one of the shape, color, and position of at least one object of the second target content changes over time in order to detect anti-saccades of the user. ,
The determined degree of attention is output through the user terminal,
Methods for determining the degree of attention.
According to claim 1,
The first target content or the second target content includes an animation in which at least one of the shape, color, and position of at least one object changes, and a voice or text instruction indicating an action to the user. doing,
Methods for determining the degree of attention.
According to claim 1,
The first coordinate values continuously received are determined by the user terminal,
the user terminal,
detecting a first eye of the user in a first user image;
determining a first gaze direction of the user based on the first eye;
determining a first coordinate value on the display based on the first gaze direction; and
associating the first coordinate value with a first time stamp of the first user image;
The first coordinate values are continuously determined by performing
Methods for determining the degree of attention.
According to claim 1,
The first sub-biomarkers corresponding to the first target content include a fixation time max, a fixation average time, and an average of times taken to switch from the previous saccade to the next saccade (fixation time max) Including one or more of saccade latency mean, fixation count, and average speed of transition from fixation to saccade (saccade velocity mean),
Methods for determining the degree of attention.
According to claim 1,
The second sub-biomarkers corresponding to the second target content include a standard deviation of saccade duration times (saccadeStd), a total time the user viewed the target content (screen duration total), and a standard of gaze fixation times. Including one or more of a deviation (fixationStd), a fixation count, and an average of x coordinate values (coordX mean) among the coordinate values,
Methods for determining the degree of attention.
According to claim 1,
outputting previously produced third target content to determine the level of attention of the user through the display of the user terminal;
continuously receiving third coordinate values on the display at which eyes of the user observing the third target content gaze; and
Generating third sub-biomarkers corresponding to the third target content for eye movements of the user based on the third coordinate values
Including more,
Generating the target biomarker set,
generating the target biomarker set based on the third sub biomarkers;
including,
The third target content is content for detecting memory guided saccades of the user,
The third sub-biomarkers corresponding to the third target content include a maximum fixation time (fixation time max), a total sum of fixation times (fixation duration), an average fixation time (fixation mean), among the coordinate values. Including one or more of the average of y coordinate values (coordY mean), and the total time of viewing the target content (screen duration total),
Methods for determining the degree of attention.
According to claim 1,
outputting pre-produced fourth target content to determine the level of attention of the user through the display of the user terminal;
continuously receiving fourth coordinate values on the display at which eyes of the user observing the fourth target content gaze; and
Generating fourth sub-biomarkers corresponding to the fourth target content for eye movements of the user based on the fourth coordinate values
Including more,
Generating the target biomarker set,
generating the target biomarker set based on the fourth sub biomarkers;
including,
The fourth target content is content for detecting change detection of the user,
The fourth sub-biomarkers corresponding to the fourth target content are the total elapsed times, the standard deviation of the times taken to switch from the previous saccade to the next saccade (saccade latencyStd) ), the average of the times taken to switch from the previous saccade to the next saccade (saccade latency mean), the average of the times of looking at the target content (screen duration mean), the average fixation time (fixation mean), looking at the target content Including one or more of the total time (screen duration total) and the sum of the times it took to switch from the previous saccade to the next saccade (saccade latency total),
Methods for determining the degree of attention.
According to claim 1,
outputting pre-produced fifth target content to determine the level of attention of the user through the display of the user terminal;
continuously receiving fifth coordinate values on the display at which eyes of the user observing the fifth target content gaze; and
Generating fifth sub-biomarkers corresponding to the fifth target content for eye movement of the user based on the fifth coordinate value
Including more,
Generating the target biomarker set,
generating the target biomarker set based on the fifth sub biomarkers;
including,
The fifth target content is content for a stroop test of the user,
The fifth sub-biomarkers corresponding to the fifth target content are the total time of viewing the target content (screen duration total), the number of saccades (saccade count), and the sum of the duration of saccades (saccade duration) , the average of x coordinate values among the coordinate values (coordX mean), the standard deviation of the times that saccades lasted (saccadeStd), the average of the times of viewing the target content (screen duration mean), and the fixation average time (fixation mean) ) containing one or more of,
Methods for determining the degree of attention.
delete According to claim 1,
The step of generating the first sub biomarkers,
generating first raw target biomarkers for predefined eye movements of the user with respect to the first target content based on the first coordinate values; and
Generating the first sub-biomarkers corresponding to the first original target biomarkers by applying a preset normalization scheme to each of the first original target biomarkers.
including,
Methods for determining the degree of attention.
delete delete According to claim 1,
The first model is K Neighbors Classifier, Extra Trees Classifier, Logistic Regression, Random Forest Classifier, Quadratic Discriminant Analysis, Light Gradient Boosting Machine, Ada Boost Classifier, Ridge Classifier, SVM-Linear Kernel, Decision Tree Classifier, Gradient Boosting Classifier, Linear A model trained based on any one of Discriminant Analysis and Naive Bayes,
The second model is a model different from the first model,
Methods for determining the degree of attention.
Claims 1, 2, 3, 4, 5, 6, 7, 8, 10, and 13, a program that performs any one of the methods A computer-readable recording medium recorded thereon.
The electronic device that determines the level of attention of the user,
a memory in which a program for determining a user's level of attention is recorded; and
Processor that runs the above program
including,
said program,
outputting pre-produced first target content to determine a level of attention of a user through a display of a user terminal;
continuously receiving first coordinate values on the display at which eyes of the user observing the first target content gaze;
generating first sub-biomarkers corresponding to the first target content for eye movement of the user based on the first coordinate values;
outputting previously produced second target content to determine the level of attention of the user through the display of the user terminal;
continuously receiving second coordinate values on the display at which eyes of the user observing the second target content gaze;
generating second sub-biomarkers corresponding to the second target content for eye movements of the user based on the second coordinate values;
generating a target biomarker set based on the first sub-biomarkers and the second sub-biomarkers; and
Determining the level of attention of the user based on the target biomarker set
do,
Determining the level of attention of the user based on the target biomarker set,
inputting the target biomarker set to each of a plurality of models including a first model and a second model trained in different ways to determine a user's attentional level; and
A voting method is applied to a plurality of results of the plurality of models including a first result of the first model for the target biomarker set and a second result of the second model for the target biomarker set; or determining the degree of attention of the user based on an average value of the plurality of results;
including,
The first target content is content in which at least one of the shape, color, and position of at least one object of the first target content changes over time in order to detect pro-saccades of the user, and ,
The second target content is content in which at least one of the shape, color, and position of at least one object of the second target content changes over time in order to detect anti-saccades of the user. ,
The determined degree of attention is output through the user terminal,
electronic device.

Images