KR102729359B1 - Apparatus and method for supporting teaching and learning based on virtual helper - Google Patents

Abstract

According to an embodiment of the present disclosure, a teaching and learning support method may include a step of processing a connection procedure of a plurality of user terminals and an instructor terminal, a step of transmitting content to a plurality of user terminals, a step of generating a virtual object and transmitting a control signal requesting that the virtual object be displayed on the plurality of user terminals or the instructor terminals, a step of generating additional content based on at least one of information collected from at least some of the plurality of user terminals, content, and voice data received from the instructor terminal, and a step of transmitting a control signal requesting that the virtual object display additional content on the plurality of user terminals or the instructor terminals.

Description

{APPARATUS AND METHOD FOR SUPPORTING TEACHING AND LEARNING BASED ON VIRTUAL HELPER}

The present disclosure relates to a device and method for supporting teaching and learning, and more particularly, to a device and method for supporting teaching and learning based on a virtual assistant.

Non-face-to-face technologies, represented by untact, are gradually developing, and in particular, the market for remote video conferencing and video lectures is also growing.

Current video conferencing and video lecture technologies are mostly limited to simply displaying content.

Accordingly, prior art 1 presents a configuration that utilizes not only the learner's learning information but also the learner's biometric information.

However, the current video conferencing and video lecture technologies have the problem of not being able to provide a way to change the one-way transmission of the instructor (speaker) and the passive attitude of the learner (listener).

Prior art 1: Korean Patent Publication No. 10-2383457 (announced on April 8, 2022)

One embodiment of the present disclosure provides a teaching and learning support device and method that improves the passive situation of a learner or listener.

Another embodiment of the present disclosure provides a teaching and learning support device and method that can enhance lectures and understanding of instructors/speakers and learners/listeners by generating additional content from content.

According to an embodiment of the present disclosure, a teaching and learning support method may include a step of processing a connection procedure of a plurality of user terminals and an instructor terminal, a step of transmitting content to a plurality of user terminals, a step of generating a virtual object and transmitting a control signal requesting that the virtual object be displayed on the plurality of user terminals or the instructor terminals, a step of generating additional content based on at least one of information collected from at least some of the plurality of user terminals, content, and voice data received from the instructor terminal, and a step of transmitting a control signal requesting that the virtual object display additional content on the plurality of user terminals or the instructor terminals.

According to another embodiment of the present disclosure, a teaching and learning support method is a method in which each step is performed by a processor, and may include a step of processing a connection procedure of a plurality of user terminals, a step of displaying content provided by an instructor on a first display device, a step of generating additional content based on at least one of information, content, and instructor's voice data collected from at least some of the plurality of user terminals, and a step of displaying the additional content on a second display device.

According to one embodiment of the present disclosure, a teaching and learning support device includes a processor and a memory electrically connected to the processor and storing at least one code executed by the processor, and the memory may store code that, when executed through the processor, causes the processor to process a connection procedure of a plurality of user terminals and an instructor terminal, transmit content to the plurality of user terminals, generate an object and transmit a control signal requesting that a virtual object be displayed on the plurality of user terminals or the instructor terminal, generate additional content based on at least one of information collected from at least some of the plurality of user terminals, content, and voice data received from the instructor terminal, and transmit a control signal requesting that a virtual object be displayed on the plurality of user terminals or the instructor terminal to the plurality of user terminals or the instructor terminal.

According to another embodiment of the present disclosure, a teaching and learning support device includes a processor and a memory electrically connected to the processor and storing at least one code to be executed by the processor, wherein the memory may store a code that, when executed through the processor, causes the processor to process a connection procedure of a plurality of user terminals, display content provided by an instructor on a first display device, generate additional content based on at least one of information, content, and instructor's voice data collected from at least some of the plurality of user terminals, and display the additional content on a second display device.

The teaching and learning support device and method according to the embodiment of the present disclosure can provide a two-way active conference or learning environment between an instructor/speaker and a learner/listener.

The teaching and learning support device and method according to the embodiment of the present disclosure can improve the learning efficiency of learners.

FIG. 1 is a diagram illustrating an example of an interface according to a teaching and learning support method according to an embodiment of the present disclosure.
FIG. 2 is a diagram showing an environment for performing a teaching and learning support method or operating a teaching and learning support device according to an embodiment of the present disclosure.
FIG. 3 is a diagram showing an environment for performing a teaching and learning support method or operating a teaching and learning support device according to another embodiment of the present disclosure.
FIG. 4 is a block diagram showing the configuration of a teaching and learning support device according to one embodiment of the present disclosure.
FIG. 5 is a flowchart for explaining a teaching and learning support method according to one embodiment of the present disclosure.
FIG. 6 is a diagram for explaining a method for generating additional content and a method for determining a transmission time of additional content according to one embodiment of the present disclosure.
FIG. 7 is a diagram illustrating a method for determining the concentration or understanding of a learner or listener according to one embodiment of the present disclosure.

Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing symbols, identical or similar components will be given the same reference numerals and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves. In addition, when describing embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The terms are used only to distinguish one component from another.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

Referring to FIG. 1, an embodiment of an interface according to a teaching and learning support method according to an embodiment of the present disclosure is described.

Figure 1 illustrates an example of an interface of a user terminal (200) by a teaching and learning support method.

Although this specification describes a remote lecture as an example, it is also applicable to a remote video conference, and the instructor in this specification is a concept that includes a speaker in a remote video conference, and the learner is a concept that includes a listener or user in a remote video conference. Therefore, the embodiments in this specification should not be understood as being limited to remote lectures.

The user terminal may have an area for displaying content (101) provided by the instructor, an area for optionally receiving and displaying the instructor's image from the instructor terminal (300), and an area for receiving and displaying the image of a user (learner) from multiple user terminals (200).

The teaching and learning support device (100) can transmit a control signal to the instructor terminal (300) and the user terminal (200) to display a virtual object in a specific area. The virtual object may be a shape other than a specific shape, or may be a virtual student. The specific area may be a blank area that displays additional content without a virtual object.

The teaching and learning support device (100) may generate additional content based on at least one of information collected from at least some of a plurality of user terminals (200), content (101) that changes according to the progress of a lecture, and voice data received from an instructor terminal (300), and transmit a control signal to the user terminal (200) and the instructor terminal (300) to display the additional content on a virtual object or the specific area. At this time, the additional content displayed on the user terminal (200) and the instructor terminal (300) may be different from each other, and may be displayed on only one of them.

Referring to FIG. 2, an embodiment of an environment for performing a teaching and learning support method according to an embodiment of the present disclosure or for operating a teaching and learning support device is described.

In one embodiment, the teaching and learning support device (100) may include a lecture platform (100a) and an internal teaching and learning support unit (100b). The lecture platform (100a) may process a connection process with a user terminal (200) and an instructor terminal (300), distribute content uploaded from an instructor terminal (300) to the user terminal (200), and distribute real-time lecture data (voice, writing, etc.) of the instructor terminal (300) to the user terminal (200). The teaching and learning support unit (100b) may control the display of virtual objects on the user terminal (200) and the instructor terminal (300), and generate additional content based on information collected from the user terminal (200) and the instructor terminal (300), content uploaded from the instructor terminal (300), and real-time lecture data (voice, writing, etc.) of the instructor terminal (300).

In another embodiment, the lecture platform (100a) and the teaching and learning support device may be configured as separate devices, in which case the teaching and learning support device may be linked to the lecture platform (100a) to request distribution of virtual objects and generated additional content to the user terminal (200) and the instructor terminal (300) through the lecture platform (100). In addition, for the purpose of generating additional content, information collected from the user terminal (200) and the instructor terminal (300), content uploaded from the instructor terminal (300), and real-time lecture data (voice, writing, etc.) of the instructor terminal (300) may be received from the lecture platform (100a).

Referring to FIG. 3, an embodiment of an environment for performing a teaching and learning support method according to another embodiment of the present disclosure or for operating a teaching and learning support device is described.

The embodiment described with reference to Fig. 2 is a technology for supporting teaching and learning through remote video conferencing, while the embodiment described with reference to Fig. 3 is a technology for supporting teaching and learning in offline-based lectures, conferences, etc. such as face-to-face classes. The following description assumes that the lecture platform (100a) and the internal teaching and learning support unit (100b) are configured in one device, but they may be configured as separate devices, as in Fig. 2.

The teaching and learning support device (100) processes the connection procedure of multiple user terminals (201) and displays content uploaded by an instructor (301) on the first display device (101). The teaching and learning support device (100) may collect voice data of the instructor (301) or generate additional content based on the content uploaded by multiple user terminals (201) and the instructor (301), and then display the additional content on the second display device (103). The additional content may be displayed as a sound or visual element, such as a virtual student or virtual object displayed on the second display device (103). In the embodiment of FIG. 3, a virtual student may be displayed as a three-dimensional hologram on the second display device (103) or may be displayed on a two-dimensional display device.

In this specification, the term “indication” and “display” are concepts that include all methods of expressing something externally as a visual or auditory element, such as sound or LED, in addition to necessarily expressing it as a visual element.

Referring to FIG. 4, the configuration of a teaching and learning support device (100) according to one embodiment of the present disclosure is described.

The teaching and learning support device (100) may include a communication unit (110) that performs connection processing of a user terminal (200) and an instructor terminal (300), receives question information of a user terminal (200), receives selection information of a user terminal (200), receives voice of an instructor terminal (300), or receives image information acquired from a camera of a user terminal (200) and an instructor terminal (300) or eye position information processed from image information, and a memory (120) that stores code or content that instructs the processor (130) to process the execution. The teaching and learning support device (100) may include a learning model (150) that inputs image information acquired from a camera of a user terminal (200) and an instructor terminal (300) into a deep learning-based learning model to determine an expression or extract a keyword from voice information, and a deep learning processor (140) that accelerates a deep learning-based process. The teaching and learning support device (100) may include an interface (150) and a display (170) for user interfacing, such as an administrator.

In one embodiment, the communication unit (110) of the teaching and learning support device (100) may include a wireless communication unit or a wired communication unit as a communication interface.

The wireless communication unit may include at least one of a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

The mobile communication module transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to a communication method for mobile communication, such as LTE (Long Term Evolution).

The wireless Internet module is a module for wireless Internet access, and can be built into or externally installed in the teaching and learning support device (100), and WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), etc. can be used.

The short-range communication module is a module for transmitting and receiving data through short-range communication, and can use Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), etc.

The location information module is a module for obtaining the location of the teaching and learning support device (100), and may be a GPS (Global Positioning System) module based on satellite navigation technology, or a module for obtaining the location based on wireless communication with a wireless communication base station or wireless access point. The location information module may include a WiFi module.

In one embodiment, the teaching and learning support device (100) may include an input unit or an output unit for user input.

The input unit includes a user interface (UI) including a microphone and a touch interface for inputting information from a user. The user interface may include a mouse, a keyboard, and a mechanical or electronic interface implemented in the device, and is not particularly limited in its method or form as long as it can input a user's command. The electronic interface includes a display capable of touch input.

The display (170) is intended to convey information to the user by externally displaying the output of the teaching and learning support device (100), and may include a display, LED, speaker, etc. for displaying visual output, auditory output, or tactile output.

The teaching and learning support device (100) may include a peripheral interface (160) for data transmission with various types of connected external devices, and may include a memory card port, an external device I/O (Input/Output) port, etc.

Deep learning-based learning models can include neural networks with structures such as CNN or R-CNN (Region based CNN), C-RNN (Convolutional Recursive Neural Network), Fast R-CNN, Faster R-CNN, R-FCN (Region based Fully Convolutional Network), YOLO (You Only Look Once) or SSD (Single Shot Multibox Detector), RNN (Recursive neural network), and LSTM (Long Short-Term Memory).

The learning model may be implemented in hardware, software, or a combination of hardware and software, and if part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory.

Referring to FIG. 5, a teaching and learning support method of a teaching and learning support device (100) according to one embodiment of the present disclosure is described.

As explained above, this method is explained on the premise that the user terminal (200) and the instructor terminal (300) that process the connection procedure or the lecture platform (100a) that distributes the content are configured as one device, but it does not exclude that they are configured separately.

The teaching and learning support device (100) processes the connection procedure of the user terminal (200) and the instructor terminal (300) (S110). If the teaching and learning support device (100) is a lecture support device, the user terminal (200) may be a learner terminal.

The teaching and learning support device (100) transmits content uploaded in advance or in real time by the teaching terminal (300) to the user terminal (200) for which the connection procedure has been completed (S120).

The teaching and learning support device (100) transmits a control signal to display a virtual object on the user terminal (200) and the instructor terminal (300) (S130). The virtual object is an object displayed on the display as the subject of the display when displaying additional content thereafter, and is a virtual character in the form of a person, or its form is not particularly limited.

Thereafter, the teaching and learning support device (100) generates additional content based on information collected from the user terminal (200) and the instructor terminal (300), content uploaded from the instructor terminal (300), real-time lecture data (voice, writing, etc.) of the instructor terminal (300), etc. (S140), and transmits a control signal to display the additional content on at least one of the user terminal (200) and the instructor terminal (300) (S150).

The information collected from the user terminal (200) may be feedback information from users (learners) regarding the content. For example, an interface that enables feedback on the level of understanding of the current content may be displayed on the user terminal (200), and the user's selection input for this may be (for example, a button saying 'I don't understand' or a selection of a scale value on a 5-point scale, etc.). As another example, the feedback information may be questions from users or a selection of agreement to a question from a specific user. For example, questions collected from the user terminal (200) may be feedback information, or the collected questions may be displayed on the user terminal (200), and information collected regarding whether each user terminal (200) agrees or selects may be feedback information.

The examples below are explained on the assumption that the feedback information is question information.

Referring to FIG. 6, an embodiment of additional content and an embodiment of a method for determining when to transmit a control signal to display the additional content are described.

In one embodiment, the teaching and learning support device (100) may receive a plurality of question information about content from at least some of the user terminals (200), and generate additional content based on the plurality of question information. For example, after receiving a plurality of questions from a plurality of user terminals (200), the received plurality of questions may be listed and transmitted to the user terminals (200), and the plurality of questions may be prioritized according to the selection (preference) of the user terminal (200), and then some of the questions may be selected and generated as additional content, and a control signal may be transmitted to display the determined questions on the user terminal (200) and the instructor terminal (300). An instructor who has confirmed such additional content may provide an additional explanation on the displayed question through a virtual object and then move on to the next content.

In this case, it provides a way for students who spend a lot of time asking common questions from many learners or have difficulty actively participating in the lecture to actively participate in the lecture in an indirect way, and it has the effect of helping learners understand the lecture.

In one embodiment, the teaching and learning support device (100) receives voice data of the instructor terminal (300) or the instructor (301), converts the voice data into text data, and then generates additional content in the form of a word cloud (610) based on the text data using a learning model based on deep learning, or extracting words and the frequency of occurrence among the extracted words.

In one embodiment, the teaching and learning support device (100) may form a word cloud based on words (keywords) included in content uploaded from the teaching terminal (300), or may simply form a word cloud based on the teaching and learning support device (100). The teaching and learning support device (100) transmits a control signal to change and display the word cloud content when the salience order changes according to a preset time interval or the frequency of words included in the word cloud, thereby enabling learners to visually understand important lecture content in a timely manner as the content progresses. Accordingly, the learning efficiency of the learners can be improved.

For example, when forming a word cloud based on words (keywords) included in content uploaded from the instructor terminal (300), the word cloud can be formed with content contents from the start of content display to the present. Alternatively, the word cloud can be formed based on words (keywords) included in content within a preset range from the currently displayed page of the content in multiple user terminals. The preset range can form a word cloud based on content before and/or after a certain page range from the currently displayed page when the content is composed of pages. Therefore, the effect of allowing learners to visually understand important lecture contents in a timely manner can be doubled as the content progresses.

In one embodiment, the teaching and learning support device (100) can compare at least one of keywords with a high frequency of occurrence in voice data received from the instructor terminal (300), keywords with a high frequency of occurrence analyzed from content, and keywords in a word cloud with words included in content currently being displayed on the user terminal (200), and display some words determined to match the keywords differently from other words, which may be displayed in different morphological sizes or colors. For example, additional content can be generated so that words with different morphological sizes are displayed based on the frequency of words appearing in the voice data, while words included in the content (including similar words) are displayed in different colors. Or, conversely, by modifying the content so that words with a certain frequency or higher included in the word cloud among the content uploaded from the instructor terminal (300) are displayed in different colors or shapes, it is possible to help learners visually understand important lecture content in a timely manner.

The teaching and learning support device (100) may determine the timing of transmission of a control signal requesting display of generated additional content to at least one of the user terminal (200) and the instructor terminal (300) rather than transmitting the signal at the time of generation (620). In this way, learning efficiency can be improved by not interrupting the flow of the instructor's lecture or presentation or by not allowing the instructor to miss cases where the students lack understanding.

In one embodiment, the teaching and learning support device (100) may determine a time point for transmitting a control signal requesting display of additional content based on a plurality of question information received from a plurality of user terminals (200). For example, if the appearance frequency of a word included in the plurality of question information is calculated and the appearance frequency of a specific word exceeds a certain number of times, or if the preference information of the user terminal (200) for the plurality of question information exceeds a certain number of times for a specific question information, a control signal requesting display of additional content may be transmitted by determining that the learners have insufficient understanding of the content.

In another embodiment, the teaching and learning support device (100) may determine the timing for transmitting a control signal requesting display of additional content based on the context of the voice data or content received from the instructor terminal (300). For example, the teaching and learning support device (100) may distinguish between content explaining a lecture and content encouraging questions among the voice data received from the instructor terminal (300), and if it determines that the content encourages questions, it may transmit a control signal requesting display of additional content. As another example, the teaching and learning support device (100) may monitor content changes according to the content display control signal of the instructor terminal (300) for content provided by the instructor, determine a time point at which there is a large change in the content before and after (for example, a change in a chapter), and transmit a control signal requesting display of additional content at that time point. In this case, the screen images according to the progress of the content can be input into a deep learning-based learning model that has learned training data that labels each page of the content as the content of a specific chapter or the start of the next chapter, and the point of greatest change can be determined. Alternatively, the point of greatest change can be determined by comparing the text corresponding to the chapter indication of the content.

Referring to FIG. 7, another embodiment of an embodiment of additional content and a method for determining when to transmit a control signal to display additional content are described.

In one embodiment, the teaching and learning support device (100) can determine the timing for transmitting a control signal to display additional content based on image information acquired from the user terminal (200) and the instructor terminal (300) or eye movements of the user and instructor acquired from the image information.

For example, the teaching and learning support device (100) can determine first location information (710a, 710b), which is the location of content currently being explained by the instructor, from the eye movements of the instructor's terminal (300). In addition, the teaching and learning support device (100) can determine second location information (720a, 720b), which is the location of content currently being watched by the learners, from the eye movements of the learners' terminals (200). The first location information and the second location information may be determined as coordinates, or may be determined as areas based on eye movements over a certain time interval.

Thereafter, the teaching and learning support device (100) can determine whether the learners understand the instructor's lecture well based on the distance, degree of overlap, etc., of the first location information and the second location information. For example, as shown in Fig. 7 (a), if the first location information and the second location information are determined as regions and the degree of overlap of the two regions exceeds a preset standard, it can be determined that the learners understand the content explained by the instructor well. Conversely, as shown in Fig. 7 (a), if the first location information and the second location information are determined as regions and the distance between the center points of the two regions exceeds a preset standard or there is no overlapping part, it can be determined that the learners do not understand the content explained by the instructor well, and a control signal can be transmitted to display additional content. In this case, the second location information can be displayed as additional content while indicating "the learners do not understand", or question information received from user terminals (200) can be displayed as additional content.

As another embodiment, the teaching and learning support device (100) may determine the timing for transmitting a control signal to display additional content based on image information acquired from the user terminal (200) or the user's facial expression acquired from the image information.

For example, a facial image acquired from a user terminal (200) may be input into a deep learning-based learning model to determine the facial expression. If the facial expression is determined to be a negative image, such as an expression that many users do not understand or an expression of embarrassment, a message such as “learners do not understand” may be displayed, or question information received from the user terminals (200) may be displayed as additional content.

The above-described present disclosure can be implemented as a computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices that store data that can be read by a computer system. Examples of the computer-readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. In addition, the computer may include a processor of each device.

Meanwhile, the above program may be specially designed and configured for the present disclosure or may be known and available to those skilled in the art of computer software. Examples of the program may include not only machine language codes produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

The use of the term "above" and similar referential terms in the specification of this disclosure (especially in the claims) may refer to both the singular and the plural. In addition, when a range is described in this disclosure, it is intended that the invention be applied to individual values falling within the range (unless otherwise stated), and it is the same as describing each individual value constituting the range in the detailed description of the invention.

Unless the steps constituting the method according to the present disclosure are explicitly described in an order or contrary description, the steps can be performed in any suitable order. The present disclosure is not necessarily limited by the order in which the steps are described. The use of all examples or exemplary terms (e.g., etc.) in this disclosure is merely intended to describe the present disclosure in more detail and is not intended to limit the scope of the present disclosure due to the examples or exemplary terms, unless otherwise defined by the claims. In addition, those skilled in the art will appreciate that various modifications, combinations, and changes can be configured according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are included in the scope of the spirit of the present disclosure.

100: Teaching and Learning Support Device
101: Content
103: Second display device
200: User Terminal
300: Professor Terminal
610: Word Cloud Additional Content
620: When additional content is available
710: 1st location information
720: Second location information

Claims (14)

As a method in which each step is performed by a processor,
A step for processing the connection procedure of multiple user terminals and instructor terminals;
A step of transmitting content to the plurality of user terminals;
A step of generating a virtual object, which is a virtual student appearance, and transmitting a control signal requesting display of the virtual object on the plurality of user terminals or the instructor terminal to the plurality of user terminals or the instructor terminal;
A step of generating additional content based on at least one of information collected in real time from at least some of the plurality of user terminals, the content, and real-time voice data received from the instructor terminal; and
A step of transmitting a control signal requesting that the virtual object display the additional content to the plurality of user terminals or the instructor terminal,
Receive a plurality of question information about a lecture and preference information of the plurality of user terminals for the plurality of question information from at least some of the plurality of user terminals,
The timing for transmitting the additional content to the plurality of user terminals and the instructor terminal is determined based on the frequency of occurrence of words included in the plurality of question information or preference information for specific question information among the plurality of question information.
How to support teaching and learning.
As a method in which each step is performed by a processor,
A step for processing the connection procedure of multiple user terminals;
A step of displaying content provided by an instructor on a first display device;
A step of generating additional content based on at least one of the information collected from at least some of the plurality of user terminals, the content, and the instructor's voice data; and
A step of displaying said additional content on a second display device,
The above additional content includes virtual students or virtual objects displayed as sound or visual elements,
Receive a plurality of question information about a lecture and preference information of the plurality of user terminals for the plurality of question information from at least some of the plurality of user terminals,
The timing for transmitting the additional content to the plurality of user terminals and instructor terminals is determined based on the frequency of occurrence of words included in the plurality of question information or preference information for specific question information among the plurality of question information.
How to support teaching and learning.
processor; and
A memory electrically connected to the processor and storing at least one code to be executed by the processor,
The above memory, when executed through the processor, causes the processor to:
A code is stored that processes a connection procedure of a plurality of user terminals and an instructor terminal, transmits content to the plurality of user terminals, generates a virtual object which is a virtual student appearance, and transmits a control signal requesting that the plurality of user terminals or the instructor terminal display the virtual object, to the plurality of user terminals or the instructor terminal, generates additional content based on at least one of information collected from at least some of the plurality of user terminals, the content, and voice data received from the instructor terminal, and causes the plurality of user terminals or the instructor terminal to transmit a control signal requesting that the virtual object display the additional content, to the plurality of user terminals or the instructor terminal.
The above memory receives a plurality of question information about a lecture and preference information of the plurality of user terminals for the plurality of question information from at least some of the plurality of user terminals, and stores a code for determining a time point for transmitting the additional content to the plurality of user terminals and the instructor terminal based on the occurrence frequency of words included in the plurality of question information or preference information for specific question information among the plurality of question information.
Teaching and learning support device.
In the third paragraph,
The above memory causes the processor to:
Further storing a code that causes a plurality of feedback information about content to be received from at least some of the plurality of user terminals and generates the additional content based on the plurality of feedback information.
Teaching and learning support device.
In the fourth paragraph,
The above memory causes the processor to:
A code is further stored that transmits a control signal to the plurality of user terminals requesting to receive a plurality of feedback information about a lecture from at least some of the plurality of user terminals and display the plurality of feedback information on the plurality of user terminals, receives preference information for the plurality of feedback information from at least some of the plurality of user terminals, sets priorities of the plurality of feedback information based on the preference information, and causes at least some of the plurality of feedback information to be generated as the additional content based on the priorities.
Teaching and learning support device.
In the third paragraph,
The above memory causes the processor to:
Further storing code causing the processor to determine when to transmit a control signal requesting display of the above additional content to the plurality of user terminals and the instructor terminal.
Teaching and learning support device.
In Article 6,
The above memory causes the processor to:
Further storing a code that causes the timing of transmitting the additional content to the plurality of user terminals and the instructor terminal based on the context of the voice data or the content received from the instructor terminal.
Teaching and learning support device.
delete In Article 6,
The above memory causes the processor to:
Further storing code that causes the content to determine first location information based on eye movements of users of the instructor terminal, the content to determine second location information based on eye movements of at least some users among the plurality of user terminals, and the time to transmit the additional content to the plurality of user terminals and the instructor terminal based on similarity between the first location information and the second location information.
Teaching and learning support device.
In Article 6,
The above memory causes the processor to:
Further storing a code that causes the facial images of users of the plurality of user terminals to be received from at least some of the plurality of user terminals, the facial images of users of the plurality of user terminals to be input into a deep learning-based learning model to determine facial expressions, and the timing of transmitting the additional content to the plurality of user terminals and the instructor terminal based on the facial expressions of users of the plurality of user terminals.
Teaching and learning support device.
In the third paragraph,
The above memory causes the processor to:
Further storing a code that causes a word cloud to be generated as additional content based on voice data received from the instructor terminal or the content,
The above word cloud changes as the content progresses.
Teaching and learning support device.
In Article 11,
The above memory causes the processor to:
A code is further stored that determines at least one of a keyword with a high frequency of occurrence in voice data received from the instructor terminal, a keyword with a high frequency of occurrence by analyzing the content, and a keyword included in the word cloud from the content, and causes the content with the keyword emphasized to be transmitted to the plurality of user terminals.
Teaching and learning support device.
In Article 11,
The above memory causes the processor to:
Further storing a code that causes the word cloud to be generated as the additional content from the content within a preset range from the currently displayed page of the content in the plurality of user terminals.
Teaching and learning support device.
processor; and
A memory electrically connected to the processor and storing at least one code to be executed by the processor,
The above memory, when executed through the processor, causes the processor to:
A code is stored that processes the connection procedure of multiple user terminals, displays content provided by an instructor on a first display device, generates additional content based on at least one of information collected from at least some of the multiple user terminals, the content, and the instructor's voice data, and causes the additional content to be displayed on a second display device.
The above memory receives a plurality of question information about a lecture and preference information of the plurality of user terminals for the plurality of question information from at least some of the plurality of user terminals, and stores a code for determining a time point for transmitting the additional content to the plurality of user terminals and the instructor terminal based on the appearance frequency of words included in the plurality of question information or preference information for specific question information among the plurality of question information.
Teaching and learning support device.

Images