KR20160044315A - Analysis system and method for class attitude - Google Patents

Abstract

The present invention relates to an analysis system and an analysis method for class attitude of a student, and more particularly, to an analysis system and an analysis method for class attitude of a student enabling an automatic measurement of class attitude and efficiency of individual student and provision of useful information to the student, a teacher and parents of the student. The classroom attitude analysis system according to the present invention comprises: an RGB-D camera installed in a classroom and capturing a depth image with depth gradient based on position by pixel through a pixel-by-pixel depth image functional relation of a three-dimensional image; a pan/tilt/zoom (PTZ) camera capturing everywhere of the classroom through location change by pan, tilt and zoom functions in cooperation with the RGB-D camera; a sound detection sensor detecting a sound synchronized with the image captured by the RGB-D camera and the PTZ camera; and a classroom attitude analysis server receiving image information captured by the RGB-D camera and the PTZ camera, and the sound information detected by the sound detection sensor, so as to analyze a classroom attitude of students within the classroom. Accordingly, the system improves the classroom attitude by effectively managing the students, thereby providing an effect capable of enhancing grades.

Description

METHOD AND APPARATUS FOR AN ANALYSIS SYSTEM AND METHOD FOR CLASS ATTITUDE

The present invention relates to a system and method for analyzing a student's learner attitude, and more particularly, to a system and method for analyzing a student's learner attitude using a sensor network composed of video and audio sensors, automatically measuring learning attitude and efficiency for each student, The present invention relates to a system and method for analyzing a student's learner attitude capable of providing useful information to a student.

Although the number of students per class is decreasing as compared with the past, there is a problem that it is impossible to directly manage the learning efficiency or attitude of each student in a short period. First of all, if another teacher teaches each class, it is difficult to know the efficiency and attitude of all other students in each class, and it is difficult to manage students objectively.

However, with the development of hardware and software technologies such as ubiquitous big data and computer science, it is time to build a smart system that does not require human judgment and help.

Also, in the past, the student's attitude analyzing device has analyzed the attitude of the class by wearing a brain wave measuring sensor or an attitude measuring sensor or using a device (tablet, special terminal) There is a troublesome problem that the device needs to wear the sensor, and there is a problem that there is a cheating behavior such as use of a proxy when using the device and lack of measures to cope with it.

Korean Patent Publication No. 10-2013-0133664 (Mar. 12, 2013)

Therefore, in order to solve the above-described problem, the present invention uses a sensor network composed of video and audio sensors to automatically measure learning attitude and efficiency for each student and to provide useful information to students, teachers, and parents The purpose of this study is to provide an analysis system and analysis method for students' attitudes toward students.

In order to achieve the above object, the lattice attitude analyzing system according to the present invention is installed in a lecture room and has a depth-based function relationship between pixels of a three-dimensional image, -D camera; A PTZ camera that cooperates with an RGB-D camera and captures every corner of the room through position changes with Pan, Tilt, and Zoom functions; An acoustic sensor for sensing sound synchronized with the image captured by the RGB-D camera and the PTZ camera; And a learner analysis server for receiving the image information captured by the RGB-D camera and the PTZ camera and the acoustic information sensed by the acoustic sensor, and analyzing the lecture attitude of the students in the lecture room.

In order to achieve the above-mentioned object, a class attitude analysis server of a class attitude analysis system according to the present invention includes an image receiving unit receiving an image captured by an RGB-D camera and a PTZ camera; A sound receiving unit receiving sound synchronized with an image from the sound sensing sensor; An image analyzer receiving the image information from the image receiver and analyzing the image information; An acoustic analyzer for receiving acoustic information from the acoustic receiver and analyzing the acoustic information; A data base unit for receiving and storing the image information and the sound information from the image receiving unit and the sound receiving unit; And a main controller for evaluating the attitude of the learner using the image and sound analyzed by the image analyzer and the sound analyzer.

In order to achieve the above object, a data base of a learner attitude analysis system according to the present invention comprises: an image DB for storing and managing image information received by an image receiving unit; An acoustic DB for storing and managing acoustic information received by the acoustic receiver; A student information database that is collected and stored in advance for all students in the classroom; And a learning evaluation reference DB for storing and managing evaluation criteria for evaluating the learning of the student.

According to another aspect of the present invention, there is provided an apparatus for analyzing a learner attitude according to the present invention, comprising: a student recognition module for recognizing a location of an object (student), recognizing a face, and recognizing a behavior from image information; Attendance analysis module for verifying identity and analyzing attendance by comparing and analyzing the location, face, and behavior of the object (student) recognized by the student recognition module and the prior information stored in the student information DB; And a task submission analysis module for analyzing whether or not to submit a task by recognizing a specific activity in a situation created for confirming whether or not the task submission is confirmed.

The system for analyzing the learner attitude according to the present invention automatically measures the attitude and efficiency of learning for each student without the cumbersome process of wearing a brain wave measuring sensor or an attitude measuring sensor, By providing information, it is possible to improve grades by improving student attitude by effectively managing students.

FIG. 1 is a diagram illustrating a system for analyzing a learner attitude according to the present invention.
FIG. 2 is a detailed block diagram of a class attitude analysis server of the class attitude analysis system according to the present invention;
3 is an evaluation reference table diagram stored in the learning evaluation reference DB 464 of the learner attitude analysis system according to the present invention,
Fig. 4 is a diagram of a lecture attitude notification evaluated by the lecture attitude analysis system according to the invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings, and the inventor should properly interpret the concept of the term to describe its own invention in the best way. The present invention should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

1 is a diagram illustrating a learner attitude analysis system according to the present invention.

1, the system for analyzing the class attitude according to the present invention includes an RGB-D camera 100, an acoustic sensor 200, a PTZ camera 300, a class attitude analysis server 400, Includes a plurality of desks (500) and a desk (600).

 The RGB-D camera 100 is installed in a classroom, and generates a depth image with a depth gradient based on a pixel-by-pixel position through a depth image functional relationship of each pixel of the three-dimensional image.

The acoustic sensor 200 is installed in the classroom and detects all the voices of the students in the classroom.

The PTZ camera 300 creates an RGB image by capturing every corner of the room through a position change by Pan, Tilt, and Zoom functions, and can grasp students' meticulous movements.

The learner attitude analysis server 400 receives the image information and the sound information acquired by the RGB-D camera 100, the sound sensing sensor 200, and the PTZ camera 300, do.

2, the learner attitude analysis server 400 includes an image receiving unit 410, an acoustic receiving unit 420, a main control unit 430, an image analyzing unit 440, an acoustic analysis unit 450, a database unit 460, and a communication unit 470.

2 is a detailed block diagram of a learner attitude analysis server of the learner attitude analysis system according to the present invention.

 The image receiving unit 410 receives an image captured by the RGB-D camera 100 and the PTZ camera 300 and the sound receiving unit 420 receives the sound sensed by the sound sensing sensor 200 .

The main control unit 430 controls the image DB 461 and the sound DB 462 of the data base 460 to transmit the image information and the sound information received by the image receiving unit 410 and the sound receiving unit 420, .

The database unit 460 includes an image DB 461 for storing and managing image information received by the image receiving unit 410, an acoustic DB 462 for storing and managing the sound information received by the sound receiving unit 420, ), A student information DB (463) for storing and managing information of all the students in the classroom in which the class attitude is to be analyzed, and a learning evaluation criteria database (464) for storing and managing evaluation criteria for evaluating the student's learning.

In the student information DB 463, image information including acoustic size, acoustic frequency, etc. of each of the students in the classroom and image information including key, body size, skin color, hair condition, and the like are stored and managed in advance.

The image analysis unit 440 includes a student recognition module 441, a presence / absence analysis module 442, and a task submission analysis module 443, and analyzes the attitude of the students.

More specifically, the student recognition module 441 recognizes a student through object recognition, facial recognition, and behavior recognition based on the image received by the image receiving unit 410.

Particularly, the student recognition module 441 determines that negative behaviors such as drowsiness, drowsiness, playfulness, stuttering, and fighting are behaviors that are less concentrated in behavior recognition, and that all active It is judged that participation is high for participating act.

For reference, the student recognition module 441 determines that the action is a joining action when an action such as nodding is repeated, and judges that it is an act of playing, fighting, or fighting against abnormal movements that can not occur during the session .

In addition, the student recognition module 441 determines that the question is an act of asking if the act of lifting a hand continues.

The acoustic analysis unit 450 can determine the concentration and the degree of participation through the sound analysis unit 450. If the abnormal sound that can not be generated during the class is large and continuously generated, If the sound sensor 200 senses the sound of a certain student immediately after the sound of the teacher, or if the sound sensor 200 senses a content related to the class, it is determined to be an answer or a question.

The presence / absence analysis module 442 analyzes the position, face, and behavior information of the object (student) recognized by the student recognition module 441 and the student information stored in the student information DB 463, By verifying identity by comparing and analyzing behavior information, it confirms presence or absence.

As described above, the presence / absence analysis module 442 compares and analyzes the student information previously stored in the student information DB 463 with the student information recognized by the student recognition module 441 to confirm the identity, Can be judged together.

The task submission analysis module 443 analyzes whether the teacher submits a task by distinguishing a student who responds to a specific activity from a student who has not reacted to the created situation and a student who has not reacted when the teacher creates a task submission confirmation status .

For example, when a task is submitted to confirm whether or not the task has been submitted, the task submission analysis module 443 recognizes the position of the students, recognizes the hand movements, Analyze whether or not to submit an assignment by confirming the student's identity.

When the sound receiving unit 420 receives sound sensed by the sound sensing sensor 200, the sound analyzing unit 450 analyzes the received sound.

When a situation is created to answer when the task is submitted to confirm whether or not the task has been submitted, the sound analysis unit 450 analyzes the sound that the student answers and stores the sound information of the student stored in the student information DB 463 Depending on whether they are matched or not, you can decide whether or not to submit an assignment.

In addition, the sound analyzer 450 analyzes the sound and compares the sound with the sound information of the student stored in the student information DB 463 in advance, thereby recognizing the student and analyzing whether or not the student is present.

 Meanwhile, the main control unit 430 determines whether or not the student's behavior information (class concentration degree and class participation degree) by the student recognition module 441, attendance analysis information by the attendance determination analysis module 442, And the final learning attitude is determined by analyzing whether or not the task is submitted by the analysis module 443.

That is, the main control unit 430 receives analysis results of the image analyzing unit 440 and the sound analyzing unit 450, Evaluate and manage the attitude of the class.

3, which is an evaluation reference table diagram stored in the learning evaluation reference DB of the learner attitude analysis system according to the present invention, is stored in the learning evaluation reference DB 464.

That is, regarding the presence or absence of absence, the main control unit 430 evaluates the learning attitude is very bad by applying the evaluation criteria shown in FIG. 3 when the absence attendance analysis module 442 analyzes the absence.

In relation to whether or not to submit the task, the main control unit 430 applies the evaluation criteria shown in FIG. 3 according to the analysis of the task submission analysis module 443, and assigns '+2 points' , "-2" is given when not assigning tasks, and "0" is given to all students when there is no task to submit.

Also, regarding the class concentration, the main control unit 430 counts the number of drowsiness per 5 minutes, counts the number from the time when the reliability of the recognition module is 90% or more , assigns a score corresponding to 0 to 10, The main control unit 430 measures the number of questions and answers per 5 minutes, and counts the number of times from when the recognition reliability of the handed activity is 90% or more, Score.

As described above, the main control part 430 has a very high learning attitude when the total acceptance based on attendance, task, class concentration, and class participation is 12 ~ 9 points, and when 8 ~ 5 points, Is good, the learning attitude is usually in the range of 4 ~ 4 points, and the learning attitude is worse in the case of -5 ~ -8 point, and the learning attitude is very much in the range of -9 ~ -12 points.

The main control unit 430 evaluates and manages the data in units of days, weeks, months, and semesters as shown in FIG. 4 through the evaluation process as described above.

That is, the main control unit 430 may provide the daughter's subject preference information on the degree to which the student has high concentration and participation in the subject.

For reference, FIG. 4 is a diagram of a lecture attitude notification evaluated by the lecture attitude analysis system according to the invention.

Finally, the communication unit 470 transmits the notice of the attitudes of the students evaluated by the weekly, week and semester units by the main control unit 430 to the parents' mail or smart terminal so as to recognize the school life of the child do.

The lecture 600 includes a PC used by the teachers. The PC receives information analyzed by the learner attitude analysis server 400 and provides analysis information to the teacher so that the teacher can manage the student, .

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is to be understood that various modifications and changes may be made without departing from the scope of the appended claims.

100: RGB-D camera
200: Acoustic sensor
300: PTZ camera
400: Class attitude analysis server
410:
420: acoustic receiver
430:
440: Image analysis section
441: Student recognition module 442: Attendance analysis module
443: Project submission analysis module
450: acoustic analysis section
460:
461: Image DB 462: Acoustic DB
463: Student information DB 464: Learning evaluation standard DB
470:
500: Desk
600: Board

Claims (13)

An RGB-D camera 100 installed in a classroom and photographing a depth image with a depth gradient based on a pixel-by-pixel position through a pixel-by-pixel depth image function relationship of the three-dimensional image;
A PTZ camera 300 for photographing the inside of the lecture room through a position change in cooperation with the RGB-D camera 100 using a Pan, a Tilt, and a Zoom function;
An acoustic sensor 200 for sensing sound synchronized with the image captured by the RGB-D camera 100 and the PTZ camera 300; And
An attitude of a learner who receives the image information taken by the RGB-D camera 100 and the PTZ camera 300 and the sound information sensed by the sound sensing sensor 200 and analyzes the lecture attitude of the students in the lecture room And a server (400).
The method according to claim 1,
The learner attitude analysis server 400
An image receiving unit 410 receiving images taken by the RGB-D camera 100 and the PTZ camera 300;
An acoustic receiver 420 receiving the sound synchronized with the image from the acoustic sensor 200;
An image analyzer 440 receiving image information from the image receiver 410 and analyzing the image information;
An acoustic analyzer 450 for receiving the acoustic information from the acoustic receiver 420 and analyzing the acoustic information;
A data base unit 460 for receiving and storing image information and sound information from the image receiving unit 410 and the sound receiving unit 420;
And a main controller (430) for evaluating the learner attitude by using the image and sound analyzed by the image analyzer (440) and the sound analyzer (450).
3. The method of claim 2,
The database unit 460
An image DB 461 for storing and managing image information received by the image receiving unit 410;
An acoustic DB 462 for storing and managing acoustic information received by the acoustic receiver 420;
A student information DB 463 that is previously collected and stored and managed for all the students in the classroom; And
And a learning evaluation criteria database (464) for storing and managing evaluation criteria for evaluating the learning of the student.
The method of claim 3,
The image analysis unit 440
A student recognition module 441 for recognizing the location of the object (student), recognizing the face, and recognizing the behavior from the image information;
A presence / absence analysis module (e.g., a presence / absence determination module) 443 for comparing the location, face, and behavior of the object (student) recognized by the student recognition module 441 with the dictionary information stored in the student information DB 463, 442); And
And a task submission analysis module (443) for analyzing whether or not to submit a task by recognizing a specific action in a situation created for confirming whether or not the task submission is confirmed.
5. The method of claim 4,
The student recognition module 441
The system of class attitude analysis is characterized by judging the drowsiness, the drowsiness, the playfulness, the strangeness, the fighting as the act of decreasing the concentration in recognizing the behavior,
6. The method of claim 5,
The main control unit 430
The student's lecture attitude is evaluated according to the evaluation criteria stored in the learning evaluation criteria DB 464 of the database unit 460 with the analysis results of the image analysis unit 440 and the sound analysis unit 450 A system of class attitude analysis.
The method according to claim 6,
The main control unit 430
And evaluates the learning attitude as "very bad" by applying the evaluation criterion in case of truancy according to the analysis of the attendance analysis module 442. [
The method according to claim 6,
The main control unit 430
O 'point, a' plus score ', or a' minus point 'is applied to the evaluation criterion according to whether or not the task is submitted.
The method according to claim 6,
The main control unit 430
Wherein said concentration degree is measured by the number of drowsiness per 5 minutes, and the number of times is counted from 0.9 seconds or more, and a score corresponding to 0 to 10 points is given according to the count number.
The method according to claim 6,
The main control unit 430
Wherein the participation degree is measured as the number of questions and answers per 5 minutes, the number of times is counted from the time when the act of handing is 0.9 seconds or more, and the score corresponding to 0 to 10 points is given according to the number of counts Attitude Analysis System.
11. The method according to any one of claims 7 to 10,
The main control unit 430
And analyzing and evaluating the attitude of the learner on a day, week, month, or semester basis based on whether or not the attendance assessed according to the evaluation criteria, the presentation of the assignment, the degree of concentration, or the degree of participation is evaluated.
12. The method of claim 11,
Further comprising a communication unit (470) for transmitting the learner attitude evaluated by the main word (430) to a mail or smart terminal of a parent in the form of a notice.
5. The method of claim 4,
The sound analysis unit 450
A situation is set up to answer when the assignment is submitted, and the analyzed sound is analyzed and compared with the dictionary information stored in the student information DB 463, Class Attitude Analysis System.

Images