[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Classifying students based on cognitive state in flipped learning pedagogy

Published: 01 January 2022 Publication History

Abstract

The flipped learning (FL) is found to be an effective teaching methodology which is accomplished in two stages. In the first stage, students take instructions and learn from pre-loaded lecture videos (out-of-class learning). In the second stage, students carry out various activities such as group discussion, think-pair-share, group quiz, etc. in presence of the instructor (in-class learning). Therefore, students get enough time to brainstorm on the topic learnt from the pre-loaded lecture. This new learning pedagogy offers quality learning for many students. However, this teaching methodology does not have provision to monitor students while taking lesson from pre-loaded lecture video unlike live classroom teaching. This may lead to severe learning incompetence for weak students.
In this study, we propose to develop a prototype (model) to monitor the student in flipped learning by capturing the brain wave of individual students passively while they are engaged with lecture video. The siamese neural network is exploited to analyze captured brain waves (EEG signal) in order to classify the students into three categories (Weak, Good, Outstanding) and two categories (Weak and Strong) based on their attention level, respectively. This will help the instructor to identify the weak students and treat them with special care. We validate the performance of the proposed classification task with the data obtained from the proposed prototype. The experimental result shows that the proposed siamese neural network outperforms other classification models.

Highlights

A model is developed for capturing the brain wave signal for flipped learning classroom.
Siamese Neural Network (SNN) is exploited to analyze the captured brain wave.
A SNN model is efficiently utilized to classify the student based on the attention level.
Using SNN classification, we identify the weak student in the flipped learning.
Proposed model is validated with data acquired from the experimental set-up.

References

[1]
Aldowah H., Al-Samarraie H., Alzahrani A.I., Alalwan N., Factors affecting student dropout in MOOCs: a cause and effect decision-making model, J. Comput. Higher Educ. (2019) 1–26,.
[2]
A. Itani, L. Brisson, S. Garlatti, Understanding learner’s drop-out in MOOCs, in: Proceedings of International Conference on Intelligent Data Engineering and Automated Learning, 2018, pp. 233–244.
[3]
X. Xiao, J. Wang, Towards attentive, bi-directional MOOC learning on mobile devices, in: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, 2015, pp. 163–170.
[4]
Grossman P., Research on pedagogical approaches in teacher education, in: Studying Teacher Education: The Report of the AERA Panel on Research and Teacher Education, 2005, pp. 425–476.
[5]
Gren L., A flipped classroom approach to teaching empirical software engineering, IEEE Trans. Educ. 63 (3) (2020) 155–163.
[6]
Hao Y., Exploring undergraduates’ perspectives and flipped learning readiness in their flipped classrooms, Comput. Hum. Behav. 59 (2016) 82–92.
[7]
Hung H.-C., Liu I.-F., Liang C.-T., Su Y.-S., Applying educational data mining to explore students’ learning patterns in the flipped learning approach for coding education, Symmetry 12 (2) (2020) 213.
[8]
Alhagry S., Fahmy A.A., El-Khoribi R.A., Emotion recognition based on EEG using LSTM recurrent neural network, Emotion 8 (10) (2017) 355–358.
[9]
Lahiri R., Rakshit P., Konar A., Evolutionary perspective for optimal selection of EEG electrodes and features, Biomed. Signal Process. Control 36 (2017) 113–137.
[10]
F. Shi, J.-L. Marini, E. Audry, Towards a psycho-cognitive recommender system, in: Proceedings of the International Workshop on Emotion Representations and Modelling for Companion Technologies, 2015, pp. 25–31.
[11]
Pappas I.O., Giannakos M.N., Mikalef P., Investigating students’ use and adoption of with-video assignments: lessons learnt for video-based open educational resources, J. Comput. Higher Educ. 29 (1) (2017) 160–177.
[12]
Sharma K., Papamitsiou Z., Giannakos M., Building pipelines for educational data using AI and multimodal analytics: A “grey-box” approach, Br. J. Educ. Technol. 50 (6) (2019) 3004–3031.
[13]
Papamitsiou Z., Pappas I.O., Sharma K., Giannakos M., Utilizing multimodal data through fsQCA to explain engagement in adaptive learning, IEEE Trans. Learn. Technol. 13 (4) (2020) 689–703.
[14]
Bhaskar A., Qu M., Chung E., Bluetooth vehicle trajectory by fusing bluetooth and loops: Motorway travel time statistics, IEEE Trans. Intell. Transp. Syst. 16 (1) (2014) 113–122.
[15]
Sodhi M., Reimer B., Llamazares I., Glance analysis of driver eye movements to evaluate distraction, Behav. Res. Methods Instrum. Comput. 34 (4) (2002) 529–538.
[16]
I. Ekman, L. Ermi, J. Lahti, J. Nummela, P. Lankoski, F. Mäyrä, Designing sound for a pervasive mobile game, in: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, 2005, pp. 110–116.
[17]
A. McNamara, K. Mania, G. Koulieris, L. Itti, Attention-aware rendering, mobile graphics and games, in: Proceedings of the 2014 ACM SIGGRAPH Courses, 2014, pp. 1–119.
[18]
Pejsa T., Andrist S., Gleicher M., Mutlu B., Gaze and attention management for embodied conversational agents, ACM Trans. Interact. Intell. Syst. (TiiS) 5 (1) (2015) 1–34.
[19]
H. Jiang, K. Dykstra, J. Whitehill, Predicting when teachers look at their students in 1-on-1 tutoring sessions, in: Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, 2018, pp. 593–598.
[20]
Admoni H., Scassellati B., Social eye gaze in human-robot interaction: A review, J. Hum.-Robot Interact. 6 (1) (2017) 25–63.
[21]
Giannakos M.N., Jaccheri L., Krogstie J., Exploring the relationship between video lecture usage patterns and students’ attitudes, Br. J. Educ. Technol. 47 (6) (2016) 1259–1275.
[22]
Abeysekera L., Dawson P., Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research, High. Educ. Res. Dev. 34 (1) (2015) 1–14.
[23]
Wei X., Cheng I.-L., Chen N.-S., Yang X., Liu Y., Dong Y., Zhai X., et al., Effect of the flipped classroom on the mathematics performance of middle school students, Educ. Technol. Res. Dev. 68 (3) (2020) 1461–1485.
[24]
Balakrishnan G., Coetzee D., Predicting student retention in massive open online courses using hidden markov models, Electr. Eng. Comput. Sci. Univ. California At Berkeley 53 (2013) 57–58.
[25]
Giannakos M.N., Krogstie J., Aalberg T., Video-based learning ecosystem to support active learning: application to an introductory computer science course, Smart Learn. Environ. 3 (1) (2016) 1–13.
[26]
Roach T., Student perceptions toward flipped learning: New methods to increase interaction and active learning in economics, Int. Rev. Econ. Educ. 17 (2014) 74–84.
[27]
Chao C.-Y., Chen Y.-T., Chuang K.-Y., Exploring students’ learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education, Comput. Appl. Eng. Educ. 23 (4) (2015) 514–526.
[28]
Lin F.-R., Kao C.-M., Mental effort detection using EEG data in E-learning contexts, Comput. Educ. 122 (2018) 63–79.
[29]
Hogervorst M.A., Brouwer A.-M., van Erp J.B.F., Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload, Front. Neurosci. 8 (2014) 322.
[30]
Hwang G.-J., Lai C.-L., Wang S.-Y., Seamless flipped learning: a mobile technology-enhanced flipped classroom with effective learning strategies, J. Comput. Educ. 2 (4) (2015) 449–473.
[31]
C. Savvaki, A. Leonidis, G. Paparoulis, M. Antona, C. Stephanidis, Designing a technology–augmented school desk for the future classroom, in: Proceedings of the International Conference on Human-Computer Interaction, 2013, pp. 681–685.
[32]
D. Szafir, B. Mutlu, ARTFul: adaptive review technology for flipped learning, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013, pp. 1001–1010.
[33]
Mühl C., Jeunet C., Lotte F., EEG-Based workload estimation across affective contexts, Front. Neurosci. 8 (2014) 114.
[34]
Yang T.-Y., Brinton C.G., Joe-Wong C., Chiang M., Behavior-based grade prediction for MOOCs via time series neural networks, IEEE J. Sel. Top. Sign. Proces. 11 (5) (2017) 716–728.
[35]
Kim Y., Soyata T., Behnagh R.F., Towards emotionally aware AI smart classroom: Current issues and directions for engineering and education, IEEE Access 6 (2018) 5308–5331.
[36]
D.-Y. Lee, M. Lee, S.-W. Lee, Classification of imagined speech using siamese neural network, in: Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020, pp. 2979–2984.
[37]
Muzammal M., Talat R., Sodhro A.H., Pirbhulal S., A multi-sensor data fusion enabled ensemble approach for medical data from body sensor networks, Inf. Fusion 53 (2020) 155–164.
[38]
Kar P., Chattopadhyay S., Chakraborty S., Gestatten: Estimation of user’s attention in mobile MOOCs from eye gaze and gaze gesture tracking, Proc. ACM Hum. -Comput. Interact. 72 (4) (2020) 1–32.
[39]
Giannakos M.N., Sharma K., Pappas I.O., Kostakos V., Velloso E., Multimodal data as a means to understand the learning experience, Int. J. Inf. Manage. 48 (2019) 108–119.
[40]
Liang N.-Y., Saratchandran P., Huang G.-B., Sundararajan N., Classification of mental tasks from EEG signals using extreme learning machine, Int. J. Neural Syst. 16 (01) (2006) 29–38.
[41]
R. Bitner, N.-T. Le, N. Pinkwart, A concurrent validity approach for EEG-based feature classification algorithms in learning analytics, in: Proceedings of the International Conference on Computational Collective Intelligence, 2020, pp. 568–580.
[42]
Naseer N., Hong K.-S., fNIRS-based brain-computer interfaces: a review, Front. Hum. Neurosci. 9 (2015) 3.
[43]
S.S. Nath, D. Mukhopadhyay, K.P. Miyapuram, Emotive stimuli-triggered participant-based clustering using a novel split-and-merge algorithm, in: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, 2019, pp. 277–280.
[44]
A. Sinha, D. Chatterjee, S.K. Saha, A. Basu, Validation of stimulus for EEG signal based cognitive load analysis, in: Proceedings of the 2015 IEEE Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), 2015, pp. 1–4.
[45]
Atkinson J., Campos D., Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers, Expert Syst. Appl. 47 (2016) 35–41.
[46]
S. Tripathi, S. Acharya, R.D. Sharma, S. Mittal, S. Bhattacharya, Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset, in: Proceedings of Thr Twenty-Ninth IAAI Conference, 2017, pp. 4746–4752.
[47]
Ram D., Miculicich L., Bourlard H., Neural network based end-to-end query by example spoken term detection, IEEE/ACM Trans. Audio Speech Lang. Process. 28 (2020) 1416–1427.
[48]
T.M. Rath, R. Manmatha, Word image matching using dynamic time warping, in: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, 2003, pp. 1–7.
[49]
Zhang Y., Zhao H., Land–use and land-cover change detection using dynamic time warping–based time series clustering method, Can. J. Remote Sens. 46 (1) (2020) 67–83.
[50]
Juang B.-H., On the hidden Markov model and dynamic time warping for speech recognition—A unified view, AT&T Bell Lab. Tech. J. 63 (7) (1984) 1213–1243.
[51]
Zheng Z., Zhou M., Chen Y., Huo M., Sun L., Zhao S., Chen D., A fused method of machine learning and dynamic time warping for road anomalies detection, IEEE Trans. Intell. Transp. Syst. (2020) 1–13.
[52]
E.G. Flórez, S. Cardona, L. Jordi, Analysis of vibrations using wavelet transform and stft to characterize defects in wheel-rail contact, in: Proceedings of the 11th International Research Conference Trends in the Development of Machinery and Associated Technology, 2007, pp. 1519–1522.
[53]
Heil C.E., Walnut D.F., Continuous and discrete wavelet transforms, SIAM Rev. 31 (4) (1989) 628–666.
[54]
Shensa M.J., The discrete wavelet transform: wedding the a trous and mallat algorithms, IEEE Trans. Signal Process. 40 (10) (1992) 2464–2482.
[55]
Talukder K.H., Harada K., Haar wavelet based approach for image compression and quality assessment of compressed image, 2010, arXiv preprint arXiv:1010.4084.
[56]
Ali F., El-Sappagh S., Islam S.R., Kwak D., Ali A., Imran M., Kwak K.-S., A smart healthcare monitoring system for heart disease prediction based on ensemble deep learning and feature fusion, Inf. Fusion 63 (2020) 208–222.
[57]
Brunetti A., Buongiorno D., Trotta G.F., Bevilacqua V., Computer vision and deep learning techniques for pedestrian detection and tracking: A survey, Neurocomputing 300 (2018) 17–33.
[58]
Sreepada R.S., Patra B.K., Mitigating long tail effect in recommendations using few shot learning technique, Expert Syst. Appl. 140 (2020).
[59]
Zinovyeva E., Härdle W.K., Lessmann S., Antisocial online behavior detection using deep learning, Decis. Support Syst. 138 (2020) 113–362.
[60]
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, R. Shah, Signature verification using a “siamese” time delay neural network, in: Proceedings of the Advances in Neural Information Processing Systems, 1994, pp. 737–744.
[61]
G. Koch, R. Zemel, R. Salakhutdinov, Siamese neural networks for one-shot image recognition, in: Proceedings of the ICML Deep Learning Workshop, Vol. 2, 2015.
[62]
Norouzi M., Mikolov T., Bengio S., Singer Y., Shlens J., Frome A., Corrado G.S., Dean J., Zero-shot learning by convex combination of semantic embeddings, 2013, arXiv preprint arXiv:1312.5650.
[63]
S. Zhao, F. Rudzicz, Classifying phonological categories in imagined and articulated speech, in: Proceedings 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 992–996.
[64]
G. Salomon, A. Britto, R.H. Vareto, W.R. Schwartz, D. Menotti, Open-set Face Recognition for Small Galleries Using Siamese Networks, in: Proceedings of the 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), 2020, pp. 161–166.
[65]
L. Bertinetto, J. Valmadre, J.F. Henriques, A. Vedaldi, P.H. Torr, Fully-convolutional siamese networks for object tracking, in: Proceedings of the European Conference on Computer Vision, 2016, pp. 850–865.
[66]
K. Chen, A. Salman, Extracting speaker-specific information with a regularized siamese deep network, in: Proceedings of the Advances in Neural Information Processing Systems, 2011, pp. 298–306.
[67]
N. McLaughlin, J.M. Del Rincon, P. Miller, Recurrent convolutional network for video-based person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1325–1334.
[68]
C. Zhang, W. Liu, H. Ma, H. Fu, Siamese neural network based gait recognition for human identification, in: Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 2832–2836.
[69]
Tuncer T., Dogan S., Ertam F., Subasi A., A novel ensemble local graph structure based feature extraction network for EEG signal analysis, Biomed. Signal Process. Control 61 (2020).
[70]
Tai S.-C., Sun C., Yan W.-C., A 2-D ECG compression method based on wavelet transform and modified SPIHT, IEEE Trans. Biomed. Eng. 52 (6) (2005) 999–1008.
[71]
Bilgin A., Marcellin M.W., Altbach M.I., Compression of electrocardiogram signals using JPEG2000, IEEE Trans. Consum. Electron. 49 (4) (2003) 833–840.
[72]
R. Hadsell, S. Chopra, Y. LeCun, Dimensionality reduction by learning an invariant mapping, in: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2006, pp. 1735–1742.
[73]
Sharmila A., Geethanjali P., DWT based detection of epileptic seizure from EEG signals using naive Bayes and k-NN classifiers, IEEE Access 4 (2016) 7716–7727.
[74]
DaSalla C.S., Kambara H., Sato M., Koike Y., Single-trial classification of vowel speech imagery using common spatial patterns, Neural Netw. 22 (9) (2009) 1334–1339.
[75]
Bablani A., Edla D.R., Dodia S., Classification of EEG data using k-nearest neighbor approach for concealed information test, Procedia Comput. Sci. 143 (2018) 242–249.
[76]
S.-H. Lee, M. Lee, J.-H. Jeong, S.-W. Lee, Towards an EEG-based intuitive BCI communication system using imagined speech and visual imagery, in: Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019, pp. 4409–4414.
[77]
Kaya M., Bilge H.Ş., Deep metric learning: A survey, Symmetry 11 (9) (2019) 1066.
[78]
Cogswell M., Ahmed F., Girshick R., Zitnick L., Batra D., Reducing overfitting in deep networks by decorrelating representations, 2015, arXiv preprint arXiv:1511.06068.

Cited By

View all
  • (2024)Teacher-student interaction modes in smart classroom based on lag sequential analysisEducation and Information Technologies10.1007/s10639-024-12487-429:12(15087-15111)Online publication date: 1-Aug-2024
  • (2023)Towards Smart Education through Internet of Things: A SurveyACM Computing Surveys10.1145/361040156:2(1-33)Online publication date: 21-Jul-2023
  • (2022)A Teaching Model for a Five-week Electric Circuits Course for Engineering StudentsProceedings of the 14th International Conference on Education Technology and Computers10.1145/3572549.3572562(76-81)Online publication date: 28-Oct-2022
  • Show More Cited By

Index Terms

  1. Classifying students based on cognitive state in flipped learning pedagogy
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image Future Generation Computer Systems
        Future Generation Computer Systems  Volume 126, Issue C
        Jan 2022
        340 pages

        Publisher

        Elsevier Science Publishers B. V.

        Netherlands

        Publication History

        Published: 01 January 2022

        Author Tags

        1. Flipped learning (FL)
        2. Electroencephalography (EEG)
        3. Siamese neural network (SNN)
        4. Discrete wavelet transform (DWT)
        5. Cognitive state (CS)
        6. Classification

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)0
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 31 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Teacher-student interaction modes in smart classroom based on lag sequential analysisEducation and Information Technologies10.1007/s10639-024-12487-429:12(15087-15111)Online publication date: 1-Aug-2024
        • (2023)Towards Smart Education through Internet of Things: A SurveyACM Computing Surveys10.1145/361040156:2(1-33)Online publication date: 21-Jul-2023
        • (2022)A Teaching Model for a Five-week Electric Circuits Course for Engineering StudentsProceedings of the 14th International Conference on Education Technology and Computers10.1145/3572549.3572562(76-81)Online publication date: 28-Oct-2022
        • (2022)Guest Editorial of the FGCS Special Issue on Advances in Intelligent Systems for Online EducationFuture Generation Computer Systems10.1016/j.future.2021.09.022127:C(331-333)Online publication date: 1-Feb-2022

        View Options

        View options

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media