Abstract
Analysis of learning behavior of MOOC enthusiasts has become a posed challenge in the Learning Analytics field, which is especially related to video lecture data, since most learners watch the same online lecture videos. It helps to conduct a comprehensive analysis of such behaviors and explore various learning patterns for learners and predict their performance by MOOC courses video. This paper exploits a temporal sequential classification problem by analyzing video clickstream data and predict learner performance, which is a vital decision-making problem, by addressing their issues and improving the educational process. This paper employs a deep neural network (LSTM) on a set of implicit features extracted from video clickstreams data to predict learners’ weekly performance and enable instructors to set measures for timely intervention. Results show that accuracy rate of the proposed model is 82%–93% throughout course weeks. The proposed LSTM model outperforms baseline ANNs, Super Vector Machine (SVM) and Logistic Regression by an accuracy of 93% in real used courses’ datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
The data that support the findings of this study are available at the Center for Advanced Research Through Online Learning (CAROL) at the University of Stanford https://Iriss.Stanford.Edu/Carol, but restrictions apply to the availability of these data, which were used under license for the current study, and so that they are not publicly available. Data are however available with the authors upon reasonable request and with permission of CAROL at the University of Stanford.
References
Atapattu, T., & Falkner, K. (2018). Impact of Lecturer’s discourse for student video interactions: Video learning analytics case study of MOOCs. Journal of Learning Analytics, 5(3), 182–197. https://doi.org/10.18608/jla.2018.53.12.
Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning Analytics: From Research to Practice (pp. 61–75). New York: Springer. https://doi.org/10.1007/978-1-4614-3305-7_4.
Brinton, C, G., Buccapatnam, S., Chiang, M., & Poor, H, V. (2015). Mining MOOC clickstreams: On the relationship between learner behavior and performance.
Chakraborty, S., Preece, A., Alzantot, M., Xing, T., Braines, D., & Srivastava, M. (2017). Deep learning for situational understanding. 20th International Conference on Information Fusion, Fusion 2017 - Proceedings. https://doi.org/10.23919/ICIF.2017.8009785.
Chorianopoulos, K. (2013). Collective intelligence within web video. Human-centric Computing and Information Sciences, 3(1), 10.
Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695.
Coelho, O, B., & Silveira, I. (2017). Deep learning applied to learning analytics and educational data mining: A systematic literature review. Anais Do XXVIII Simpósio Brasileiro de Informática Na Educação (SBIE 2017), 1, 143. https://doi.org/10.5753/cbie.sbie.2017.143
Corrigan, O., & Smeaton, A, F. (2017). A course agnostic approach to predicting student success from vle log data using recurrent neural networks. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10474 LNCS, 545–548. https://doi.org/10.1007/978-3-319-66610-5_59.
De Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32(3), 218–231.
Fauvel, S., & Yu, H. (2016). A survey on artificial intelligence and data mining for MOOCs. http://arxiv.org/abs/1601.06862
Fei, M., & Yeung, D.-Y. (2015). Temporal models for predicting student dropout in massive open online courses. In Data mining workshop (ICDMW), 2015 IEEE international conference on (pp. 256–263).
Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2015). Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. International review of research in open and distributed learning, 16(1), 260–283. https://doi.org/10.19173/irrodl.v16i1.1976.
Graupe, D. (2016). Other neural networks for deep learning. In Deep Learning Neural Networks (pp. 101–109). WORLD SCIENTIFIC. https://doi.org/10.1142/9789813146464_0007.
Gross, E., Wshah, S., Skinner, G., & Simmons, I. (2015). A handwriting recognition system for the classroom. In ACM international conference proceeding series, 16–20-Marc (pp. 218–222). https://doi.org/10.1145/2723576.2723601.
Guo, P, J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. Proceedings of the First ACM Conference on Learning@ Scale Conference, 41–50.
Halawa, S., Greene, D., & Mitchell, J. (2014). Dropout prediction in MOOCs using learner activity features. Proceedings of the Second European MOOC Stakeholder Summit, 37(1), 58–65.
Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
Hsin, W.-J., & Cigas, J. (2013). Short videos improve student learning in online education. Journal of Computing Sciences in Colleges, 28(5), 253–259.
Kim, J., Guo, P, J., Seaton, D, T., Mitros, P., Gajos, K, Z., & Miller, R, C. (2014). Understanding in-video dropouts and interaction peaks inonline lecture videos. Proceedings of the First ACM Conference on Learning@ Scale Conference, 31–40.
Kingma, D, P., & Ba, J. (2014). Adam: A method for stochastic optimization. ArXiv Preprint ArXiv:1412.6980.
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. In ACM international conference proceeding series (pp. 170–179). https://doi.org/10.1145/2460296.2460330.
Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539.
Li, N., Kidzinski, L., Jermann, P., & Dillenbourg, P. (2015a). How do in-video interactions reflect perceived video difficulty?
Li, N., Kidziński, Ł., Jermann, P., & Dillenbourg, P. (2015b). MOOC video interaction patterns: What do they tell us? In Design for teaching and learning in a networked world (pp. 197–210). Springer.
Marbouti, F., Diefes-Dux, H. A., & Madhavan, K. (2016). Models for early prediction of at-risk students in a course using standards-based grading. Computers & Education, 103, 1–15.
Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students’ early dropout based on their interaction logs in online learning environment. Interactive Learning Environments., 1–20. https://doi.org/10.1080/10494820.2020.1727529.
Okubo, F., Yamashita, T., Shimada, A., & Konomi, S. (2017). Students’ performance prediction using data of multiple courses by recurrent neural network. In Proceedings of the 25th international conference on computers in education, ICCE 2017 - Main conference proceedings (pp. 439–444) https://kyushu-u.pure.elsevier.com/en/publications/students-performance-prediction-using-data-of-multiple-courses-by.
Olah, C. (2015). Understanding LSTM Networks [Blog]. Web Page, 1–13. https://doi.org/10.1007/s13398-014-0173-7.2.
Palmer, S. (2013). Modelling engineering student academic performance using academic analytics. International Journal of Engineering Education, 29(1), 132–138 http://dro.deakin.edu.au/view/DU:30051021.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(Oct), 2825–2830.
Perry, J. W., Kent, A., & Berry, M. M. (1955). Machine literature searching x. machine language; factors underlying its design and development. American Documentation, 6(4), 242–254.
Risko, E. F., Foulsham, T., Dawson, S., & Kingstone, A. (2012). The collaborative lecture annotation system (CLAS): A new TOOL for distributed learning. IEEE Transactions on Learning Technologies, 6(1), 4–13.
Sak, H, H., Senior, A., & Google, B. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. https://research.google/pubs/pub43905.pdf
Sak, H, H., Senior, A., & Google, B. (n.d.). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In ieeexplore.ieee.org. Retrieved April 28, 2020, from https://ieeexplore.ieee.org/abstract/document/7178816/
Shaw, R., & Davis, M. (2005). Toward emergent representations for video. In Proceedings of the 13th annual ACM international conference on multimedia (pp. 431–434).
Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014a). Your click decides your fate: Inferring information processing and attrition behavior from mooc video clickstream interactions. ArXiv Preprint ArXiv:1407.7131.
Sinha, T., Li, N., Jermann, P., & Dillenbourg, P. (2014b). Capturing” attrition intensifying” structural traits from didactic interaction sequences of MOOC learners. ArXiv Preprint ArXiv:1409.5887.
Smoliar, S. W., & Zhang, H. (1994). Content based video indexing and retrieval. IEEE Multimedia, 1(2), 62–72.
Stanford, U. (2017). Center for advanced research through online learning | Institute for Research in the social sciences.]. In Available: https://iriss.stanford.edu/carol (center for).
Tang, S., Peterson, J, C., & Pardos, Z, A. (2016). Deep neural networks and how they apply to sequential education data. L@S 2016 - Proceedings of the 3rd 2016 ACM Conference on Learning at Scale, 321–324. https://doi.org/10.1145/2876034.2893444.
University, S. (2017). CAROL learner data documentation. In Available: https://datastage.stanford.edu/ (center for).
Waheed, H., Ali, M., Hassan, S.-U., Ventura, S., & Herrera, F. (2019). Virtual learning environment to predict withdrawal by leveraging deep learning. Article in International Journal of Intelligent Systems, 34(8), 1935–1952. https://doi.org/10.1002/int.22129.
Wang, Y., & Baker, R. (2015). Content or platform: Why do students complete MOOCs. MERLOT Journal of Online Learning and Teaching, 11(1), 17–30.
Xu, B., & Yang, D. (2016). Motivation classification and grade prediction for MOOCs learners. Computational Intelligence and Neuroscience, 2016, 1–7. https://doi.org/10.1155/2016/2174613.
Yew, J., Shamma, D, A., & Churchill, E, F. (2011). Knowing funny: Genre perception and categorization in social video sharing. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 297–306.
Yu, C. H., Wu, J., & Liu, A. C. (2019). Predicting learning outcomes with MOOC clickstreams. Education Sciences, 9(2). https://doi.org/10.3390/educsci9020104.
Zhang, G., Eddy Patuwo, B., Hu, Y., & M. (1998). Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting, 14(1), 35–62. https://doi.org/10.1016/S0169-2070(97)00044-7.
Zhang, D., Zhou, L., Briggs, R. O., & Nunamaker Jr., J. F. (2006). Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information & Management, 43(1), 15–27.
Zhang, Q., Yang, L, T., Chen, Z., & Li, P. (2018). A survey on deep learning for big data. In Information Fusion (Vol. 42, pp. 146–157). https://doi.org/10.1016/j.inffus.2017.10.006.
Acknowledgments
The dataset of this research was taken from Stanford University’s Advanced Research Center on Online Learning (CAROL). Thus, we thank immensely for their collaboration with us. We also wish to express our full gratitude to Ms. Kathy Mirzaei for her response and collaboration.
We also thank anonymous reviewers for taking their time to review our paper and providing constructive feedback.
Funding
Not applicable.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mubarak, A.A., Cao, H. & Ahmed, S.A. Predictive learning analytics using deep learning model in MOOCs’ courses videos. Educ Inf Technol 26, 371–392 (2021). https://doi.org/10.1007/s10639-020-10273-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-020-10273-6