[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Examining university teachers’ self-regulation in using a learning analytics dashboard for online collaboration

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Learning analytics dashboards (LADs) are often used to display real-time data indicating student learning trajectories and outcomes. Successful use of LADs requires teachers to orient their dashboard reviews with clear goals, apply appropriate strategies to interpret visualized information on LADs and monitor and evaluate their interpretations to meet goals. This process is known as self-regulated learning (SRL). Critical as it is, little research investigates teachers’ SRL in LAD usage. The present study addressed the gap by examining teachers’ SRL and sought to understand how teachers’ SRL relates to their use of LADs. To this end, a case study was designed in which ten participants were invited to use a LAD for asynchronous online problem-based learning. Think-aloud techniques and process mining methods were applied. The findings show that teachers were cognitive regulation in the early stage of LAD usage and became more metacognitive regulated later. The comparison of SRL between the good and the weak regulators indicates that the good self-regulators enacted more monitoring and evaluation events. Thus their regulator pattern was more non-linear. The qualitative analysis of think-aloud protocols reveals that teachers with good SRL are more likely to use the LAD to diagnose issues in student learning and collaboration. The study highlights the importance of SRL for teachers’ success in using LAD for data-driven instructions. The study also reinforces the importance of fostering teachers’ SRL, which accounts for teachers’ professional success in the digital era.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets generated during and/or analyzed during the current study are not publicly available due to the Ethics requirements but are available from the corresponding author upon reasonable request.

Notes

  1. Lemon, Apple, Banana, Orange, and Pear are the aliases of individual groups.

  2. We did not ask the participants to use the exact terms. We considered their decisions accurate as long as their described the core features of a given dynamics. For example, instead of using the off-topic group, participants can state that students’ discussions were out of the topics or students’ discussions were little related with the topics.

  3. The names mentioned by teachers were the alias that represented participating students.

References

  • Azevedo, R. (2009). Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: A discussion. Metacognition and Learning, 4(1), 87–95. https://doi.org/10.1007/s11409-009-9035-7

    Article  Google Scholar 

  • Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students’ learning with hypermedia? Journal of Educational Psychology, 96(3), 523–535. https://doi.org/10.1037/0022-0663.96.3.523

    Article  Google Scholar 

  • Azevedo, R., Guthrie, J. T., & Seibert, D. (2004). The role of self-regulated learning in fostering students’ conceptual understanding of complex systems with hypermedia. Journal of Educational Computing Research, 30(1–2), 87–111. https://doi.org/10.2190/DVWX-GM1T-6THQ-5WC7

    Article  Google Scholar 

  • Bakharia, A., & Dawson, S. (2011). In P. Long, G. Siemens, G. Conole, & D. Gašević (Eds.), SNAPP (p. 168). ACM. https://doi.org/10.1145/2090116.2090144

  • Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9(2), 161–185. https://doi.org/10.1007/s11409-013-9107-6

    Article  Google Scholar 

  • Bingham, L. B., & O’Leary, R. (2006). Conclusion: Parallel play, not collaboration: Missing questions, missing connections. Public Administration Review, 66, 161–167. https://doi.org/10.1111/j.1540-6210.2006.00686.x

  • Chen, Y., Hmelo-Silver, C. E., Lajoie, S. P., Zheng, J., Huang, L., & Bodnar, S. (2021). Using teacher dashboards to assess group collaboration in problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 15(2). https://doi.org/10.14434/ijpbl.v15i2.28792

  • Creswell, J. W. (1999). Chapter 18 - Mixed-method research: Introduction and application. In G. J. Cizek (Ed.), Handbook of Educational Policy (pp. 455–472). Academic Press. https://doi.org/10.1016/B978-012174698-8/50045-X

  • Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. https://doi.org/10.1037/0033-295X.87.3.215

    Article  Google Scholar 

  • Fan, Y., van der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., Moore, J., Molenaar, I., Bannert, M., & Gašević, D. (2022). Towards investigating the validity of measurement of self-regulated learning based on trace data. Metacognition and Learning. https://doi.org/10.1007/s11409-022-09291-1

    Article  Google Scholar 

  • Few, S. (2013). Information Dashboard Design: Displaying data for at-a-glance monitoring (2nd ed.). Analytics Press.

  • Greene, J. A., Bolick, C. M., & Robertson, J. (2010). Fostering historical knowledge and thinking skills using hypermedia learning environments: The role of self-regulated learning. Computers & Education, 54(1), 230–243. https://doi.org/10.1016/j.compedu.2009.08.006

    Article  Google Scholar 

  • Greene, J. A., Plumley, R. D., Urban, C. J., Bernacki, M. L., Gates, K. M., Hogan, K. A., Demetriou, C., & Panter, A. T. (2021). Modeling temporal self-regulatory processing in a higher education biology course. Learning and Instruction, 72, 101201. https://doi.org/10.1016/j.learninstruc.2019.04.002

  • Greene, J., & Azevedo, R. (2009). A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemporary Educational Psychology, 34(1), 18–29. https://doi.org/10.1016/j.cedpsych.2008.05.006

    Article  Google Scholar 

  • Greene, J., & Azevedo, R. (2010). The measurement of learners’ self-regulated cognitive and metacognitive processes while using computer-based learning environments. Educational Psychologist, 45(4). https://doi.org/10.1080/00461520.2010.515935

  • Guo, L. (2022). Using metacognitive prompts to enhance self-regulated learning and learning outcomes: A meta-analysis of experimental studies in computer-based learning environments. Journal of Computer Assisted Learning, 38(3), 811–832. https://doi.org/10.1111/jcal.12650

    Article  Google Scholar 

  • Günther, C. W., & van der Aalst, W. M. P. (2007). Fuzzy Mining – Adaptive process simplification based on multi-perspective metrics. In G. Alonso (Ed.), Business process management. 5th International Conference, BPM 2007, Brisbane, Australia, September 24-28, 2007; proceedings (Vol. 4714, pp. 328–343). Springer. https://doi.org/10.1007/978-3-540-75183-0_24

  • Harkins, S. G., & Szymanski, K. (1989). Social loafing and group evaluation. Journal of Personality and Social Psychology, 56(6), 934–941. https://doi.org/10.1037/0022-3514.56.6.934

    Article  Google Scholar 

  • Hendry, G. D., Ryan, G., & Harris, J. (2003). Group problems in problem-based learning. Medical Teacher, 25(6), 609–616. https://doi.org/10.1080/0142159031000137427

    Article  Google Scholar 

  • Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., & Hlosta, M. (2019). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective. Educational Technology Research and Development, 67(5), 1273–1306. https://doi.org/10.1007/s11423-019-09685-0

    Article  Google Scholar 

  • Hofer, M. (2007). Goal conflicts and self-regulation: A new look at pupils’ off-task behaviour in the classroom. Educational Research Review, 2(1), 28–38. https://doi.org/10.1016/j.edurev.2007.02.002

    Article  Google Scholar 

  • Hogaboam, P. T., Chen, Y., Hmelo-Silver, C. E., Lajoie, S. P., Bodnar, S., Kazemitabar, M., Wiseman, J., & Chan, L. K. (2016). Data dashboards to support facilitating online problem-based learning. The Quarterly Review of Distance Education, 17(3), 75–79.

    Google Scholar 

  • Huang, L., & Lajoie, S. P. (2021). Process analysis of teachers’ self-regulated learning patterns in technological pedagogical content knowledge development. Computers & Education, 166, 104–169. https://doi.org/10.1016/j.compedu.2021.104169

    Article  Google Scholar 

  • Huang, X., Huang, L., & Lajoie, S. P. (2022). Exploring teachers’ emotional experience in a TPACK development task. Educational Technology Research and Development. https://doi.org/10.1007/s11423-022-10135-7

    Article  Google Scholar 

  • Kaliisa, R., & Dolonen, J. A. (2023). CADA: A teacher-facing learning analytics dashboard to foster teachers’ awareness of students’ participation and discourse patterns in online discussions. Technology, Knowledge and Learning, 28(3), 937–958. https://doi.org/10.1007/s10758-022-09598-7

    Article  Google Scholar 

  • Kazemitabar, M. A., Bodnar, S., Hogaboam, P., Chen, Y., Sarmiento, J. P., Lajoie, S. P., Hmelo-Silver, C., Goldman, R., Wiseman, J., & Chan, L. (2016). Creating instructor dashboards to foster collaborative learning in on-line medical problem-based learning Situations. In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. LCT 2016. Lecture Notes in Computer Science (Vol. 9753, pp. 36–47). Springer. https://doi.org/10.1007/978-3-319-39483-1_4

  • Kramarski, B., & Michalsky, T. (2015). Effect of a TPCK-SRL model on teachers’ pedagogical beliefs, self-efficacy, and technology-based lesson design. In C. Angeli (Ed.), Technological pedagogical content knowledge: Exploring, developing, and assessing TPCK (pp. 89–112). Springer. https://doi.org/10.1007/978-1-4899-8080-9_5

  • Lajoie, S. P., Hmelo-Silver, C. E., Wiseman, J. G., Chan, L. K., Lu, J., Khurana, C., Cruz-Panesso, I., Poitras, E., & Kazemitabar, M. (2014). Using online digital tools and video to support international problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 8(2). https://doi.org/10.7771/1541-5015.1412

  • Lajoie, S. P., Poitras, E. G., Doleck, T., & Jarrell, A. (2015). Modeling metacognitive activities in medical problem-solving with BioWorld. In A. Peña-Ayala (Ed.), Metacognition: Fundaments, applications, and trends (Vol. 76, pp. 323–343). Springer. https://doi.org/10.1007/978-3-319-11062-2_13

  • Li, Y., Zhang, M., Su, Y., Bao, H., & Xing, S. (2022). Examining teachers’ behavior patterns in and perceptions of using teacher dashboards for facilitating guidance in CSCL. Educational Technology Research and Development. https://doi.org/10.1007/s11423-022-10102-2

    Article  Google Scholar 

  • Molenaar, I., & Järvelä, S. (2014). Sequential and temporal characteristics of self and socially regulated learning. Metacognition and Learning. https://doi.org/10.1007/S11409-014-9114-2

    Article  Google Scholar 

  • Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education (pp. 125–138). Springer International Publishing. https://doi.org/10.1007/978-3-319-66610-5_10

  • Park, Y., & Jo, I.-H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 24110–24133.

    Google Scholar 

  • Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). Academic. https://doi.org/10.1016/B978-012109890-2/50043-3

  • Poitras, E., Doleck, T., Huang, L., Li, S., & Lajoie, S. P. (2017). Advancing teacher technology education using open-ended learning environments as research and training platforms. Australasian Journal of Educational Technology, 33(3). https://doi.org/10.14742/ajet.3498

  • Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y.-S., Echeverria, V., Srivastava, N., & Gasevic, D. (2023). How do teachers use dashboards enhanced with data storytelling elements according to their data visualisation literacy skills? LAK23: 13th International Learning Analytics and Knowledge Conference, 89–99. https://doi.org/10.1145/3576050.3576063

  • Puustinen, M., & Pulkkinen, L. (2001). Models of Self-regulated learning: A review. Scandinavian Journal of Educational Research, 45(3), 269–286. https://doi.org/10.1080/00313830120074206

    Article  Google Scholar 

  • Quintão, C., Andrade, P., & Almeida, F. (2020). How to improve the validity and reliability of a case study approach? Journal of Interdisciplinary Studies in Education, 9(2), 273–284. https://doi.org/10.32674/jise.v9i2.2026

  • Reimann, P., Markauskaite, L., & Bannert, M. (2014). e-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540. https://doi.org/10.1111/bjet.12146

    Article  Google Scholar 

  • Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A Technology acceptance perspective of 95 teachers. The International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3493

  • Rozinat, A., & van der Aalst, W. M. P. (2008). Conformance checking of processes based on monitoring real behavior. Information Systems, 33(1), 64–95. https://doi.org/10.1016/j.is.2007.07.001

    Article  Google Scholar 

  • Saint, J., Fan, Y., Gašević, D., & Pardo, A. (2022). Temporally-focused analytics of self-regulated learning: A systematic review of literature. Computers and Education: Artificial Intelligence, 3, 100060. https://doi.org/10.1016/j.caeai.2022.100060

  • Schwendimann, B. A., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(01), 30–41. https://doi.org/10.1109/TLT.2016.2599522

    Article  Google Scholar 

  • Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004

  • Sedrakyan, G., Mannens, E., & Verbert, K. (2019). Guiding the choice of learning dashboard visualizations: Linking dashboard design and data visualization concepts. Journal of Computer Languages, 50, 19–38. https://doi.org/10.1016/j.jvlc.2018.11.002

    Article  Google Scholar 

  • Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21(5), 619–634. https://doi.org/10.1080/13645579.2018.1454643

    Article  Google Scholar 

  • Sonnenberg, C., & Bannert, M. (2019). Using process mining to examine the sustainability of instructional support: How stable are the effects of metacognitive prompting on self-regulatory behavior? Computers in Human Behavior. https://doi.org/10.1016/j.chb.2018.06.003

    Article  Google Scholar 

  • Tabassum, S., Pereira, F. S. F., Fernandes, S., & Gama, J. (2018). Social network analysis: An overview. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(5). https://doi.org/10.1002/widm.1256

    Article  Google Scholar 

  • Taub, M., & Azevedo, R. (2018). Using sequence mining to analyze metacognitive monitoring and scientific inquiry based on levels of efficiency and emotions during game-based learning. Journal of Educational Data Mining, 10(3), 1–26. https://doi.org/10.5281/zenodo.3554711

    Article  Google Scholar 

  • Taub, M., Azevedo, R., Rajendran, R., Cloude, E. B., Biswas, G., & Price, M. J. (2021). How are students’ emotions related to the accuracy of cognitive and metacognitive processes during learning with an intelligent tutoring system? Learning and Instruction, 72, 101200. https://doi.org/10.1016/j.learninstruc.2019.04.001

  • Taub, M., Mudrick, N. V., Azevedo, R., Millar, G. C., Rowe, J., & Lester, J. (2016). Using multi-level modeling with eye-tracking data to predict metacognitive monitoring and self-regulated learning with Crystal Island. In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Intelligent Tutoring Systems (pp. 240–246). Springer International Publishing. https://doi.org/10.1007/978-3-319-39583-8_24

  • van der Aalst, W. (2012). Process mining: Overview and opportunities. ACM Transactions on Management Information Systems, 3(2), 7:1–7:17. https://doi.org/10.1145/2229156.2229157

  • van Leeuwen, A., & Janssen, J. (2019). A systematic review of teacher guidance during collaborative learning in primary and secondary education. Educational Research Review, 27, 71–89. https://doi.org/10.1016/j.edurev.2019.02.001

    Article  Google Scholar 

  • Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509. https://doi.org/10.1177/0002764213479363

    Article  Google Scholar 

  • Wang, Y. (2020). What is the role of emotions in educational leaders’ decision-making? Proposing an organizing framework. Educational Administration Quarterly, 1–31. https://doi.org/10.1177/0013161X20938856

  • Winne, P. H. (2011). Cognition and metacognition within self-regulated learning. In D. H. Schunk & J. A. Greene (Eds.), Handbook of Self-Regulation of Learning and Performance (pp. 36–48). Routledge.

    Google Scholar 

  • Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (First, pp. 241–249). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.021

  • Winne, P. H. (2018). Theorizing and researching levels of processing in self-regulated learning. The British Journal of Educational Psychology, 88(1), 9–20. https://doi.org/10.1111/bjep.12173

    Article  Google Scholar 

  • Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), The educational psychology series. Metacognition in educational theory and practice (pp. 277–304). Lawrence Erlbaum Associates Publishers.

  • Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self-reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572. https://doi.org/10.1016/S0361-476X(02)00006-1

    Article  Google Scholar 

  • Wise, A. F., & Jung, Y. (2019). Teaching with analytics: towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2). https://doi.org/10.18608/jla.2019.62.4

  • Zheng, J., Huang, L., Li, S., Lajoie, S. P., Chen, Y., & Hmelo-Silver, C. E. (2021). Self-regulation and emotion matter: A case study of instructor interactions with a learning analytics dashboard. Computers & Education, 161, 104061. https://doi.org/10.1016/j.compedu.2020.104061

Download references

Acknowledgements

Thanks to all participants for their time and efforts. We also appreciate the editors and reviewers for their constructive feedback and comments.

Funding

This research was supported in part by the doctoral scholarship from Fonds de Recherche du Québec—Société’s et Culture (FRQSC) and RGC Post-doctoral Fellowship (PDFS2122-7H03) awarded to Dr. Lingyun Huang, and Social Sciences and Humanities Research Council (SSHRC) Partnership Grant of Canada (895–2011-1006) awarded to Prof. Susanne Lajoie.

Author information

Authors and Affiliations

  1. Department of Curriculum and Instruction, Faculty of Education and Human Development, The Education University of Hong Kong, Hong Kong SAR, China

    Lingyun Huang

  2. Department of Education & Human Services, College of Education, Lehigh University, Bethlehem, PA, USA

    Juan Zheng

  3. Department of Educational and Counselling Psychology, McGill University, Montreal, QC, Canada

    Susanne P. Lajoie

  4. University of South Alabama, Mobile, AL, USA

    Yuxin Chen

  5. School of Education, Indiana University Bloomington, Bloomington, IN, USA

    Cindy E. Hmelo-Silver

  6. Faculty of Education, The Hong Kong University, Hong Kong SAR, China

    Minhong Wang

Authors
  1. Lingyun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susanne P. Lajoie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuxin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cindy E. Hmelo-Silver
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Minhong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Lingyun Huang: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing-Original draft, Writing—Review & Editing.

Juan Zheng. Formal analysis, Investigation, Writing—Review & Editing.

Susanne Lajoie: Supervision, Writing—Review & Editing.

Yuxin Chen: Investigation, Writing—Review & Editing.

Cindy Hmelo-Silver: Supervision, Writing—Review & Editing.

Minhong Wang: Review & Editing.

Corresponding author

Correspondence to Lingyun Huang.

Ethics declarations

Ethical approval

The research obtained Ethics approval from the Research Ethics Board (IRB) of McGill University. All procedures performed in this research were in accordance with the ethical guidelines of the IRB of McGill University.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, L., Zheng, J., Lajoie, S.P. et al. Examining university teachers’ self-regulation in using a learning analytics dashboard for online collaboration. Educ Inf Technol 29, 8523–8547 (2024). https://doi.org/10.1007/s10639-023-12131-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-023-12131-7

Keywords

Search

Navigation