More Web Proxy on the site http://driver.im/

short-paper

Evaluating a Learning Analytics Dashboard to Visualize Student Self-Reports of Time-on-task: A Case Study in a Latin American University

Authors:

Isabel Hilliger,

Constanza Miranda,

Gregory Schuit,

Fernando Duarte,

Martin Anselmo,

Denis ParraAuthors Info & Claims

LAK21: LAK21: 11th International Learning Analytics and Knowledge Conference

Pages 592 - 598

https://doi.org/10.1145/3448139.3448203

Published: 12 April 2021 Publication History

Abstract

In recent years, instructional design has become even more challenging for teaching staff members in higher education institutions. If instructional design causes student overload, it could lead to superficial learning and decreased student well-being. A strategy to avoid overload is reflecting upon the effectiveness of teaching practices in terms of time-on-task. This article presents a Work-In-Progress conducted to provide teachers with a dashboard to visualize student self-reports of time-on-task regarding subject activities. A questionnaire was applied to 15 instructors during a set trial period to evaluate the perceived usability and usefulness of the dashboard. Preliminary findings reveal that the dashboard helped instructors became aware about the number of hours spent outside of class time. Furthermore, data visualizations of time-on-task evidence enabled them to redesign subject activities. Currently, the dashboard has been adopted by 106 engineering instructors. Future work involves the development of a framework to incorporate user-based improvements.

References

[1]

A. Bennett and P. J. Burke. 2018. Reconceptualising time and temporality: an exploration of time in higher education, Discourse, vol. 39, no. 6, 913–925. https://doi.org/10.1080/01596306.2017.1312285

[2]

J. R. Ruiz-Gallardo, J. L. González-Geraldo, and S. Castaño, 2016. What are our students doing? Workload, time allocation and time management in PBL instruction. A case study in Science Education, Teach. Teach. Educ., vol. 53, 51–62. https://doi.org/10.1016/j.tate.2015.10.005

[3]

V. Kovanovic, D. Gašević, S. Dawson, S. Joksimovic, and R. Baker. 2016. Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. J. Learn. Anal., vol. 2, no. 3, pp. 81–110. https://doi.org/10.18608/jla.2015.23.6

[4]

Q. Nguyen. 2020. Rethinking time-on-task estimation with outlier detection accounting for individual, time, and task differences. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 376–381. https://doi.org/10.1145/3375462.3375538

Digital Library

[5]

R. Pérez-Álvarez, J. Maldonado-Mahauad, and M. Pérez-Sanagustín, “Tools to Support Self-Regulated Learning in Online Environments: Literature Review,” in EC-TEL, 2018, pp. 16–30.

Digital Library

[6]

D. Jamieson-Noel and P. H. Winne. 2002. Exploring students’ calibration of self reports about study tactics and achievement,” Contemp. Educ. Psychol., vol. 27, no. 4, 551–572. https://doi.org/10.1016/S0361-476X(02)00006-1

[7]

I. Hilliger 2020. For learners, with learners: Identifying indicators for an academic advising dashboard for students. In: Alario-Hoyos C., Rodríguez-Triana M.J., Scheffel M., Arnedillo-Sánchez I., Dennerlein S.M. (eds) Addressing Global Challenges and Quality Education. EC-TEL 2020. Lecture Notes in Computer Science, vol 12315. Springer, Cham. https://doi.org/10.1007/978-3-030-57717-9_9

[8]

Q. Nguyen, B. Rienties, L. Toetenel, R. Ferguson, and D. Whitelock. 2017. Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Comput. Human Behav., vol. 76, 703–714. https://doi.org/10.1016/j.chb.2017.03.028.

Digital Library

[9]

D. Gašević, S. Dawson, T. Rogers, and D. Gasevic. 2016 Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet High. Educ., vol. 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002

[10]

E. J. Maloney and J. Kim. 2020. The Challenge of Equity in Higher Education Under COVID-19. Inside Higher Ed. https://www.insidehighered.com/blogs/learning-innovation/challenge-equity-higher-education-under-covid-19

[11]

M. V. Zelkowitz and D. R. Wallace. 1998. Experimental Models for Validating Technology. Comput. Pract., vol. May, 23–31.

[12]

C. Miranda, J. I. Goñi, I. Hilliger, and J. Lugo. 2020. Assessing the work of geographically distributed teams in engineering-design: Time allocation in the design process as a form of in-class analytics. Int. J. Eng. Educ., vol. 36, no. 1 B, 399–410.

[13]

A. Häggaman, T. Honda, and M. C. Yang. 2013. The influence of timing in exploratory prototyping and other activities in design projects. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Vol. 55928, p. V005T06A023). American Society of Mechanical Engineers. https://doi.org/10.1115/DETC2013-12700

[14]

J. Barlett. 2018. Chile protests: state of emergency declared in Santiago as violence escalates. The Guardian. https://www.theguardian.com/world/2019/oct/19/chile-protests-state-of-emergency-declared-in-santiago-as-violence-escalates.

[15]

L. Ali, M. Asadi, D. Gašević, J. Jovanović, and M. Hatala. 2013. Factors influencing beliefs for adoption of a learning analytics tool: An empirical study. Comput. Educ., vol. 62, 130–148. https://doi.org/10.1016/j.compedu.2012.10.023

Digital Library

[16]

M. Scheffel. 2017. The Evaluation Framework for Learning Analytics. SIKS Dissertation Series No. 2017-34.

[17]

T. Broos, K. Verbert, G. Langie, C. Van Soom, and T. De Laet. 2018. Multi-institutional Positioning Test Feedback Dashboard for Aspiring Students Lessons Learnt from a Case Study in Flanders. In Proceedings of the 8th International Conference on learning Analytics and Knowledge, 51–55. https://doi.org/10.1145/3170358.3170419

[18]

O. Viberg, M. Hatakka, O. Bälter, and A. Mavroudi. 2018. The current landscape of learning analytics in higher education. Comput. Human Behav., vol. 89, no. July, 98–110. https://doi.org/10.1016/j.chb.2018.07.027

[19]

T. Munzner. 2009. A nested model for visualization design and validation. IEEE Trans. Vis. Comput. Graph., vol. 15, no. 6, 921–928. https://doi.org/10.1109/TVCG.2009.111.

Digital Library

[20]

I. H. Hsiao, J. Guerra, D. Parra, F. Bakalov, B. König-Ries, and P. Brusilovsky. 2012. Comparative social visualization for personalized E-learning, In Proc. Work. Adv. Vis. Interfaces AVI, no. May, 303–307. https://doi.org/10.1145/2254556.2254614

Cited By

Yan LEcheverria VJin YFernandez‐Nieto GZhao LLi XAlfredo RSwiecki ZGašević DMartinez‐Maldonado R(2024)Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learningBritish Journal of Educational Technology10.1111/bjet.1349855:5(1900-1925)Online publication date: 22-Jun-2024
https://doi.org/10.1111/bjet.13498
Kaliisa RJivet IPrinsloo P(2023)A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboardsInternational Journal of Educational Technology in Higher Education10.1186/s41239-023-00394-620:1Online publication date: 3-May-2023
https://doi.org/10.1186/s41239-023-00394-6
Chavan PMitra RSarkar APanwar A(2023)A contextualized assessment of reliability and validity of student-initiated momentary self-reports during lecturesEducational technology research and development10.1007/s11423-023-10304-272:2(503-539)Online publication date: 29-Nov-2023
https://doi.org/10.1007/s11423-023-10304-2
Show More Cited By

Evaluating a Learning Analytics Dashboard to Visualize Student Self-Reports of Time-on-task: A Case Study in a Latin American University
1. Applied computing

Recommendations

Self-tracking Time-On-Task: Web-Based Weekly Timesheets for Higher Education Students
Technology-Enhanced Learning for a Free, Safe, and Sustainable World
Abstract
Due to the transition to online education, higher education students require more support to self-regulate their learning and their time management. This paper presents a work-in-progress conducted to design and implement web-based weekly ...
Effects of Learning Analytics on Students' Self-Regulated Learning in Flipped Classroom

The present article is aimed at analyzing the effects of learning analytics on students' self-regulated learning in a flipped classroom. An experiment was conducted with 96 engineering students, enrolled in a subject offered in the Flipped Classroom ...
Educational technology approach toward learning analytics: relationship between student online behavior and learning performance in higher education
LAK '14: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge

The aim of this study is to suggest more meaningful components for learning analytics in order to help learners improving their learning achievement continuously through an educational technology approach. Multiple linear regression analysis is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK21: LAK21: 11th International Learning Analytics and Knowledge Conference

April 2021

645 pages

ISBN:9781450389358

DOI:10.1145/3448139

Program Chairs:
Maren Scheffel
Ruhr University Bochum, Germany
,
Nia Dowell
University of California, Irvine, USA
,
Srecko Joksimovic
University of South Australia, Australia
,
George Siemens
University of Texas, Arlington, USA & University of South Australia, Australia

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

Erasmus+

Conference

LAK21

LAK21: 11th International Learning Analytics and Knowledge Conference

April 12 - 16, 2021

CA, Irvine, USA

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
403
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)6

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yan LEcheverria VJin YFernandez‐Nieto GZhao LLi XAlfredo RSwiecki ZGašević DMartinez‐Maldonado R(2024)Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learningBritish Journal of Educational Technology10.1111/bjet.1349855:5(1900-1925)Online publication date: 22-Jun-2024
https://doi.org/10.1111/bjet.13498
Kaliisa RJivet IPrinsloo P(2023)A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboardsInternational Journal of Educational Technology in Higher Education10.1186/s41239-023-00394-620:1Online publication date: 3-May-2023
https://doi.org/10.1186/s41239-023-00394-6
Chavan PMitra RSarkar APanwar A(2023)A contextualized assessment of reliability and validity of student-initiated momentary self-reports during lecturesEducational technology research and development10.1007/s11423-023-10304-272:2(503-539)Online publication date: 29-Nov-2023
https://doi.org/10.1007/s11423-023-10304-2
Kaur AChahal K(2023)A learning analytics dashboard for data-driven recommendations on influences of non-cognitive factors in introductory programmingEducation and Information Technologies10.1007/s10639-023-12125-529:8(9221-9256)Online publication date: 7-Sep-2023
https://dl.acm.org/doi/10.1007/s10639-023-12125-5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents