Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback
Authors: 
Ignacio Villagrán, Rocío Hernández, Gregory Schuit, Andrés Neyem, Javiera Fuentes-Cimma, Constanza Miranda, Isabel Hilliger, Valentina Durán, Gabriel Escalona, Julián VarasAuthors Info & Claims
Pages 2079 - 2090
Abstract
This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.
References
[1]
D. Cecilio-Fernandes, R. Medina-Ramírez, J. Sandars, and M. Costa, “Self-regulated learning processes across different physiotherapy clinical procedural skills and time intervals: A SRL microanalysis study,” Med. Teacher, vol. 45, no. 10, pp. 1170–1176, 2023.
[2]
Y. Tsai, “Why feedback literacy matters for learning analytics,” in Proc. 16th Int. Conf. Learn. Sci., 2022, pp. 27–34.
[3]
R. Nash and N. Winstone, “Responsibility-sharing in the giving and receiving of assessment feedback,” Front. Psychol., vol. 8, 2017, Art. no.
[4]
J. Wood, “A dialogic technology-mediated model of feedback uptake and literacy,” Assessment Eval. Higher Educ., vol. 46, no. 8, pp. 1173–1190, 2021.
[5]
P. Dawson et al., “Technology and feedback design,” in Learning, Design, and Technology, M. Spector, B. Lockee, and M. Childress, Eds., Cham, Switzerland: Springer, 2018.
[6]
I. Villagrán et al., “Remote, asynchronous training and feedback enables development of neurodynamic skills in physiotherapy students,” BMC Med. Educ., vol. 23, no. 1, 2023, Art. no.
[7]
F. Belmar et al., “Artificial intelligence in laparoscopic simulation: A promising future for large-scale automated evaluations,” Surg. Endoscopy, vol. 37, no. 6, pp. 4942–4946, 2023.
[8]
D. Carless and N. Winstone, “Teacher feedback literacy and its interplay with student feedback literacy,” Teach. Higher Educ., vol. 28, no. 1, pp. 150–163, 2023.
[9]
J. Nieminen and D. Carless, “Feedback literacy: A critical review of an emerging concept,” Higher Educ., vol. 85, no. 6, pp. 1381–1400, 2023.
[10]
N. B. Ødegaard, H. T. Myrhaug, T. Dahl-Michelsen, and Y. Røe, “Digital learning designs in physiotherapy education: A systematic review and meta-analysis,” BMC Med. Educ., vol. 21, pp. 1–18, 2021.
[11]
C. MacDonald, E. Lonnemann, S. Petersen, D. A. Rivett, P. Osmotherly, and J. Brismée, “COVID 19 and manual therapy: International lessons and perspectives on current and future clinical practice and education,” J. Manual Manipulative Ther., vol. 28, no. 3, pp. 134–145, 2020.
[12]
L. Ng et al., “eLearning in physical therapy: Lessons learned from transitioning a professional education program to full eLearning during the COVID-19 pandemic,” Phys. Ther., vol. 101, no. 4, 2021, Art. no.
[13]
K. M. Hurst, “Using video podcasting to enhance the learning of clinical skills: A qualitative study of physiotherapy students' experiences,” Nurse Educ. Today, vol. 45, pp. 206–211, 2016.
[14]
V. D. Espinoza et al., “Available assessment tools for evaluating feedback quality: A scoping review oriented to education in digital media,” Glob. Surg. Educ., vol. 3, 2024, Art. no.
[15]
A. J. van Duijn, K. Swanick, and E. K. Donald, “Student learning of cervical psychomotor skills via online video instruction versus traditional face-to-face instruction,” J. Phys. Ther. Educ., vol. 28, no. 1, pp. 94–102, 2014.
[16]
M. I. Gaete et al., “Remote and asynchronous training network: From a sages grant to an eight-country remote laparoscopic simulation training program,” Surg. Endoscopy, vol. 37, no. 2, pp. 1458–1465, 2022.
[17]
M. E. Dogan, T. G. Dogan, and A. Bozkurt, “The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies,” Appl. Sci., vol. 13, no. 5, 2023, Art. no.
[18]
A. Nguyen, H. N. Ngo, Y. Hong, B. Dang, and B.-P. T. Nguyen, “Ethical principles for artificial intelligence in education,” Educ. Inf. Technol., vol. 28, no. 4, pp. 4221–4241, 2023.
[19]
M. G. Tolsgaard et al., “The fundamentals of artificial intelligence in medical education research: AMEE guide no. 156,” Med. Teacher, vol. 45, no. 6, pp. 565–573, 2023.
[20]
H. Lee, “The rise of ChatGPT: Exploring its potential in medical education,” Anat. Sci. Educ., vol. 17, pp. 926–931, 2024.
[21]
M. T. Duong et al., “Artificial intelligence for precision education in radiology,” Brit. J. Radiol., vol. 92, no. 1103, 2019, Art. no.
[22]
E. Kasneci et al., “ChatGPT for good? On opportunities and challenges of large language models for education,” Learn. Individual Differences, vol. 103, 2023, Art. no.
[23]
F. F. Moghaddam, M. Ahmadi, S. Sarvari, M. Eslami, and A. Golkar, “Cloud computing challenges and opportunities: A survey,” in Proc. 1st Int. Conf. Telematics Future Gener. Netw., 2015, pp. 34–38.
[24]
F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.
[25]
J. Wei and K. Zou, “EDA: Easy data augmentation techniques for boosting performance on text classification tasks,” in Proc. Conf. Empirical Methods Natural Lang. Process. 9th Int. Joint Conf. Natural Lang. Process., Assoc. Comput. Linguistics, 2019, pp. 6381–6387.
[26]
C. Lewis, C. Wharton, and M. Helander, “Cognitive walkthroughs,” in Handbook of Human-Computer Interaction. Amsterdam, The Netherlands: Elsevier, 1997.
[27]
A. Neyem, L. A. Gonzalez, M. Mendoza, J. P. S. Alcocer, L. Centellas-Claros, and C. Paredes-Robles, “Towards an AI knowledge assistant for context-aware learning experiences in software capstone project development,” IEEE Trans. Learn. Technol., vol. 17, pp. 1639–1654, 2024.
[28]
J. Kincaid, R. Fishburne Jr, R. Rogers, and B. Chissom, “Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel,” Inst. Simul. Training, Univ. Central Florida, Orlando, FL, USA, Res. Rep. 8-75, 1975.
[29]
R. Gunning, The Technique of Clear Writing. New York, NY, USA: McGraw-Hill, 1952.
[30]
E. Dale and J. S. Chall, “A formula for predicting readability: Instructions,” Educ. Res. Bull., vol. 27, pp. 37–54, 1948.
[31]
G. R. Klare, “Assessing readability,” Reading Res. Quart., vol. 10, no. 1, pp. 62–102, 1974.
[32]
S. Nicoll, K. Douglas, and B. Christopher, “Giving feedback on feedback: An assessment of grader feedback construction on student performance,” in Proc. 12th Int. Learn. Analytics Knowl. Conf., 2022, pp. 239–249.
[33]
A. Cavalcanti, R. de Mello, V. Rolim, M. André, F. Freitas, and D. Gaševic, “An analysis of the use of good feedback practices in online learning courses,” in Proc. IEEE 19th Int. Conf. Adv. Learn. Technol., 2019, pp. 153–157.
[34]
Q. Solano et al., “Cognitive walkthroughs. handbook of human-computer interaction,” J. Surg. Educ., vol. 78, no. 6, pp. e72–e77, 2021.
Index Terms
- Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback
Index terms have been assigned to the content through auto-classification.
Recommendations
Artificial Intelligence in English Language Education: The Path to Personalized Learning
EKI '24: Proceedings of the 2nd International Conference on Educational Knowledge and InformatizationWith the continuous development of artificial intelligence technology, its application in the field of education is becoming increasingly widespread. This article starts from the perspective of "the path of artificial intelligence personalized learning ...
Artificial Intelligence Literacy to Enhance Teacher Critical Thinking
Innovative Technologies and LearningAbstractNowadays the teacher must be open to the transformation of the socio-educational context in which he/she works and, consequently, must develop new skills, including those linked to the use of Information and Communication Technologies equipped ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1939-1382 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Computer Society Press
Washington, DC, United States
Publication History
Published: 26 August 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Jan 2025