[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3576123.3576124acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesaus-ceConference Proceedingsconference-collections
research-article

Automated Assessment: Experiences From the Trenches

Published: 30 January 2023 Publication History

Abstract

Automated assessment is commonly used across the spectrum of computing courses offered by Tertiary institutions. Such assessment is frequently intended to address the scalability of feedback that is essential for learning, and assessment for accreditation purposes. Although many reviews of automated assessment have been reported, the voices of teachers are not present. In this paper we present a variety of cases that illustrate some of the varied motivations and experiences of teaching using automated assessment.

References

[1]
Kirsti M Ala-Mutka. 2005. A Survey of Automated Assessment Approaches for Programming Assignments. Computer Science Education 15, 2 (2005), 83–102.
[2]
Elisa Baniassad, Lucas Zamprogno, Braxton Hall, and Reid Holmes. 2021. STOP THE (AUTOGRADER) INSANITY: Regression Penalties to Deter Autograder Overreliance. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE ’21). ACM, New York, NY, USA, 1062–1068. https://doi.org/10.1145/3408877.3432430
[3]
Leopold Bayerlein. 2014. Students’ feedback preferences: how do students react to timely and automatically generated assessment feedback?Assessment & Evaluation in Higher Education 39, 8 (2014), 916–931.
[4]
Janet Carter, Kirsti Ala-Mutka, Ursula Fuller, Martin Dick, John English, William Fone, and Judy Sheard. 2003. How Shall We Assess This?SIGCSE Bull. 35, 4 (June 2003), 107–123. https://doi.org/10.1145/960492.960539
[5]
Sophie H. Cormack, Laurence A. Eagle, and Mark S. Davies. 2020. A large-scale test of the relationship between procrastination and performance using learning analytics. Assessment & Evaluation in Higher Education 45, 7 (2020), 1046–1059.
[6]
Galina Deeva, Daria Bogdanova, Estefanía Serral, Monique Snoeck, and Jochen De Weerdt. 2021. A review of automated feedback systems for learners: Classification framework, challenges and opportunities. Computers & Education 162 (2021), 104094. https://doi.org/10.1016/j.compedu.2020.104094
[7]
Paul Denny, Jacqueline Whalley, and Juho Leinonen. 2021. Promoting Early Engagement with Programming Assignments Using Scheduled Automated Feedback. In Australasian Computing Education Conference(Virtual, SA, Australia) (ACE ’21). ACM, New York, NY, USA, 88–95. https://doi.org/10.1145/3441636.3442309
[8]
Christopher Douce, David Livingstone, and James Orwell. 2005. Automatic test-based assessment of programming: A review. Journal on Educational Resources in Computing (JERIC) 5, 3 (2005), 1–14.
[9]
Stephen H. Edwards, Joshua Martin, and Clfford A. Shaffer. 2015. Examining Classroom Interventions to Reduce Procrastination. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education(Vilnius, Lithuania) (ITiCSE ’15). ACM, New York, NY, USA, 254–259.
[10]
Emily Faulconer, John Griffith, and Amy Gruss. 2022. The impact of positive feedback on student outcomes and perceptions. Assessment & Evaluation in Higher Education 47, 2 (2022), 259–268. https://doi.org/10.1080/02602938.2021.1910140
[11]
Graham Gibbs and Claire Simpson. 2005. Conditions under which assessment supports students’ learning. Learning and teaching in higher education1 (2005), 3–31.
[12]
Yoshiko Goda, Masanori Yamada, Hiroshi Kato, Takeshi Matsuda, Yutaka Saito, and Hiroyuki Miyagawa. 2015. Procrastination and other learning behavioral types in e-learning and their relationship with learning outcomes. Learning and Individual Differences 37 (2015), 72–80. https://doi.org/10.1016/j.lindif.2014.11.001
[13]
John Hattie and Helen Timperley. 2007. The Power of Feedback. Review of Educational Research 77, 1 (2007), 81–112. https://doi.org/10.3102/003465430298487
[14]
Fiona Hyland and Ken Hyland. 2001. Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing 10, 3 (2001), 185–212. https://doi.org/10.1016/S1060-3743(01)00038-8
[15]
Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Seppälä. 2010. Review of Recent Systems for Automatic Assessment of Programming Assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research(Koli, Finland) (Koli Calling ’10). ACM, New York, NY, USA, 86–93. https://doi.org/10.1145/1930464.1930480
[16]
Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE) 19, 1 (2018), 1–43.
[17]
Eun Jung Kim and Kyeong Ryong Lee. 2019. Effects of an examiner’s positive and negative feedback on self-assessment of skill performance, emotional response, and self-efficacy in Korea: a quasi-experimental study. BMC medical education 19, 1 (2019), 1–7.
[18]
George Kinnear, Ian Jones, Chris Sangwin, Maryam Alarfaj, Ben Davies, Sam Fearn, Colin Foster, André Heck, Karen Henderson, Tim Hunt, Paola Iannone, Igor’ Kontorovich, Niclas Larson, Tim Lowe, John Christopher Meyer, Ann O’Shea, Peter Rowlett, Indunil Sikurajapathi, and Thomas Wong. 2022. A collaboratively-derived research agenda for e-assessment in undergraduate mathematics. Int. J. Res. Undergrad. Math. Ed. (8 2022), 31 pages. https://doi.org/10.1007/s40753-022-00189-6
[19]
Richard Lobb and Jenny Harlow. 2016. Coderunner: A Tool for Assessing Computer Programming Skills. ACM Inroads 7, 1 (feb 2016), 47–51. https://doi.org/10.1145/2810041
[20]
Andrew Luxton-Reilly, Simon, Ibrahim Albluwi, Brett A. Becker, Michail Giannakos, Amruth N. Kumar, Linda Ott, James Paterson, Michael James Scott, Judy Sheard, and Claudia Szabo. 2018. Introductory Programming: A Systematic Literature Review. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (Larnaca, Cyprus) (ITiCSE 2018 Companion). ACM, New York, NY, USA, 55–106. https://doi.org/10.1145/3293881.3295779
[21]
Sathiamoorthy Manoharan and Ulrich Speidel. 2020. Individualized Assessments using Dividni-Enhancing Learning via Assessments Unique to Every Student. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM, New York, NY, USA, 1419–1419. https://doi.org/10.1145/3328778.3372545
[22]
Emma Mulliner and Matthew Tucker. 2017. Feedback on feedback practice: perceptions of students and academics. Assessment & Evaluation in Higher Education 42, 2 (2017), 266–288. https://doi.org/10.1080/02602938.2015.1103365
[23]
David J. Nicol and Debra Macfarlane-Dick. 2006. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31, 2 (2006), 199–218. https://doi.org/10.1080/03075070600572090
[24]
Claudia Ott, Anthony Robins, and Kerry Shephard. 2016. Translating Principles of Effective Feedback for Students into the CS1 Context. ACM Trans. Comput. Educ. 16, 1, Article 1 (jan 2016), 27 pages. https://doi.org/10.1145/2737596
[25]
José Carlos Paiva, José Paulo Leal, and Álvaro Figueira. 2022. Automated Assessment in Computer Science Education: A State-of-the-Art Review. ACM Trans. Comput. Educ. 22, 3, Article 34 (jun 2022), 40 pages. https://doi.org/10.1145/3513140
[26]
Chris Sangwin. 2013. Computer aided assessment of mathematics. Oxford University Press.
[27]
Draylson M. Souza, Katia R. Felizardo, and Ellen F. Barbosa. 2016. A Systematic Literature Review of Assessment Tools for Programming Assignments. In 2016 IEEE 29th International Conference on Software Engineering Education and Training (CSEET). IEEE, 147–156. https://doi.org/10.1109/CSEET.2016.48
[28]
Jacqueline L. Whalley and Anne Philpott. 2011. A Unit Testing Approach to Building Novice Programmers’ Skills and Confidence. In Proceedings of the 13th Australasian Computing Education Conference (Perth, Australia) (ACE ’11). Australian Computer Society, Inc., Australia, 113–118.
[29]
Benedikt Wisniewski, Klaus Zierer, and John Hattie. 2020. The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research. Frontiers in Psychology 10 (2020). https://doi.org/10.3389/fpsyg.2019.03087
[30]
Shulin Yu, Feng Geng, Chunhong Liu, and Yao Zheng. 2021. What works may hurt: The negative side of feedback in second language writing. Journal of Second Language Writing 54 (2021), 100850. https://doi.org/10.1016/j.jslw.2021.100850

Cited By

View all
  • (2024)Towards the Integration of Large Language Models and Automatic Assessment Tools: Enhancing Student Support in Programming AssignmentsProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699588(1-2)Online publication date: 12-Nov-2024
  • (2024)Projects and Portfolios — An Educator's Reflections on the Summative Assessments in a Game Programming CourseSIGGRAPH Asia 2024 Educator's Forum10.1145/3680533.3697071(1-8)Online publication date: 3-Dec-2024
  • (2023)Codeflex 2.0Internet of Behaviors Implementation in Organizational Contexts10.4018/978-1-6684-9039-6.ch003(40-67)Online publication date: 1-Nov-2023

Index Terms

  1. Automated Assessment: Experiences From the Trenches

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ACE '23: Proceedings of the 25th Australasian Computing Education Conference
    January 2023
    139 pages
    ISBN:9781450399418
    DOI:10.1145/3576123
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 January 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. automated assessment
    2. computing education
    3. feedback
    4. teaching

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ACE '23
    ACE '23: Australasian Computing Education Conference
    January 30 - February 3, 2023
    VIC, Melbourne, Australia

    Acceptance Rates

    Overall Acceptance Rate 161 of 359 submissions, 45%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)89
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 19 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Towards the Integration of Large Language Models and Automatic Assessment Tools: Enhancing Student Support in Programming AssignmentsProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699588(1-2)Online publication date: 12-Nov-2024
    • (2024)Projects and Portfolios — An Educator's Reflections on the Summative Assessments in a Game Programming CourseSIGGRAPH Asia 2024 Educator's Forum10.1145/3680533.3697071(1-8)Online publication date: 3-Dec-2024
    • (2023)Codeflex 2.0Internet of Behaviors Implementation in Organizational Contexts10.4018/978-1-6684-9039-6.ch003(40-67)Online publication date: 1-Nov-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media