Frequent, Timed Coding Tests for Training and Assessment of Full-Stack Web Development Skills: An Experience Report

This experience report describes the use of frequent, timed coding tests in a project-intensive software engineering course in which students first learn full-stack web development using Ruby on Rails and then apply their skills in a team project. The goal of the skills tests was twofold: (1) to help motivate students to engage in distributed practice and, thus, gain adequate coding skills to be an effective team member during the team project and (2) to accurately assess whether students had acquired the requisite skills and, thereby, catch deficiencies early, while there was still time to address them. Regarding the first goal, although several students indicated that the tests motivated them to engage in substantial practice coding, it was ultimately inconclusive as to the extent of the tests' impact on students' distributed practice behavior and on their preparation for the project. Regarding the second goal, the skills testing approach was indeed considerably more effective than graded homework assignments for assessing coding skill and detecting struggling students early. Lessons learned from our experiences included that students had significant concerns about the strict time limit on the tests, that the tests caused a spike in mid-semester withdrawals from the course that disproportionately impacted students from underrepresented groups, and that detecting struggling students was one thing, but effectively helping them catch up was a whole other challenge.