Relationships between reading, tracing and writing skills in introductory programming

Reviewer: Keith Harrow

Do students who complete an introductory programming course really know how to program__?__ Most computer science instructors have probably wondered about this at one time or another. The authors of this paper do not claim to answer the question, but they have taken one more step in discussing this issue, by continuing a series of investigations into a possible hierarchy of programming-related skills: filling in missing or misplaced statements in a program fragment, code tracing, describing in English what a piece of code does (either line-by-line or an overall description of the computation), and so on. The three specific questions addressed in this study are: Is skill in tracing associated with the ability to write a program__?__ Is skill in reading a program (and describing what it does) associated with the ability to write a program__?__ Is student performance on the exam consistent with a hierarchy of programming-related skills__?__ Students were given an exam at the end of an introductory programming course in Java. About half the students (38 out of 78) agreed to participate in the experiment. (The authors note that this sample seemed to reflect the overall grade distribution of the larger group.) The questions were classified into various categories, for example, basic Java constructs, sequencing of instructions, tracing (both iterative and noniterative), explaining what code does, writing new code, and handling exceptions. The student grades on these questions were analyzed in a variety of ways, starting with descriptive statistics and then proceeding to "a more sophisticated statistical analysis, which examines the relationships between the exam questions, and which provides evidence (for or against) a hierarchy of programming-related skills." A few words of disclaimer: I am writing this review as a long-time (over 35 years) instructor in a variety of languages, including PL/I, Pascal, C, and C++, not as a statistician. I know a reasonable amount about introductory programming courses, but relatively little about the statistical methods that are used in the paper. In particular, much of the detailed analysis of the data presented here is beyond me, for example: "A degree of multicollinearity is to be expected in any valid assessment instrument, but has the consequence that estimates of the relative contribution of each predictor to a criterion have an increased confidence interval, even though the overall regression remains stable." The authors found "strong support for an association between code tracing and code writing skills, particularly when the tracing involved loops." They found somewhat weaker support for an association between code reading and code writing skills. As for the hierarchy of skills, they identify elementary tracing skills and sequencing at the lowest level, followed by more complex (iterative) tracing and explaining, followed by writing. The topic of exceptions seemed to be unconnected to the other items. On the one hand, these conclusions seem obvious to anyone who has taught introductory programming courses. On the other hand, it is nice to get backing even for obvious points. Of course, there are a number of questions that can be raised about this study. (It is important to note that the authors themselves point out a number of potential problems with their analysis.) In particular, the results are based on one exam, given to one group of students, and the grading of parts of the exam may be somewhat subjective. I have noted over the course of many years that there is a variation from class to class, from term to term, and from student to student. How much time was spent in class emphasizing tracing versus other topics__?__ How much did the students practice before the exam__?__ Did the students spend more time on some of the earlier exam questions rather than the later ones__?__ Did the overall reading and writing ability of the students play a role in answering some of the questions__?__ There are many skills associated with learning how to program, and for most students there is a natural progression of these skills, from tracing to describing to coming up with a solution to a brand new problem. However, this hierarchy is not strict for all students; a small number of students can write a program, but are very poor at tracing or explaining what another program does. To sum up, this paper performs a significant service by providing evidence for a number of important points about how students learn how to program, no matter how obvious some of these points may seem. Online Computing Reviews Service