More Web Proxy on the site http://driver.im/

Article

How shall we assess this?

Authors:

Kirsti Ala-Mutka,

Judy SheardAuthors Info & Claims

ITiCSE-WGR '03: Working group reports from ITiCSE on Innovation and technology in computer science education

Pages 107 - 123

https://doi.org/10.1145/960875.960539

Published: 30 June 2003 Publication History

Abstract

Increased class sizes are forcing academics to reconsider approaches to setting and marking assessments for their students. Distributed and distance learning are creating some of the biggest changes. Some educators are embracing new technologies but others are more wary of what they do not know. In order to address this issue it is first necessary to investigate the types of assessment currently in use and the perceptions that are held by academics with and without experience of the new technologies that are becoming available.In this paper we present the findings of an international survey of Computer Science academics teaching a variety of topics within the discipline. The findings are split into two sections: a snapshot of current assessment practices and an analysis of respondents' perceptions of Computer Aided Assessment (CAA). Academics' opinions about the advantages and disadvantages of CAA are split in line with level of experience of using such techniques. Those with no experience of CAA suggest that it cannot be used to test higher-order learning outcomes and that the quality of the immediate feedback is poor; these negative opinions diminish as experience is gained.

References

[1]

Ala-Mutka K, Computer-assisted Software Engineering Courses. Proceedings of IASTED International Conference Computers and Advanced Technology in Education, Cancun, 2002

[2]

Anderson W, Krathwohl D, A Taxonomy for Learning, Teaching and assessing. http://www.cours.polymtl.ca/plu6035/PDF/anderson.pdf, 2001

[3]

Arnow D, Barshay O, Online Programming Examinations using Web To Teach, Proceedings of ITiCSE'99, Krakow, 1999

Digital Library

[4]

Arnow D, Barshay O, Web To Teach: an interactive focused programming exercise system, 29th Annual IEEE Frontiers in Education Conference, 1999

[5]

Ashworth P, Bannister P, Thorne P, Guilty in whose eyes? University students' perceptions of cheating and plagiarism in academic work and assessment, Studies in Higher Education, 22, 1997

[6]

Barnett DC, Dalton JC, Why college students cheat? Journal of College Student Personnel, 22, 1981

[7]

Bedny GZ, Seglin MH, Meister D, Activity theory: history, research and application, Theoretical Issues in Ergonomics Science, 1(2), 2000

[8]

Beevers E, Paterson JS, Automatic assessment of problemsolving skills in mathematics, Active Learning in Higher Education, 4(2), 2003

[9]

Blackboard, http://www.blackboard.com/

[10]

Bloom B, Taxonomy of Educational Objectives: The Classification of Educational Goals: Handbook I, Longman, New York, 1956

[11]

Buck D, Stucki DJ, Design Early considered Harmful: Graduated Exposure to Complexity and Structure Based on Levels of Cognitive Development, SIGCSE 2000, Austin, 2000

Digital Library

[12]

Burstein J, Leacock C, Swartz R, Automated Evaluation of Essays and Short Answers, Proceedings of 5th Annual CAA Conference, Loughborough, 2001

[13]

Carneson J, Delpierre G, Masters K, Designing and Managing Multiple Choice Questions, http://www.le.ac.uk/cc/ltg/castle/resources/mcqman/mcqman01.html, 1999

[14]

Carpenter DD, Harding TS, Montgomery SM, Steneck N, P.A.C.E.S --- A study on academic integrity among engineering undergraduates (preliminary conclusions), ASEE Annual Conference & Exposition, Montreal, 2002

[15]

Chan CC, Tsui MS, Chan MYC, Applying the Structure of the Observed Learning Outcomes (SOLO) Taxonomy on Student's Learning Outcomes: an empirical study, Assessment & Evaluation in Higher Education, 27(6), 2002

[16]

Christie JR, Automated Essay Marking for both Style and Content, Proceedings of 3rd Annual CAA Conference, Loughborough, 1999

[17]

Clariana R, Wallace P, Paper-based versus computer-based assessment: key factors associated with the test mode effect, British Journal of Educational Technology, 33, 2002

[18]

Cook J, Leatherwood C, Oriogun P, Online Conferencing with Multimedia Students: Monitoring Gender Participation and Promoting Critical Debate, Proceedings of 2nd Annual LTSN-ICS Conference, London, 2001

[19]

Cox K, Clark D, The use of formative quizzes for deep learning, Computers and Education, 30, 1998

Digital Library

[20]

Culwin F, MacLeod A, Lancaster T, Source Code Plagiarism in UK HE Computing Schools, Issues, Attitudes and Tools, Southbank University, London, Commissioned by JISC, 2001

[21]

Daly C, RoboProf and an Introductory Programming Course, Proceedings of ITiCSE'99, Krakow, 1999

Digital Library

[22]

Dalziel J, Gazzard S, Next generation computer assisted assessment software: the design and implementation of WebMCQ, Proceedings of 3rd Annual CAA Conference, Loughborough 1999

[23]

Davies P, Computer Aided Assessment MUST be more than multiple-choice tests for it to be academically credible? Proceedings of 5th Annual CAA Conference, Loughborough 2001

[24]

Davis H, Carr L, Cooke E, White S, Managing Diversity: Experiences Teaching Programming Principles in Proceedings of 2nd Annual LTSN-ICS Conference, London, 2001

[25]

Delandshire G, Implicit theories, unexamined assumptions and the status quo of educational assessment, Assessment in Education, 8(2), 2001

[26]

Dick M, Sheard J, Markham S, Is it OK to cheat? Proceedings of ITiCSE'01, Canterbury, 2001

[27]

Diekhoff GM, LaBeff EE, Clark RE, Williams LE, Francis B, and Haines VJ, College cheating: ten years later, Research in Higher Education, 37, 1996

[28]

Duke-Williams E, King T, Using Computer-Aided Assessment to Test Higher Level Learning Outcomes, Proceedings of 5th Annual CAA Conference, Loughborough, 2001

[29]

Dunn R, Beaudy J, Kalvan A, Survey of research on learning styles, Education Leadership, 46, 1989

[30]

English J, Siviter P, Experience with an Automatically Assessed Course, Proceedings of ITiCSE'00, Helsinki, 2000

Digital Library

[31]

English J, Experience with a Computer-Assisted Formal Programming Examination, in Proceedings of ITiCSE'02, Aarhus, 2002

Digital Library

[32]

Farthing DW, Jones DM, McPhee D, Permutational Multiple-Choice Questions: An Objective and Efficient Alternative to Essay-Type Examination Questions, Proceedings of ITiCSE'98, Dublin, 1998

Digital Library

[33]

Farthing DW, McPhee D, Multiple choice for honours-level students? A statistical evaluation, Proceedings of 3rd Annual CAA Conference, Loughborough, 1999

[34]

Freeman M, McKenzie, SPARK, a confidential web-based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects, British Journal of Educational Technology, 33, 2002

[35]

Frosini G, Lazzerini B, Marcelloni F, Performing automatic exams, Computers & Education, 31, 1998

Digital Library

[36]

Fuller M, Assessment for real in Virtual Learning Environments --- how far can we go? Proceedings of Interface --- Virtual Learning and Higher Education, Mansfield College, Oxford 2002

[37]

Fuller U, Slater J, Tardivel G, Virtual Seminars, Real Networked Results? Proceedings of ITiCSE'98, Dublin, 1998

Digital Library

[38]

Hagan D, Sheard J, Monitoring and Evaluating a Redesigned First Year Programming Course, Proceedings of ITiCSE'97, Uppsala, 1997

Digital Library

[39]

Hargreaves DJ, Student learning and assessment are inextricably linked, European Journal of Engineering Education 22(4), 1997

[40]

Hatzimoysis A, A Philosophy Primer in Virtual Seminars http://www.prsltsn.leeds.ac.uk/philosophy/events/hatzimoysis1.html, 2002

[41]

Higgins C, Symeonidis P, Tsintifas A, The Marking System for CourseMaster, Proceedings of ITiCSE'02, Aarhus, 2002

Digital Library

[42]

Hogarth S, Virtual Seminars in Psychology and Computinghttp://ctipsy.york.ac.uk/aster/resources/case_studies/reports/yk_6/yk_06.html, 2002

[43]

Hoggarth G, Lockyer M, An Automated Student Diagram Assessment System, Proceedings of ITiCSE'98, Dublin, 1998

Digital Library

[44]

Hunt N, Hughes J, Rowe G, Formative automated computer testing (FACT), British Journal of Educational Technology, 33, 2002

[45]

IMS http://www.imsglobal.org

[46]

Jackson D, A Semi-Automated Approach to Online Assessment, Proceedings of ITiCSE'00, Helsinki, 2000

Digital Library

[47]

Jackson D, A Software System for Grading Student Computer Programs, Computers & Education, 27(3/4), 1996

Digital Library

[48]

Janson S, Gender and the Information Society: A Socially Structured Silence, in Siefert M, Gerbner G, Fisher J (Eds.), The Information Gap: How Computers and Other Communication Technologies Affect the Social Distribution of Power, Oxford: Oxford University Press, 1989

[49]

Joy M, Luck M, Effective Electronic Marking for Online Assessment, Proceedings of ITiCSE'98, Dublin, 1998

Digital Library

[50]

JPLAG, http://www.jplag.de/

[51]

Kashy E, Tsai Y, Thoenssen M, Morrissey D, CAPA, An Integrated Computer Assisted Personal Assignment System, American Journal of Physics, 61(12), 1993

[52]

Korhonen A, Malmi L, Algorithm simulation with automatic assessment, Proceedings of ITiCSE'00, Helsinki, 2000

Digital Library

[53]

Kumar A, Learning the Interaction Between Pointers and Scope in C++, ITiCSE'01, Canterbury, 2001

Digital Library

[54]

Lee S, Development of instructional strategy of computer application software for group instruction in Computers and Education, 37(1), 2001

[55]

Le Heron J, Plagiarism, learning dishonest or just plain cheating: The context and countermeasures in information systems teaching, Australian Journal of Educational Technology, 17, 2001

[56]

Lister R, Objectives and Objective Assessment in CS1, Proceedings of SIGCSE '01, Charlotte, 2001

Digital Library

[57]

Malmi L, Korhonen A, Saikkonen R, Experiences in Automatic Assessment on Mass Courses, Proceedings of ITiCSE'02, Aarhus, 2002

Digital Library

[58]

Mason DV, and Woit DM, Integrating technology into computer science education. Proceedings of 29th SIGCSE Technical Symposium on Computer Science Education, Atlanta, 1998

Digital Library

[59]

Mason O, Grove-Stephenson I, Automated free text marking with Paperless School, Proceedings of 6th Annual CAA Conference, Loughborough 2002

[60]

Medley MD, Online Finals for CS1 and CS2, Proceedings of ITiCSE'98, Dublin, 1998

Digital Library

[61]

MOSS, http://www.cs.berkley.edu/~aiken/moss.html

[62]

Mulligan B, Pilot study on the impact of frequent computerized assessment on student work rates, Proceedings of 3rd Annual CAA Conference, Loughborough, 1999

[63]

Question Mark, http://www.questionmark.com/

[64]

Ross JL, Drysdale MTB, & Schulz RA, Cognitive learning styles and academic performance in two postsecondary computer courses. Journal of Research on Computing in Education, 33(4), 2001

[65]

Saikkonen R, Malmi L, Korhonen A, Fully Automatic Assessment of Programming Exercises, Proceedings of ITiCSE'01, Canterbury, 2001

Digital Library

[66]

Shimatani H, Kitao K, Computer Aided Instruction: A bibliography, http://ilc2.doshisha.ac.jp/users/kkitao/library/biblio/caibib.htm

[67]

Simon, Summons P, Automated testing of Databases and Spreadsheets --- the Long and the Short of it, Proceedings of ACE2000, Melbourne, 2000

Digital Library

[68]

Sloman J, Use of virtual seminars in Economic Principles, http://www.economics.ltsn.ac.uk/showcase/sloman_virtual.htm, 2002

[69]

Smailes J, Experiences of using Computer Aided Assessment within a Virtual Learning Environment, http://www.business.ltsn.ac.uk/events/BEST 2002/Papers/smailes.PDF, 2002

[70]

Solomon A, http://www-staff.it.uts.edu.au/~andrews/personal communication, 2003

[71]

Thelwall M, Computer-based assessment: a versatile educational tool, Computers & Education 34(1), 2000

Digital Library

[72]

Thomas P, Price B, Petre M, Carswell L, Richards M, Experiments with Electronic Examinations over the Internet, Proceedings of 5th Annual CAA Conference, Loughborough 2001

[73]

Thomas P, Price B, Paine C, Richards M, Remote electronic examinations: student experiences, British Journal of Educational Technology, 33, 2002

[74]

Thorburn G, Rowe G, PASS: An Automated System for Program Assessment, Computers & Education, 29(4), 1997

Digital Library

[75]

Trentin G, Computerized adaptive tests and formative assessment, Journal of Educational Multimedia and Hypermedia, 6, 1997

Digital Library

[76]

Ward A, Using peer assessment assisted by ICT for programming assignments, Interactions, 5(1), 2001

[77]

WebCT, http://www.webct.com/

[78]

White S, Davis H, Creating large-scale test banks: a briefing for participative discussion of issues and agendas, Proceedings of 4th Annual CAA Conference, Loughborough 2000

[79]

Whittington D, Hunt H, Approaches to the computerized assessment of free text responses, Proceedings of 3rd Annual CAA Conference, Loughborough 1999

[80]

Wood D, Aspects of Teaching and Learning, in Light P, Sheldon S and Woodhead M (eds.) Learning to Think, Routledge, London, 1991

Cited By

Tigina MBirillo AGolubev YKeuning HVyahhi NBryksin T(2023)Analyzing the Quality of Submissions in Online Programming Courses2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)10.1109/ICSE-SEET58685.2023.00031(271-282)Online publication date: May-2023
https://doi.org/10.1109/ICSE-SEET58685.2023.00031
Jeong HJeon C(2023)Can Coding Education Go Completely Online? Time, Work, and Relationship in Online CoursesEast Asian Science, Technology and Society: An International Journal10.1080/18752160.2023.221916518:1(4-21)Online publication date: 15-Aug-2023
https://doi.org/10.1080/18752160.2023.2219165
Luukkainen RKasurinen JNikula ULenarduzzi VPéraire CSerebrenik A(2022)ASPAProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3510456.3514149(29-39)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510456.3514149
Show More Cited By

Index Terms

How shall we assess this?
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education

Recommendations

How shall we assess this?

Increased class sizes are forcing academics to reconsider approaches to setting and marking assessments for their students. Distributed and distance learning are creating some of the biggest changes. Some educators are embracing new technologies but ...
E-Assess: A Web-Based Tool for Coordinating and Managing Program Assessment (Abstract Only)
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

This poster describes the operation and implementation of the E-Assess web-based program assessment tool. The E-Assess tool provides a centralized management system for generalized program assessment and has been used to support ABET program assessment ...
Development of information literacy in primary and secondary schools in Castile and León (Spain)
TEEM'18: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality

The development of information literacy has become part of anyone's pool of key competencies. This project arises from this need, and seeks the development of information literacy in primary and secondary school pupils through the implementation of a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE-WGR '03: Working group reports from ITiCSE on Innovation and technology in computer science education

June 2003

291 pages

ISBN:9781450374538

DOI:10.1145/960875

Conference Chairs:
Vassilios Dagdilelis
University of Macedonia, Greece
,
Maya Satratzemi
University of Macedonia, Greece
,
Editor:
David Finkel
Worcester Polytechnic Institute
,
Program Chairs:
Roger Boyle
University of Leeds, UK
,
Georgios Evangelidis
University of Macedonia, Greece

ACM SIGCSE Bulletin Volume 35, Issue 4
December 2003
144 pages
ISSN:0097-8418
DOI:10.1145/960492
Issue’s Table of Contents

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ITiCSE03

Sponsor:

SIGCSE

ITiCSE03: 8th Annual Conference on Innovation and Technology in Computer Science Education

June 30 - July 2, 2003

Thessaloniki, Greece

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

77
Total Citations
View Citations
2,576
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tigina MBirillo AGolubev YKeuning HVyahhi NBryksin T(2023)Analyzing the Quality of Submissions in Online Programming Courses2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)10.1109/ICSE-SEET58685.2023.00031(271-282)Online publication date: May-2023
https://doi.org/10.1109/ICSE-SEET58685.2023.00031
Jeong HJeon C(2023)Can Coding Education Go Completely Online? Time, Work, and Relationship in Online CoursesEast Asian Science, Technology and Society: An International Journal10.1080/18752160.2023.221916518:1(4-21)Online publication date: 15-Aug-2023
https://doi.org/10.1080/18752160.2023.2219165
Luukkainen RKasurinen JNikula ULenarduzzi VPéraire CSerebrenik A(2022)ASPAProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3510456.3514149(29-39)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510456.3514149
Luukkainen RKasurinen JNikula ULenarduzzi V(2022)ASPA: A Static Analyser to Support Learning and Continuous Feedback on Programming Courses. An Empirical Validation2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)10.1109/ICSE-SEET55299.2022.9794188(29-39)Online publication date: May-2022
https://doi.org/10.1109/ICSE-SEET55299.2022.9794188
Cassel LDaniels MGoldweber MSheard J(2018)Four reflections on the history of ITiCSEACM Inroads10.1145/32763029:4(40-46)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3276302
Buyrukoglu SBatmaz FLock R(2017)A new marking technique in semi-automated assessment2017 12th International Conference on Computer Science and Education (ICCSE)10.1109/ICCSE.2017.8085551(545-550)Online publication date: Aug-2017
https://doi.org/10.1109/ICCSE.2017.8085551
Lajis AAziz N(2014)Competency assessment of short free text answers2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)10.1109/ICE2T.2014.7006216(41-45)Online publication date: Aug-2014
https://doi.org/10.1109/ICE2T.2014.7006216
Dagiene VSkupas B(2011)Semi-automatic testing of program codes in the high school student maturity examProceedings of the 12th International Conference on Computer Systems and Technologies10.1145/2023607.2023701(564-569)Online publication date: 16-Jun-2011
https://dl.acm.org/doi/10.1145/2023607.2023701
Brandle S(2010)Automated grading of student programming assignmentsJournal of Computing Sciences in Colleges10.5555/1747137.174719125:5(284-286)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.5555/1747137.1747191
Rößling GMcNally MCrescenzi PRadenski AIhantola PSánchez-Torrubia M(2010)Adapting moodle to better support CS educationProceedings of the 2010 ITiCSE working group reports10.1145/1971681.1971684(15-27)Online publication date: 28-Jun-2010
https://dl.acm.org/doi/10.1145/1971681.1971684
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents