More Web Proxy on the site http://driver.im/

research-article

A Method to Analyze Computer Science Students’ Teamwork in Online Collaborative Learning Environments

Authors:

Rebecca Vivian,

Katrina Falkner,

Nickolas Falkner,

Hamid TarmazdiAuthors Info & Claims

ACM Transactions on Computing Education (TOCE), Volume 16, Issue 2

Article No.: 7, Pages 1 - 28

https://doi.org/10.1145/2793507

Published: 17 February 2016 Publication History

Abstract

Although teamwork has been identified as an essential skill for Computer Science (CS) graduates, these skills are identified as lacking by industry employers, which suggests a need for more proactive measures to teach and assess teamwork. In one CS course, students worked in teams to create a wiki solution to problem-based questions. Through a case-study approach, we test a developed teamwork framework, using manual content analysis and sentiment analysis, to determine if the framework can provide insight into students’ teamwork behavior and to determine if the wiki task encouraged students to collaborate, share knowledge, and self-adopt teamwork roles. Analysis revealed the identification of both active and cohesive teams, disengaged students, and particular roles and behaviors that were lacking. Furthermore, sentiment analysis revealed that teams moved through positive and negative emotions over the course of developing their solution, toward satisfaction. The findings demonstrate the value of the detailed analysis of online teamwork. However, we propose the need for automated measures that provide real-time feedback to assist educators in the fair and efficient assessment of teamwork. We present a prototype system and recommendations, based on our analysis, for automated teamwork analysis tools.

References

[1]

W. Archer and J. Davison. 2008. Graduate employability: What do employers think and want? In Graduate Employability: The Views of Employers, R. Brown and K. Herrmann (Eds.). The Council for Industry and Higher Education, London, 1--18.

[2]

K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, and D. Thomas. 2001. Manifesto for Agile Software Development. Retrieved from http://agilemanifesto.org/.

[3]

H. Beetham and R. Sharpe. 2007. Rethinking Pedagogy for a Digital Age: Designing and Delivering e-Learning. Routledge, New York.

Digital Library

[4]

A. Begel and B. Simon. 2008a. Novice software developers, all over again. In Proceedings of the Workshop on Computing Education Research.

Digital Library

[5]

A. Begel and B. Simon. 2008b. Struggles of new college graduates in their first software development job. In Proceedings of the SIGCSE Technical Symposium on Computer Science Education.

Digital Library

[6]

L. Bender, G. Walia, K. Kambhampaty, K. Nygard, and T. Nygard. 2012. Social sensitivity correlations with the effectiveness of team process performance: An empirical study. In Proceedings of the International Computing Education Research Conference.

Digital Library

[7]

S. Bull and J. Kay. 2010. Open learner models. In Advances in Intelligent Tutoring Systems, R. Nkambou, J. Bourdeau, and R. Mizoguchi (Eds.). Springer-Verlag, Berlin, 301--322.

[8]

L.-J. ChanLin and K.-C. Chan. 2010. Group learning strategies for online course. Procedia - Social and Behavioral Sciences 22, 397--401.

[9]

Z. Chen, B. Liu, M. Hsu, M. Castellanos, and R. Ghosh. 2013. Identifying intention posts in discussion forums. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

[10]

J. Creswell. 2008. Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 3rd ed. Pearson, Upper Saddle River, NJ.

[11]

J. Creswell. 2013. Qualitative Inquiry and Research Design: Choosing Among Five Approaches 3rd ed. Sage, Thousand Oaks, CA.

[12]

S. Crowe, K. Cresswell, A. Robertson, G. Huby, A. Avery, and A. Sheikh. 2011. The case study approach. BMC Medical Research Methodology 111, 100.

[13]

S. Dawson, A. Bakharia, and E. Heathcote. 2010. SNAPP: Realising the affordances of real-time SNA within networked learning environments. In Proceedings of the 7th International Conference on Networked Learning.

[14]

A. De Liddo, S. Buckingham Shum, I. Quinto, M. Bachler, and L. Cannavacciuolo. 2011. Discourse-centric learning analytics. In Proceedings of the International Conference on Learning Analytics and Knowledge (LAK).

Digital Library

[15]

T. Dickinson and R. McIntyre. 2009. A conceptual framework for teamwork measurement. In Team Performance Assessment and Measurement: Theory, Methods, and Applications, M. Bannick, E. Salas, and C. Prince (Eds.). Lawrence Erlbaum, Mawah, NJ.

[16]

T. Dingsoyr and T. Dyba. 2012. Team effectiveness in software development: Human and cooperative aspects in team effectiveness models and priorities for future studies. In Proceedings of the International Workshop on Cooperative and Human Aspects of Software Engineering.

Digital Library

[17]

A. El-Abbassy, R. Muawad, and A. Gaber. 2010. Evaluating agile principles in CS Education. International Journal of Computer Science and Network Security 10, 10, 19--28.

[18]

K. Falkner and N. Falkner. 2012. Supporting and structuring “contributing student pedagogy” in computer science curricula. Computer Science Education 224, 413--443.

[19]

K. Falkner, N. Falkner, and R. Vivian. 2013. Collaborative learning and anxiety: A phenomenographic study of collaborative learning activities. In Proceedings of the Technical Symposium on Computer Science Education.

Digital Library

[20]

A. Forte and A. Bruckman. 2007. Constructing Text: Wiki as a Toolkit for Collaborative Learning? Paper presented at the WikiSym.

Digital Library

[21]

H. Fournier, R. Kop, and H. Sitlia. 2011. The value of learning analytics to networked learning on a Personal Learning Environment. In Proceedings of the Learning Analytics and Knowledge Conference.

Digital Library

[22]

S. Govaerts, K. Verbert, J. Klerkx, and E. Duval. 2010. Visualizing activities for self-reflection and awareness. In Proceedings of the 9th International Conference on Web-based Learning.

[23]

G. Gweon, S. Jun, J. Lee, S. Finger, and C. Penstein Rosé. 2011. A framework for assessment of student project groups on-line and off-line. In Analyzing Interactions in CSCL, S. Puntambekar, G. Erkens, and C. Hmelo-Silver (Eds.),Vol. 12. Springer, New York, 293--317.

[24]

A. Hadwin, S. Järvelä, and M. Miller. 2011. Self-regulated, co-regulated, and socially shared regulation. In Handbook of Self-Regulation of Learning and Performance, B. Zimmerman and D. Schunk (Eds.). Taylor & Francis, New York, 65--63.

[25]

T. Heafner and A. Friedman. 2008. Wikis and constructivism in secondary social studies: Fostering a deeper understanding. Computers in the Schools 253--254, 288--302.

[26]

C. Hmelo-Silver. 2003. Analyzing collaborative knowledge construction: Multiple methods for integrated understanding. Computers & Education 414, 397--420.

Digital Library

[27]

C. Hmelo-Silver and E. Chemobilsky. 2004. Understanding collaborative activity systems: The relation of tools and discourse in mediating learning. In Proceedings of the 6th International Conference on Learning Sciences.

Digital Library

[28]

C. Hmelo-Silver, E. Chernobilsky, and R. Jordan. 2008. Understanding collaborative learning processes in new learning environments. Instructional Science 365, 409--430.

[29]

C. Hmelo-Silver and C. Eberbach. 2012. Learning Theories and Problem-Based Learning: Problem-Based Learning in Clinical Education. In S. Bridges, C. McGrath, and T. Whitehill (Eds.), Vol. 8. Springer, 3--17.

[30]

H.-F. Hsieh and S. Shannon. 2005. Three approaches to qualitative content analysis. Qualitative Health Research 159, 1277--1288.

[31]

R. Hughes and S. Jones. 2011. Developing and assessing college student teamwork skills. New Directions for Institutional Research 149, 53--64.

[32]

K. Krippendorff. 2004. Content Analysis: An Introduction to its Methodology 2nd ed. Sage, Thousand Oaks, CA.

[33]

N. Law, J. Yuen, W. Wong, and J. Leng. 2011. Understanding learners’ knowledge building trajectory through visualizations of multiple automated analyses. In Analyzing Interactions in CSCL, S. Puntambekar, G. Erkens, and C. Hmelo-Silver (Eds.),Vol. 12. New York: Springer, New York, 47--82.

[34]

R. Lingard and S. Barkataki. 2011. Teaching teamwork in engineering and computer science. In Proceedings of the Frontiers in Education Conference.

Digital Library

[35]

C. Loftus, L. Thomas, and C. Zander. 2011. Can graduating students design: Revisited. In Proceedings of the Technical Symposium on Computer Science Education.

Digital Library

[36]

R. Mazza and V. Dimitrova. 2004. Visualising student tracking data to support instructors. In Proceedings of the 13th International World Wide Web Conference.

Digital Library

[37]

S. Minocha and P. Thomas. 2007. Collaborative learning in a wiki environment: experiences from a software engineering course. New Review of Hypermedia and Multimedia 132, 187--209.

Digital Library

[38]

M. Munezero, C. Calkin Suero Montero, M. Mozgovoy, and E. Sutinen. 2013. Exploiting sentiment analysis to track emotions in students’ learning diaries. In Proceedings of the 13th Koli Calling International Conference on Computing Education Research.

Digital Library

[39]

B. Oakley, D. Hanna, Z. Kuzmyn, and R. Felder. 2007. Best practices involving teamwork in the classroom: Results from a survey of 6435 Engineering student respondents. IEEE Transactions on Education 503, 266--272.

Digital Library

[40]

B. Pang and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval 21--22, 1--135.

Digital Library

[41]

L. Polanya and A. Zaenen. 2004. Contextual valence shifters Working Notes -- Exploring Attitude and Affect in Text. AAAI Spring Symposium Series.

[42]

H. Prakken, C. Reed, and D. Walton. 2003. Argumentation schemes and generalisations in reasoning about evidence. In Proceedings of the 9th International Conference on Artificial Intelligence and Law (ICAL).

Digital Library

[43]

A. Radermacher and G. Walia. 2013. Gaps between industry expectations and the abilities of graduates. In Proceedings of the Technical Symposium on Computer Science Education.

Digital Library

[44]

M. Robles. 2012. Executive perceptions of the top 10 soft skills needed in today's workplace. Business Communication Quarterly.

[45]

S. Ruff and M. Carter. 2009. Communication learning outcomes from software engineering professionals: A basis for teaching communication in the engineering curriculum. In Proceedings of the Frontiers in Education Conference.

Digital Library

[46]

M. Sandelowski. 1995. Sample size in qualitative research. Research in Nursing & Health 182, 179--183.

[47]

J. Santos, K. Verbert, S. Govaerts, and E. Duval. 2011. Visualizing PLE usage. In Proceedings of the 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments.

[48]

H. Schmidt, J. Rotgans, and E. Yew. 2011. The process of problem-based learning: What works and why. Medical Education 458, 792--806.

[49]

H. Schmitz, M. Scheffel, M. Friedrich, M. Jahn, K. Niemann, and M. Wolpers. 2009. Learning in the synergy of multiple disciplines. In Lecture Notes in Computer Science, U. Cress, V. Dimitrova, and M. Specht (Eds.). Springer. 507--520

[50]

D. Schunk. 2008. Learning Theories: An Educational Perspective 5th ed. Pearson, Upper Saddle River, NJ.

[51]

J.-W. Strijbos, M. De Laat, R. Martens, and W. Jochems. 2005. Functional versus spontaneous roles during CSCL. In Proceedings of the Computer Support for Collaborative Learning Conference.

Digital Library

[52]

G. Trentin. 2009. Using a wiki to evaluate individual contribution to a collaborative learning project. Journal of Computer Assisted Learning 251, 43--55.

[53]

R. Vatrapu, C. Teplovs, N. Fujita, and S. Bull. 2011. Towards visual analytics for teachers’ dynamic diagnostic pedagogical decision-making. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK).

Digital Library

[54]

R. Vivian, K. Falkner, and K. Falkner. 2013a. Analysing computer science students’ teamwork role adoption in an online self-organised teamwork activity. In Proceedings of the Koli Calling Conference.

Digital Library

[55]

R. Vivian, K. Falkner, and N. Falkner. 2013b. Building consensus: students’ cognitive and metacognitive behaviours during wiki construction. In Proceedings of the Learning and Teaching in Computer Engineering (LaTiCE) Conference.

Digital Library

[56]

R. Weber. 1990. Basic Content Analysis. Sage, Newbury Park, CA.

[57]

A. Weinberger and F. Fischer. 2006. A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education 46, 1, 71--95.

Digital Library

[58]

M. Wells and R. Clougherty. 2008. Use of wikis in chemistry instruction for problem-based learning assignments: an example in instrumental analysis. Journal of Chemical Education 85, 10, 1446.

[59]

R. Wilson Lundin. 2008. Teaching with wikis: Toward a networked pedagogy. Computers and Composition 254, 432--448.

[60]

T. Wilson, J. Wiebe, and P. Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Human Language Technology and Empirical Methods in Natural Language Processing Conference.

Digital Library

[61]

T. Yasseri, R. Sumi, A. Rung, A. Kornai, and J. Kertész. 2012. Dynamics of conflicts in wikipedia. PLoS ONE 76, e38869.

[62]

Z. Zanial. 2007. Case study as a research method. Jurnal Kemanusiaan 9, 1--6.

[63]

B. Zimmerman. 1986. Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology 114, 307--313.

[64]

B. Zimmerman and M. Pons. 1986. Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal 234, 614--628.

Cited By

Vossen PAjit S(2024)Measurement Models for Group-Peer Assessment of Project-Based Learning in Software Engineering2024 International Symposium on Computer Science and Educational Technology (ISCSET)10.1109/ISCSET58624.2024.10808114(1-5)Online publication date: 17-Jul-2024
https://doi.org/10.1109/ISCSET58624.2024.10808114
Hu WChan C(2024)Evaluating technological interventions for developing teamwork competency in higher education: A systematic review and meta-ethnographyStudies in Educational Evaluation10.1016/j.stueduc.2024.10138283(101382)Online publication date: Dec-2024
https://doi.org/10.1016/j.stueduc.2024.101382
Graßl IKrusche SFraser G(2023)Diversity and Teamwork in Student Software TeamsProceedings of the 5th European Conference on Software Engineering Education10.1145/3593663.3593687(110-119)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3593663.3593687
Show More Cited By

Index Terms

A Method to Analyze Computer Science Students’ Teamwork in Online Collaborative Learning Environments
1. Applied computing
  1. Education
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        Information science education

Recommendations

Analysing computer science students' teamwork role adoption in an online self-organised teamwork activity
Koli Calling '13: Proceedings of the 13th Koli Calling International Conference on Computing Education Research

Computer Science (CS) professionals are regularly required to work in teams to solve complex problems. Agile software development processes are increasingly popular in organisations as a method for teamwork but the self-organising nature of the method ...
Relating Collaborative Technology Use to Teamwork Quality and Performance: An Empirical Analysis

Although team-based work systems are pervasive in the workplace, the use of collaborative systems designed to facilitate and support ongoing teamwork is a relatively recent development. An understanding of how teams embrace and use such collaborative ...
Gender Effects on Collaborative Online Brainstorming Teamwork
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

It is common for individuals with diverse demographic backgrounds to collaborate through computer-mediated communication (CMC) technologies. Groups with internal diversity are typically considered to be advantageous to group performance due to the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing Education

ACM Transactions on Computing Education Volume 16, Issue 2

March 2016

121 pages

EISSN:1946-6226

DOI:10.1145/2894200

Editor:
Chris Hundhausen
Washington State University, USA

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 February 2016

Accepted: 01 April 2015

Revised: 01 December 2014

Received: 01 March 2014

Published in TOCE Volume 16, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
1,319
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)22

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vossen PAjit S(2024)Measurement Models for Group-Peer Assessment of Project-Based Learning in Software Engineering2024 International Symposium on Computer Science and Educational Technology (ISCSET)10.1109/ISCSET58624.2024.10808114(1-5)Online publication date: 17-Jul-2024
https://doi.org/10.1109/ISCSET58624.2024.10808114
Hu WChan C(2024)Evaluating technological interventions for developing teamwork competency in higher education: A systematic review and meta-ethnographyStudies in Educational Evaluation10.1016/j.stueduc.2024.10138283(101382)Online publication date: Dec-2024
https://doi.org/10.1016/j.stueduc.2024.101382
Graßl IKrusche SFraser G(2023)Diversity and Teamwork in Student Software TeamsProceedings of the 5th European Conference on Software Engineering Education10.1145/3593663.3593687(110-119)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3593663.3593687
Sim YLua ZKelaver KChua JLim ICao QKeoh SLim L(2023)Experiences and Lessons Learned from Real-World Projects in Software Engineering Subject2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET58097.2023.00036(173-177)Online publication date: Aug-2023
https://doi.org/10.1109/CSEET58097.2023.00036
Jhan YLuarn PLin H(2023)A study on EMBA students’ cognitive and decision-making models in learning through playThe International Journal of Management Education10.1016/j.ijme.2023.10076521:1(100765)Online publication date: Mar-2023
https://doi.org/10.1016/j.ijme.2023.100765
Chen WTan JZhang SPi ZLyu Q(2023)AppleTree system for effective computer-supported collaborative argumentation: an exploratory studyEducational technology research and development10.1007/s11423-023-10258-571:5(2103-2136)Online publication date: 5-Jul-2023
https://doi.org/10.1007/s11423-023-10258-5
Bóbó MCampos FStroele VDavid JBraga RTorrent T(2022)Using Sentiment Analysis to Identify Student Emotional State to Avoid Dropout in E-LearningInternational Journal of Distance Education Technologies10.4018/IJDET.30523720:1(1-24)Online publication date: 24-Jun-2022
https://dl.acm.org/doi/10.4018/IJDET.305237
Sukackė VGuerra AEllinger DCarlos VPetronienė SGaižiūnienė LBlanch SMarbà-Tallada ABrose A(2022)Towards Active Evidence-Based Learning in Engineering Education: A Systematic Literature Review of PBL, PjBL, and CBLSustainability10.3390/su14211395514:21(13955)Online publication date: 27-Oct-2022
https://doi.org/10.3390/su142113955
Barreto DOyarzun BConklin S(2022)Integration of cooperative learning strategies in online settingsE-Learning and Digital Media10.1177/2042753022110418719:6(574-594)Online publication date: 27-May-2022
https://doi.org/10.1177/20427530221104187
Mitchell AScott MWalton-Rivers JWatkins MNew WBrown D(2022)An Exploratory Analysis of Student Experiences with Peer Evaluation in Group Game Development ProjectsProceedings of the 2022 Conference on United Kingdom & Ireland Computing Education Research10.1145/3555009.3555021(1-7)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1145/3555009.3555021
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents