More Web Proxy on the site http://driver.im/

research-article

The Success and Failure of Quality Improvement Projects in Peer Production Communities

Authors:

Morten Warncke-Wang,

Vladislav R. Ayukaev,

Loren G. TerveenAuthors Info & Claims

CSCW '15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing

Pages 743 - 756

https://doi.org/10.1145/2675133.2675241

Published: 28 February 2015 Publication History

Abstract

Peer production communities have been proven to be successful at creating valuable artefacts, with Wikipedia as a prime example. However, a number of studies have shown that work in these communities tends to be of uneven quality and certain content areas receive more attention than others. In this paper, we examine the efficacy of a range of targeted strategies to increase the quality of under-attended content areas in peer production communities. Mining data from five quality improvement projects in the English Wikipedia, the largest peer production community in the world, we show that certain types of strategies (e.g. creating artefacts from scratch) have better quality outcomes than others (e.g. improving existing artefacts), even if both are done by a similar cohort of participants. We discuss the implications of our findings for Wikipedia as well as other peer production communities.

References

[1]

Anderson, A., Huttenlocher, D., Kleinberg, J., and Leskovec, J. Steering user behavior with badges. In Proc. of WWW (2013), 95--106.

Digital Library

[2]

André, P., Kraut, R. E., and Kittur, A. Effects of Simultaneous and Sequential Work Structures on Distributed Collaborative Interdependent Tasks. In Proc. of CHI (2014), 139--148.

Digital Library

[3]

Ardichvili, A. Learning and Knowledge Sharing in Virtual Communities of Practice: Motivators, Barriers, and Enablers. Advances in Developing Human Resources 10, 4 (2008), 541--554.

[4]

Brooks, F. P. The Mythical Man-Month, vol. 1995. Addison-Wesley Reading, 1975.

Digital Library

[5]

Burke, M., and Kraut, R. Mopping Up: Modeling Wikipedia Promotion Decisions. In Proc. of CSCW (2008), 27--36.

Digital Library

[6]

Chen, J., Ren, Y., and Riedl, J. The Effects of Diversity on Group Productivity and Member Withdrawal in Online Volunteer Groups. In Proc. of CHI (2010), 821--830.

Digital Library

[7]

Cosley, D., Frankowski, D., Terveen, L., and Riedl, J. Using Intelligent Task Routing and Contribution Review to Help Communities Build Artifacts of Lasting Value. In Proc. of CHI, CHI '06 (2006), 1037--1046.

Digital Library

[8]

Cosley, D., Frankowski, D., Terveen, L., and Riedl, J. SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia. In Proc. IUI (2007), 32--41.

Digital Library

[9]

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. From game design elements to gamefulness: Defining "gamification". In Proc. of Mindtrek (2011), 9--15.

Digital Library

[10]

Farzan, R., and Kraut, R. E. Wikipedia classroom experiment: Bidirectional benefits of students' engagement in online production communities. In Proc. of CHI (2013).

Digital Library

[11]

Forte, A., Kittur, N., Larco, V., Zhu, H., Bruckman, A., and Kraut, R. E. Coordination and Beyond: Social Functions of Groups in Open Content Production. In Proc. of CSCW (2012), 417--426.

Digital Library

[12]

Garfield, S. 10 reasons why people don't share their knowledge. KM Review 9, 2 (2006), 10--11.

[13]

Goodchild, M. Citizens as sensors: the world of volunteered geography. GeoJournal 69, 4 (2007), 211--221.

[14]

Haklay, M. How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets. Environment and planning. B, Planning & design 37, 4 (2010), 682.

[15]

Halfaker, A., Keyes, O., and Taraborelli, D. Making Peripheral Participation Legitimate: Reader engagement experiments in Wikipedia. In Proc. of CSCW (2013), 849--860.

Digital Library

[16]

Halfaker, A., Kittur, A., Kraut, R., and Riedl, J. A jury of your peers: quality, experience and ownership in Wikipedia. In Proc. WikiSym (2009), 15:1--15:10.

Digital Library

[17]

Japkowicz, N., and Stephen, S. The class imbalance problem: A systematic study. Intelligent data analysis 6, 5 (2002), 429--449.

Digital Library

[18]

Karau, S. J., and Williams, K. D. Social loafing: A meta-analytic review and theoretical integration. Journal of personality and social psychology 65, 4 (1993), 681.

[19]

Keegan, B., Gergle, D., and Contractor, N. Hot off the wiki: Structures and dynamics of wikipedias coverage of breaking news events. American Behavioral Scientist 57, 5 (2013), 595--622.

[20]

Kittur, A., and Kraut, R. E. Harnessing the wisdom of crowds in Wikipedia: quality through coordination. In Proc. CSCW (2008).

Digital Library

[21]

Kittur, A., Pendleton, B., and Kraut, R. E. Herding the Cats: The Influence of Groups in Coordinating Peer Production. In Proc. of WikiSym (2009), 7:1--7:9.

Digital Library

[22]

Kriplean, T., Beschastnikh, I., McDonald, D. W., and Golder, S. A. Community, Consensus, Coercion, Control: Cs*W or How Policy Mediates Mass Participation. In Proc. of GROUP (2007), 167--176.

Digital Library

[23]

Lam, S. T. K., Uduwage, A., Dong, Z., Sen, S., Musicant, D. R., Terveen, L., and Riedl, J. WP:Clubhouse?: An Exploration of Wikipedia's Gender Imbalance. In Proc. of WikiSym (2011), 1--10.

Digital Library

[24]

Lampe, C., Obar, J., Ozkaya, E., Zube, P., and Velasquez, A. Classroom wikipedia participation effects on future intentions to contribute. In Proc. of CSCW (2012), 403--406.

Digital Library

[25]

Lave, J., and Wenger, E. Situated learning: Legitimate peripheral participation. Cambridge university press, 1991.

[26]

Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., Cosley, D., Frankowski, D., Terveen, L., Rashid, A. M., et al. Using social psychology to motivate contributions to online communities. Journal of Computer-Mediated Communication 10, 4 (2005), 00--00.

[27]

Liu, I., and Agresti, A. The analysis of ordered categorical data: An overview and a survey of recent developments. Test 14, 1 (2005), 1--73.

[28]

Masli, M., and Terveen, L. G. Leveraging the Contributory Potential of User Feedback. In Proc. of CSCW, CSCW '14, ACM (New York, NY, USA, 2014), 956--966.

Digital Library

[29]

Panciera, K., Halfaker, A., and Terveen, L. Wikipedians Are Born, Not Made: A Study of Power Editors on Wikipedia. In Proc. GROUP (2009), 51--60.

Digital Library

[30]

Preece, J. Online communities: Designing Usability and Supporting Socialbilty. John Wiley & Sons, Inc., 2000.

Digital Library

[31]

Priedhorsky, R., Masli, M., and Terveen, L. Eliciting and Focusing Geographic Volunteer Work. In Proc. of CSCW (2010), 61--70.

Digital Library

[32]

Rashid, A. M., Ling, K., Tassone, R. D., Resnick, P., Kraut, R., and Riedl, J. Motivating Participation by Displaying the Value of Contribution. In Proc. of CHI, CHI '06 (2006), 955--958.

Digital Library

[33]

Reagle, J., and Rhue, L. Gender Bias in Wikipedia and Britannica. International Journal of Communication 5, 0 (2011).

[34]

Roth, A. Student Contributions to Wikipedia. https://outreach.wikimedia.org/wiki/Student_ Contributions_to_Wikipedia. Retrieved June 2, 2014.

[35]

Simon, H. A. Rational choice and the structure of the environment. Psychological review 63, 2 (1956), 129.

[36]

Solomon, J., and Wash, R. Bootstrapping wikis: developing critical mass in a fledgling community by seeding content. In Proc. CSCW (2012), 261--264.

Digital Library

[37]

Stephens, M. Gender and the GeoWeb: divisions in the production of user-generated cartographic information. GeoJournal (2013), 1--16. 00001.

[38]

Stvilia, B., Twidale, M. B., Smith, L. C., and Gasser, L. Assessing information quality of a community-based encyclopedia. In Proc. ICIQ (2005), 442--454.

[39]

Stvilia, B., Twidale, M. B., Smith, L. C., and Gasser, L. Information quality work organization in Wikipedia. J. Am. Soc. Inf. Sci. Technol. 59 (April 2008), 983--1001.

Digital Library

[40]

Thom-Santelli, J., Cosley, D., and Gay, G. What Do You Know?: Experts, Novices and Territoriality in Collaborative Systems. In Proc. of CHI (2010), 1685--1694.

Digital Library

[41]

Thom-Santelli, J., Cosley, D. R., and Gay, G. What's Mine is Mine: Territoriality in Collaborative Authoring. In Proc. of CHI (2009), 1481--1484.

Digital Library

[42]

Warncke-Wang, M., Cosley, D., and Riedl, J. Tell me more: An actionable quality model for wikipedia. In Proc. of WikiSym (2013), 8:1--8:10.

Digital Library

[43]

WMF. Spring 2012 USA and Canada student article quality research results. https://en.wikipedia.org/wiki/Wikipedia: Ambassadors/Research/Article_quality/Results. Retrieved June 2, 2014.

[44]

Yee, T. W. The vgam package for categorical data analysis. Journal of Statistical Software 32, 10 (2010), 1--34.

[45]

Zhu, H., Kraut, R., and Kittur, A. Organizing without formal organization: Group identification, goal setting and social modeling in directing online production. In Proc. of CSCW (2012), 935--944.

Digital Library

Cited By

Palea DGuo ANair AAnderson RCharagulla NSu NLee D(2024)Forming Shared Interest Pods: Barriers to Self-Assembly of Interest-Based Small Groups, and Dynamics of Retaining and Giving up Control to Find Collective FitProceedings of the ACM on Human-Computer Interaction10.1145/36869848:CSCW2(1-50)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686984
Fu H(2024)Navigating the inception stage in online peer production communities: a comparative study on community building activities, user roles and interaction dynamicsOnline Information Review10.1108/OIR-07-2023-0360Online publication date: 11-Jul-2024
https://doi.org/10.1108/OIR-07-2023-0360
Dolphin GDolphin G(2024)Transformative Teaching StrategiesTeaching Geology Using the History and Philosophy of Science10.1007/978-3-031-68667-2_4(103-145)Online publication date: 2-Oct-2024
https://doi.org/10.1007/978-3-031-68667-2_4
Show More Cited By

Index Terms

The Success and Failure of Quality Improvement Projects in Peer Production Communities
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work
2. Social and professional topics
  1. Professional topics
    1. Computing and business
      1. Computer supported cooperative work

Recommendations

Increasing Participation in Peer Production Communities with the Newcomer Homepage
CSCW

For peer production communities to be sustainable, they must attract and retain new contributors. Studies have identified social and technical barriers to entry and discovered some potential solutions, but these solutions have typically focused on a ...
Predicting Member Productivity and Withdrawal from Pre-Joining Attachments in Online Production Groups
CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

Productive and dedicated members are critical to the success of online production communities like Wikipedia. Many communities organize in subgroups where members voluntarily work on projects of shared interest. In this paper, we investigate how members'...
Quality Hackathon: Evaluating the Products of Online Co-Production Systems
GROUP '14: Proceedings of the 2014 ACM International Conference on Supporting Group Work

This full-day workshop focuses on building Big Social Data research competencies for scholars interested in issues of contribution quality and contributor performance in online co-production systems that generate value through contributions by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing

February 2015

1956 pages

ISBN:9781450329224

DOI:10.1145/2675133

General Chairs:
Dan Cosley
Cornell University, USA
,
Andrea Forte
Drexel University, USA
,
Program Chairs:
Luigina Ciolfi
Sheffield Hallam University, UK
,
David McDonald
University of Washington, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 February 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

CSCW '15

Sponsor:

SIGCHI

CSCW '15: Computer Supported Cooperative Work and Social Computing

March 14 - 18, 2015

BC, Vancouver, Canada

Acceptance Rates

CSCW '15 Paper Acceptance Rate 161 of 575 submissions, 28%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
484
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)5

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Palea DGuo ANair AAnderson RCharagulla NSu NLee D(2024)Forming Shared Interest Pods: Barriers to Self-Assembly of Interest-Based Small Groups, and Dynamics of Retaining and Giving up Control to Find Collective FitProceedings of the ACM on Human-Computer Interaction10.1145/36869848:CSCW2(1-50)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686984
Fu H(2024)Navigating the inception stage in online peer production communities: a comparative study on community building activities, user roles and interaction dynamicsOnline Information Review10.1108/OIR-07-2023-0360Online publication date: 11-Jul-2024
https://doi.org/10.1108/OIR-07-2023-0360
Dolphin GDolphin G(2024)Transformative Teaching StrategiesTeaching Geology Using the History and Philosophy of Science10.1007/978-3-031-68667-2_4(103-145)Online publication date: 2-Oct-2024
https://doi.org/10.1007/978-3-031-68667-2_4
Ren YZhang HKraut R(2023)How Did They Build the Free Encyclopedia? A Literature Review of Collaboration and Coordination among Wikipedia EditorsACM Transactions on Computer-Human Interaction10.1145/361736931:1(1-48)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1145/3617369
Das PGuda BSeelaboyina SSarkar SMukherjee A(2022)Quality Change: Norm or Exception? Measurement, Analysis and Detection of Quality Change in WikipediaProceedings of the ACM on Human-Computer Interaction10.1145/35129596:CSCW1(1-36)Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1145/3512959
Chandrasekharan EJhaver SBruckman AGilbert E(2022)Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on RedditACM Transactions on Computer-Human Interaction10.1145/349049929:4(1-26)Online publication date: 31-Mar-2022
https://dl.acm.org/doi/10.1145/3490499
Sugandhika CAhangama S(2022)Assessing Information Quality of Wikipedia Articles Through Google’s E-A-T ModelIEEE Access10.1109/ACCESS.2022.317296210(52196-52209)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3172962
Adak SChakraborty SDas PDas MDash AHazra RMathew BSaha PSarkar SMukherjee A(2022)Mining the online infosphere: A surveyWIREs Data Mining and Knowledge Discovery10.1002/widm.145312:5Online publication date: 28-Feb-2022
https://doi.org/10.1002/widm.1453
Teblunthuis N(2021)Measuring Wikipedia Article Quality in One Dimension by Extending ORES with Ordinal RegressionProceedings of the 17th International Symposium on Open Collaboration10.1145/3479986.3479991(1-10)Online publication date: 15-Sep-2021
https://dl.acm.org/doi/10.1145/3479986.3479991
Liu YMedlar AGlowacka DHasibi FFang YAizawa A(2021)Can Language Models Identify Wikipedia Articles with Readability and Style Issues?Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3471158.3472234(113-117)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3471158.3472234
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents