[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3308558.3313638acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Trajectories of Blocked Community Members: Redemption, Recidivism and Departure

Published: 13 May 2019 Publication History

Abstract

Community norm violations can impair constructive communication and collaboration online. As a defense mechanism, community moderators often address such transgressions by temporarily blocking the perpetrator. Such actions, however, come with the cost of potentially alienating community members. Given this tradeoff, it is essential to understand to what extent, and in which situations, this common moderation practice is effective in reinforcing community rules.
In this work, we introduce a computational framework for studying the future behavior of blocked users on Wikipedia. After their block expires, they can take several distinct paths: they can reform and adhere to the rules, but they can also recidivate, or straight-out abandon the community. We reveal that these trajectories are tied to factors rooted both in the characteristics of the blocked individual and in whether they perceived the block to be fair and justified. Based on these insights, we formulate a series of prediction tasks aiming to determine which of these paths a user is likely to take after being blocked for their first offense, and demonstrate the feasibility of these new tasks. Overall, this work builds towards a more nuanced approach to moderation by highlighting the tradeoffs that are in play.

References

[1]
Moira Burke and Robert Kraut. 2008. Mopping up: Modeling Wikipedia Promotion Decisions. In Proceedings of CSCW.
[2]
Carlos Castillo, Francesco Fabbri, and Diego Saez-Trumper. 2017. Current Practices of Online Community Managers. (2017).
[3]
Stevie Chancellor, Andrea Hu, and Munmun De Choudhury. 2018. Norms Matter: Contrasting Social Support Around Behavior Change in Online Weight Loss Communities. In Proceedings of CHI.
[4]
Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet's Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. In Proceedings of CSCW, Vol. 2.
[5]
Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How Community Feedback Shapes User Behavior. In Proceedings of ICWSM.
[6]
Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Antisocial Behavior in Online Discussion Communities. In Proceedings of CSCW.
[7]
Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic Decision Making and the Cost of Fairness. In Proceedings of KDD.
[8]
Tiago Oliveira Cunha, Ingmar Weber, Hamed Haddadi, and Gisele L. Pappa. 2016. The Effect of Social Feedback in a Reddit Weight Loss Community. In Proceedings of DH.
[9]
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of ACL.
[10]
Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. No Country for Old Members: User Lifecycle and Linguistic Change in Online Communities. In Proceedings of WWW.
[11]
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of ICWSM.
[12]
Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In Proceedings of KDD.
[13]
A. Forte and A. Bruckman. 2008. Scaling Consensus: Increasing Decentralization in Wikipedia Governance. In Proceedings of HICSS.
[14]
R. Stuart Geiger. 2016. Bot-Based Collective Blocklists in Twitter: The Counterpublic Moderation of Harassment in a Networked Public Space. Information, Communication & Society19, 6 (June 2016).
[15]
R. Stuart Geiger and David Ribes. 2010. The Work of Sustaining Order in Wikipedia: The Banning of a Vandal. In Proceedings of CSCW.
[16]
Harold G. Grasmick and George J. Bryjak. 1980. The Deterrent Effect of Perceived Severity of Punishment. Social Forces59, 2 (Dec. 1980).
[17]
Aaron Halfaker, R. Stuart Geiger, Jonathan T. Morgan, and John Riedl. 2013. The Rise and Decline of an Open Collaboration System. American Behavioral Scientist57, 5 (May 2013).
[18]
Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't Bite the Newbies: How Reverts Affect the Quantity and Quality of Wikipedia Work. In Proceedings of WikiSym.
[19]
William L. Hamilton, Justine Zhang, Cristian Danescu-Niculescu-Mizil, Dan Jurafsky, and Jure Leskovec. 2017. Loyalty in Online Communities. In Proceedings of ICWSM.
[20]
Yiqing Hua, Cristian Danescu-Niculescu-Mizil, Dario Taraborelli, Nithum Thain, Jeffery Sorensen, and Lucas Dixon. 2018. WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community. In Proceedings of EMNLP.
[21]
Shagun Jhaver, Larry Chan, and Amy Bruckman. 2018. The View from the Other Side: The Border between Controversial Speech and Harassment on Kotaku in Action. First Monday23, 2 (Feb. 2018).
[22]
Shagun Jhaver, Sucheta Ghoshal, Amy Bruckman, and Eric Gilbert. 2018. Online Harassment and Content Moderation: The Case of Blocklists. In Proceedings of CSCW.
[23]
Joseph M. Kayany. 1998. Contexts of Uninhibited Online Behavior: Flaming in Social Newsgroups on Usenet. Journal of the American Society for Information Science49, 12 (Jan. 1998).
[24]
Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating Behavior in Online Communities.In Building Successful Online Communities: Evidence-Based Social Design. MIT Press.
[25]
Gary King, Jennifer Pan, and Margaret E. Roberts. 2013. How Censorship in China Allows Government Criticism but Silences Collective Expression. American Political Science Review107, 02 (May 2013).
[26]
Aniket Kittur, Bryan Pendleton, and Robert E. Kraut. 2009. Herding the Cats: The Influence of Groups in Coordinating Peer Production. In Proceedings of WikiSym.
[27]
Steven Klepper and Daniel Nagin. 1989. Tax Compliance and Perceptions of the Risks of Detection and Criminal Prosecution. Law & Society Review23, 2 (1989).
[28]
Srijan Kumar, William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2018. Community Interaction and Conflict on the Web.
[29]
Cliff Lampe and Paul Resnick. 2004. Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space. In Proceedings of CHI.
[30]
J Leskovec, D P Huttenlocher, and Kleinberg. 2010. Governance in Social Media: A Case Study of the Wikipedia Promotion Process. In Proceedings of ICWSM.
[31]
Ping Liu, Joshua Guberman, Libby Hemphill, and Aron Culotta. 2018. Forecasting the Presence and Intensity of Hostility on Instagram Using Linguistic and Social Features. In Proceedings of ICWSM.
[32]
Toni Makkai and John Braithwaite. 1994. Reintegrative Shaming and Compliance with Regulatory Standards. Criminology32, 3 (Aug. 1994).
[33]
Aiden R McGillicuddy, Jean-Gregoire Bernard, and Jocelyn Ann Cranefield. 2016. Controlling Bad Behavior in Online Communities: An Examination of Moderation Work. In Proceedings of ICIS.
[34]
Burt L. Monroe, Michael P. Colaresi, and Kevin M. Quinn. 2008. Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict. Political Analysis16, 04 (2008).
[35]
Lev Muchnik, Sinan Aral, and Sean Taylor. 2013. Social Influence Bias: A Randomized Experiment. Science341(2013).
[36]
Kevin Munger. 2017. Tweetment Effects on the Tweeted: Experimentally Reducing Racist Harassment. Political Behavior39, 3 (Sept. 2017).
[37]
Edward Newell, David Jurgens, Haji Mohammad Saleem, Hardik Vala, Jad Sassine, Caitrin Armstrong, and Derek Ruths. 2016. User Migration in Online Social Networks: A Case Study on Reddit During a Period of Community Unrest. In Proceedings of ICWSM.
[38]
Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive Language Detection in Online User Content. In Proceedings of WWW.
[39]
Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush Varshney. 2018. The Effect of Extremist Violence on Hateful Speech Online. In Proceedings of ICWSM.
[40]
Raymond Paternoster, Robert Brame, Ronet Bachman, and Lawrence W. Sherman. 1997. Do Fair Procedures Matter? The Effect of Procedural Justice on Spouse Assault. Law & Society Review31, 1 (1997).
[41]
Umashanthi Pavalanathan, Xiaochuang Han, and Jacob Eisenstein. 2018. Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's Neutrality Norm. In Proceedings of CSCW.
[42]
Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgílio A. F. Almeida, and Wagner Meira Jr.2018. Characterizing and Detecting Hateful Users on Twitter. In Proceedings of ICWSM.
[43]
Paul R Rosenbaum. 2010. Design of Observational Studies. Springer, New York.
[44]
Joseph Seering, Robert Kraut, and Laura Dabbish. 2017. Shaping Pro and Anti-Social Behavior on Twitch Through Moderation and Example-Setting. In Proceedings of CSCW.
[45]
Qinlan Shen, Michael Miller Yoder, Yohan Jo, and Carolyn P. Rose. 2018. Perceptions of Censorship and Moderation Bias in Political Debate Forums. In Proceedings of ICWSM.
[46]
Lawrence W. Sherman. 1993. Defiance, Deterrence, and Irrelevance: A Theory of the Criminal Sanction. Journal of Research in Crime and Delinquency30, 4 (Nov. 1993).
[47]
Vivek K. Singh, Marie L. Radford, Qianjia Huang, and Susan Furrer. 2017. ”They Basically like Destroyed the School One Day”: On Newer App Features and Cyberbullying in Schools. In Proceedings of CSCW.
[48]
Michael Tonry. 2008. Learning from the Limitations of Deterrence Research. Crime and Justice37, 1 (Jan. 2008).
[49]
Tom R. Tyler. 1990. Why People Obey the Law. Yale University Press, New Haven.
[50]
Jessica Vitak, Kalyani Chadha, Linda Steiner, and Zahra Ashktorab. 2017. Identifying Women's Experiences With and Strategies for Mitigating Negative Effects of Online Harassment. In Proceedings of CSCW.
[51]
Svitlana Volkova and Eric Bell. 2017. Identifying Effective Signals to Predict Deleted and Suspended Accounts on Twitter across Languages. In Proceedings of ICWSM.
[52]
Kirk R. Williams. 2005. Arrest and Intimate Partner Violence: Toward a More Complete Application of Deterrence Theory. Aggression and Violent Behavior10, 6 (Sept. 2005).
[53]
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex Machina: Personal Attacks Seen at Scale. In Proceedings of WWW.
[54]
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification Without Disparate Mistreatment. In Proceedings of WWW.
[55]
Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. In Proceedings of ACL.
[56]
Justine Zhang, William L. Hamilton, Cristian Danescu-Niculescu-Mizil, Dan Jurafsky, and Jure Leskovec. 2017. Community Identity and User Engagement in a Multi-Community Landscape. In Proceedings of ICWSM.

Cited By

View all
  • (2024)Opening Up: Interdisciplinary Guidance for Managing Open EcosystemsSSRN Electronic Journal10.2139/ssrn.4821969Online publication date: 2024
  • (2024)EXPRESS: Fighting Misinformation on Social Media: an Empirical Investigation of the Impact of Prominence Reduction PoliciesProduction and Operations Management10.1177/10591478241283839Online publication date: 9-Sep-2024
  • (2024)Investigating Influential Users' Responses to Permanent Suspension on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/36373568:CSCW1(1-41)Online publication date: 26-Apr-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
WWW '19: The World Wide Web Conference
May 2019
3620 pages
ISBN:9781450366748
DOI:10.1145/3308558
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

WWW '19
WWW '19: The Web Conference
May 13 - 17, 2019
CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)38
  • Downloads (Last 6 weeks)3
Reflects downloads up to 01 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Opening Up: Interdisciplinary Guidance for Managing Open EcosystemsSSRN Electronic Journal10.2139/ssrn.4821969Online publication date: 2024
  • (2024)EXPRESS: Fighting Misinformation on Social Media: an Empirical Investigation of the Impact of Prominence Reduction PoliciesProduction and Operations Management10.1177/10591478241283839Online publication date: 9-Sep-2024
  • (2024)Investigating Influential Users' Responses to Permanent Suspension on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/36373568:CSCW1(1-41)Online publication date: 26-Apr-2024
  • (2024)Community Begins Where Moderation Ends: Peer Support and Its Implications for Community-Based RehabilitationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642675(1-18)Online publication date: 11-May-2024
  • (2023)User Features and Gendered Patterns in Repeat Online Advice-Seeking BehaviorSSRN Electronic Journal10.2139/ssrn.4616409Online publication date: 2023
  • (2023)You Can (Not) Say What You Want: Using Algospeak to Contest and Evade Algorithmic Content Moderation on TikTokSocial Media + Society10.1177/205630512311945869:3Online publication date: 31-Aug-2023
  • (2023)Towards Intersectional Moderation: An Alternative Model of Moderation Built on Care and PowerProceedings of the ACM on Human-Computer Interaction10.1145/36100477:CSCW2(1-32)Online publication date: 4-Oct-2023
  • (2023)Understanding the Behaviors of Toxic Accounts on RedditProceedings of the ACM Web Conference 202310.1145/3543507.3583522(2797-2807)Online publication date: 30-Apr-2023
  • (2023)Automated Content Moderation Increases Adherence to Community GuidelinesProceedings of the ACM Web Conference 202310.1145/3543507.3583275(2666-2676)Online publication date: 30-Apr-2023
  • (2021)After Violation But Before Sanction: Understanding Volunteer Moderators' Profiling Processes Toward Violators in Live Streaming CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/34795545:CSCW2(1-25)Online publication date: 18-Oct-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media