More Web Proxy on the site http://driver.im/

research-article

Open access

People May Punish, But Not Blame Robots

Authors:

Yvonne de Kort,

Wijnand IJsselsteijnAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 715, Pages 1 - 11

https://doi.org/10.1145/3411764.3445284

Published: 07 May 2021 Publication History

All formats PDF

Abstract

As robots may take a greater part in our moral decision-making processes, whether people hold them accountable for moral harm becomes critical to explore. Blame and punishment signify moral accountability, often involving emotions. We quantitatively looked into people’s willingness to blame or punish an emotional vs. non-emotional robot that admits to its wrongdoing. Studies 1 and 2 (online video interaction) showed that people may punish a robot due to its lack of perceived emotional capacity than its perceived agency. Study 3 (in the lab) demonstrated that people were neither willing to blame nor punish the robot. Punishing non-emotional robots seems more likely than blaming them, yet punishment towards robots is more likely to arise online than offline. We reflect on if and why victimized humans (and those who care for them) may seek out retributive justice against robot scapegoats when there are no humans to hold accountable for moral harm.

References

[1]

Arthur Aron, Elaine N Aron, and Danny Smollan. 1992. Inclusion of other in the self scale and the structure of interpersonal closeness. Journal of Personality and Social Psychology 63, 4(1992), 596–612.

[2]

Christoph Bartneck and Merel Keijsers. 2020. The morality of abusing a robot. Paladyn, Journal of Behavioral Robotics 11, 1 (2020), 271–283.

[3]

Iris Bohnet and Bruno S Frey. 1999. Social distance and other-regarding behavior in dictator games: Comment. American Economic Review 89, 1 (1999), 335–339.

[4]

Kevin M Carlsmith, John M Darley, and Paul H Robinson. 2002. Why do we punish? Deterrence and just deserts as motives for punishment. Journal of Personality and Social Psychology 83, 2(2002), 284.

[5]

Andreas Brekke Carlsson. 2017. Blameworthiness as deserved guilt. The Journal of Ethics 21, 1 (2017), 89–115.

[6]

Mark Coeckelbergh. 2009. Virtual moral agency, virtual moral responsibility: on the moral significance of the appearance, perception, and performance of artificial agents. AI & Society 24, 2 (2009), 181–189.

Digital Library

[7]

John Danaher. 2016. Robots, law and the retribution gap. Ethics and Information Technology 18, 4 (2016), 299–309.

Digital Library

[8]

John Danaher. 2019. Welcoming Robots into the Moral Circle: A Defence of Ethical Behaviourism. Science and Engineering Ethics(2019), 1–27.

[9]

Stephen Darwall. 2004. Respect and the second-person standpoint. In Proceedings and Addresses of the American Philosophical Association, Vol. 78. JSTOR, 43–59.

[10]

Celso M de Melo and Jonathan Gratch. 2015. People show envy, not guilt, when making decisions with machines. In 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 315–321.

Digital Library

[11]

Daniel Dennett. 2009. Darwin’s “strange inversion of reasoning”. Proceedings of the National Academy of Sciences 106, Supplement 1(2009), 10061–10065.

[12]

Daniel C Dennett. 2017. From bacteria to Bach and back: The evolution of minds. WW Norton & Company.

[13]

Antony Duff and Zachary Hoskins. 2017. Legal Punishment. Stanford Encyclopedia of Philosophy (Jul 2017). https://plato.stanford.edu/entries/legal-punishment/PosRetMeaDes (Accessed on 12/23/2020).

[14]

Robin Antony Duff. 2003. Probation, punishment and restorative justice: Should Al Turism be engaged in punishment?The Howard Journal of Criminal Justice 42, 2 (2003), 181–197.

[15]

Luciano Floridi and Jeff W Sanders. 2004. On the morality of artificial agents. Minds and Machines 14, 3 (2004), 349–379.

Digital Library

[16]

Philippa Foot. 1967. The problem of abortion and the doctrine of double effect. Oxford Review 5(1967).

[17]

Michel Foucault. 2012 [1975]. Discipline and punish: The birth of the prison. Vintage.

[18]

Miranda Fricker. 2016. What’s the point of blame? A paradigm based explanation. Noûs 50, 1 (2016), 165–183.

[19]

Caleb Furlough, Thomas Stokes, and Douglas J Gillan. 2019. Attributing Blame to Robots: I. The Influence of Robot Autonomy. Human Factors (2019), 1–11.

[20]

Geoffrey P Goodwin, Jared Piazza, and Paul Rozin. 2014. Moral character predominates in person perception and evaluation.Journal of Personality and Social Psychology 106, 1(2014), 148–168.

[21]

Heather M Gray, Kurt Gray, and Daniel M Wegner. 2007. Dimensions of mind perception. science 315, 5812 (2007), 619–619.

[22]

Kurt Gray, Chelsea Schein, and Adrian F Ward. 2014. The myth of harmless wrongs in moral cognition: Automatic dyadic completion from sin to suffering.Journal of Experimental Psychology: General 143, 4 (2014), 1600.

[23]

Kurt Gray, Liane Young, and Adam Waytz. 2012. Mind perception is the essence of morality. Psychological inquiry 23, 2 (2012), 101–124.

[24]

Jonathan Haidt. 2003. The moral emotions. In Series in Affective Science. Handbook of Affective Sciences, & H. H. Goldsmit R. J. Davidson, K. R. Scherer (Ed.). Vol. 11. Oxford University Press, 852–870.

[25]

Francis Hutcheson. 2008 [1725]. An Inquiry into the Original of Our Ideas of Beauty and Virtue: In Two Treatises. Liberty Fund.

[26]

David O Johnson and Raymond H Cuijpers. 2019. Investigating the effect of a humanoid robot’s head position on imitating human emotions. International Journal of Social Robotics 11, 1 (2019), 65–74.

[27]

Merel Keijsers and Christoph Bartneck. 2018. Mindless Robots Get Bullied. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (Chicago, IL, USA) (HRI ’18). Association for Computing Machinery, New York, NY, USA, 205–214.

Digital Library

[28]

Mansur Khamitov, Jeff D Rotman, and Jared Piazza. 2016. Perceiving the agency of harmful agents: A test of dehumanization versus moral typecasting accounts. Cognition 146(2016), 33–47.

[29]

Taemie Kim and Pamela Hinds. 2006. Who should I blame? Effects of autonomy and transparency on attributions in human-robot interaction. In 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 80–85.

[30]

Joshua Knobe. 2003. Intentional action and side effects in ordinary language. Analysis 63, 3 (2003), 190–194.

[31]

Takanori Komatsu. 2016. How do people judge moral wrongness in a robot and in its designers and owners regarding the consequences of the robot’s behaviors?. In 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 1168–1171.

Digital Library

[32]

Minha Lee, Gale Lucas, and Jonathan Gratch. 2021. Comparing mind perception in strategic exchanges: Human-agent negotiation, dictator and ultimatum games. Journal of Multimodal User Interfaces. In press (2021), 1–15.

[33]

Minha Lee, Gale Lucas, Johnathan Mell, Emmanuel Johnson, and Jonathan Gratch. 2019. What’s on Your Virtual Mind? Mind Perception in Human-Agent Negotiations. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (Paris, France) (IVA ’19). Association for Computing Machinery, New York, NY, USA, 38–45.

Digital Library

[34]

Daniel T Levin, Stephen S Killingsworth, Megan M Saylor, Stephen M Gordon, and Kazuhiko Kawamura. 2013. Tests of concepts about different kinds of minds: Predictions about the behavior of computers, robots, and people. Human–Computer Interaction 28, 2 (2013), 161–191.

[35]

Bertram F. Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano. 2015. Sacrifice One For the Good of Many? People Apply Different Moral Norms to Human and Robot Agents. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (Portland, Oregon, USA) (HRI ’15). Association for Computing Machinery, New York, NY, USA, 117–124.

Digital Library

[36]

Andreas Matthias. 2004. The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6, 3 (2004), 175–183.

Digital Library

[37]

Daniel McDermott. 2001. The permissibility of punishment. Law and Philosophy 20, 4 (2001), 403–432.

[38]

Robert M McFatter. 1978. Sentencing strategies and justice: Effects of punishment philosophy on sentencing decisions.Journal of Personality and Social Psychology 36, 12(1978), 1490–1500.

[39]

Catrin Misselhorn. 2015. Collective agency and cooperation in natural and artificial systems. In Collective Agency and Cooperation in Natural and Artificial Systems, Catrin Misselhorn (Ed.). Springer, 3–24.

[40]

Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers Are Social Actors. In Conference Companion on Human Factors in Computing Systems (Boston, Massachusetts, USA) (CHI ’94). Association for Computing Machinery, New York, NY, USA, 204.

Digital Library

[41]

Sven Nyholm. 2018. Attributing agency to automated systems: Reflections on human–robot collaborations and responsibility-loci. Science and Engineering Ethics 24, 4 (2018), 1201–1219.

[42]

Sven Nyholm. 2020. Humans and robots: Ethics, agency, and anthropomorphism. Rowman & Littlefield Publishers.

[43]

Yohsuke Ohtsubo. 2007. Perceived intentionality intensifies blameworthiness of negative behaviors: Blame-praise asymmetry in intensification effect. Japanese Psychological Research 49, 2 (2007), 100–110.

[44]

Jesse Prinz. 2008. Is morality innate?In Moral psychology. The evolution of morality: Adaptations and innateness, Walter Sinnott-Armstrong (Ed.). Vol. 1. MIT Press Cambridge, 367–406.

[45]

Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places.Cambridge University Press.

Digital Library

[46]

Beat Rossmy, Sarah Theres Völkel, Elias Naphausen, Patricia Kimm, Alexander Wiethoff, and Andreas Muxel. 2020. Punishable AI: Examining Users’ Attitude Towards Robot Punishment. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. Association for Computing Machinery, New York, NY, USA, 179–191.

Digital Library

[47]

P. Salvini, G. Ciaravella, W. Yu, G. Ferri, A. Manzi, B. Mazzolai, C. Laschi, S. R. Oh, and P. Dario. 2010. How safe are service robots in urban environments? Bullying a robot. In 19th International Symposium in Robot and Human Interactive Communication (ROMAN). 1–7.

[48]

Thomas Michael Scanlon. 2013. Interpreting blame. In Blame. Its nature and norms, D. Justin Coates and Neal A. Tognazzini (Eds.). 84–99.

[49]

Simone Schnall, Jonathan Haidt, Gerald L Clore, and Alexander H Jordan. 2008. Disgust as embodied moral judgment. Personality and social psychology bulletin 34, 8 (2008), 1096–1109.

[50]

David Shoemaker. 2013. Blame and punishment. Blame: Its nature and norms(2013), 100–118.

[51]

Elaine Short, Justin Hart, Michelle Vu, and Brian Scassellati. 2010. No fair!! an interaction with a cheating robot. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 219–226.

[52]

Eva EA Skoe, Nancy Eisenberg, and Amanda Cumberland. 2002. The role of reported emotion in real-life and hypothetical moral dilemmas. Personality and Social Psychology Bulletin 28, 7 (2002), 962–973.

[53]

Robert Sparrow. 2007. Killer robots. Journal of Applied Philosophy 24, 1 (2007), 62–77.

[54]

Peter Frederick Strawson. 2008 [1963]. Freedom and resentment and other essays. Routledge.

[55]

Judith Jarvis Thomson. 1976. Killing, letting die, and the trolley problem. The Monist 59, 2 (1976), 204–217.

[56]

John Voiklis, Boyoung Kim, Corey Cusimano, and Bertram F Malle. 2016. Moral judgments of human vs. robot agents. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 775–780.

Digital Library

Cited By

Hindennach SShi LMiletić FBulling A(2024)Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI ResearchProceedings of the ACM on Human-Computer Interaction10.1145/36410098:CSCW1(1-43)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641009
Segate R(2024)Exposing, Reversing, and Inheriting Crimes as Traumas from the Neurosciences to Epigenetics: Why Criminal Law Cannot Yet Afford A(nother) Biology-induced OverhaulCriminal Justice Ethics10.1080/0731129X.2024.2376444(1-48)Online publication date: 24-Jul-2024
https://doi.org/10.1080/0731129X.2024.2376444
van de Poel IHopster JLöhr GZiliotti EBuijsman SBrey P(2023)1: IntroductionEthics of Socially Disruptive Technologies10.11647/obp.0366.01(11-32)Online publication date: 5-Sep-2023
https://doi.org/10.11647/obp.0366.01
Show More Cited By

Index Terms

People May Punish, But Not Blame Robots
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
2. Human-centered computing
  1. Human computer interaction (HCI)

Index terms have been assigned to the content through auto-classification.

Recommendations

Blaming the Reluctant Robot: Parallel Blame Judgments for Robots in Moral Dilemmas across U.S. and Japan
HRI '21: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Previous work has shown that people provide different moral judgments of robots and humans in the case of moral dilemmas. In particular, robots are blamed more when they fail to intervene in a situation in which they can save multiple lives but must ...
Do people hold a humanoid robot morally accountable for the harm it causes?
HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

Robots will increasingly take on roles in our social lives where they can cause humans harm. When robots do so, will people hold robots morally accountable? To investigate this question, 40 undergraduate students individually engaged in a 15-minute ...
An Explorative Comparison of Blame Attributions to Companion Robots Across Various Moral Dilemmas
HAI '18: Proceedings of the 6th International Conference on Human-Agent Interaction

We report results from an exploratory study with a humanoid robot asking participants (n = 30) to attribute blameworthiness to other robots that made decisions in moral dilemmas. Drawing from current research in machine ethics, we identify three ethical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,682
Total Downloads

Downloads (Last 12 months)406
Downloads (Last 6 weeks)50

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hindennach SShi LMiletić FBulling A(2024)Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI ResearchProceedings of the ACM on Human-Computer Interaction10.1145/36410098:CSCW1(1-43)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641009
Segate R(2024)Exposing, Reversing, and Inheriting Crimes as Traumas from the Neurosciences to Epigenetics: Why Criminal Law Cannot Yet Afford A(nother) Biology-induced OverhaulCriminal Justice Ethics10.1080/0731129X.2024.2376444(1-48)Online publication date: 24-Jul-2024
https://doi.org/10.1080/0731129X.2024.2376444
van de Poel IHopster JLöhr GZiliotti EBuijsman SBrey P(2023)1: IntroductionEthics of Socially Disruptive Technologies10.11647/obp.0366.01(11-32)Online publication date: 5-Sep-2023
https://doi.org/10.11647/obp.0366.01
Lima GGrgic-Hlaca NJeong JCha M(2023)Who Should Pay When Machines Cause Harm? Laypeople’s Expectations of Legal Damages for Machine-Caused HarmProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3593992(236-246)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3593992
Wester JLee Mvan Berkel N(2023)Moral Transparency as a Mitigator of Moral Bias in Conversational User InterfacesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603752(1-6)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603752
Lima GGrgić-Hlača NCha M(2023)Blaming Humans and Machines: What Shapes People’s Reactions to Algorithmic HarmProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580953(1-26)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580953
Lee MRuijten PFrank LIJsselsteijn W(2023)Here’s Looking at You, Robot: The Transparency Conundrum in HRI2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309653(2120-2127)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309653
Tacci AManzi FDio CMarchetti ARiva GMassaro D(2023)Moral Context Matters: A study of Adolescents’ Moral Judgment towards Robots2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)10.1109/ACIIW59127.2023.10388182(1-6)Online publication date: 10-Sep-2023
https://doi.org/10.1109/ACIIW59127.2023.10388182
Deroy O(2023)The Ethics of Terminology: Can We Use Human Terms to Describe AI?Topoi10.1007/s11245-023-09934-142:3(881-889)Online publication date: 8-Jun-2023
https://doi.org/10.1007/s11245-023-09934-1
Zouganeli ELentzas A(2023)Cognitive Robotics - Towards the Development of Next-Generation Robotics and Intelligent SystemsNordic Artificial Intelligence Research and Development10.1007/978-3-031-17030-0_2(16-25)Online publication date: 2-Feb-2023
https://doi.org/10.1007/978-3-031-17030-0_2
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents