Trust in an AI versus a Human teammate: : The effects of teammate identity and performance on Human-AI cooperation

Recent advances in artificial intelligence (AI) enable researchers to create more powerful AI agents that are becoming competent teammates for humans. However, human distrust of AI is a critical factor that may impede human-AI cooperation. Although AI agents have been endowed with anthropomorphic traits, such as a human-like appearance, in prior studies to improve human trust in AI, it is still an open question whether humans have more trust in an AI teammate and achieve better human-AI joint performance if they are deceived about the identity of their AI teammate as another human. This research assesses the effects of teammate identity (“human” vs. AI) and teammate performance (low-performing vs. high-performing AI) on human-AI cooperation through a human subjects study. The results of this study show that humans behaviorally trust the AI more than another human by accepting their AI teammate's decisions more often. In addition, teammate performance has a significant effect on human-AI joint performance in the study, while teammate identity does not. These results caution against deceiving humans about the identity of AI in future applications involving human-AI cooperation.