Abstract
The ability to make inferences about other’s mental states is referred to as having a Theory of Mind (ToM). Such ability is fundamental for human social activities such as empathy, teamwork, and communication. As intelligent agents being involved in diverse human-agent teams, they are also expected to be socially intelligent to become effective teammates. In this paper, we propose a computational ToM model which observes team behaviors and infer their mental states in a simulated search and rescue task. The model structure consists of a transformer-based language module and an RNN-based sequential mental state module in order to capture both team communication and behaviors for the ToM inference. To provide a feasible baseline for our ToM model, we present the same inference task to human observers recruited from Amazon MTurk. Results show that our proposed computational model achieves a comparable performance with human observers in the ToM inference task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baker, C., Saxe, R., Tenenbaum, J.: Bayesian theory of mind: modeling joint belief-desire attribution. In: Proceedings of the Thirty-Third Annual Conference of the Cognitive Science Society (2011)
Baker, C.L., Jara-Ettinger, J., Saxe, R., Tenenbaum, J.B.: Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nat. Hum. Behav. 1(4), 1–10 (2017)
Baker, C.L., Saxe, R., Tenenbaum, J.B.: Action understanding as inverse planning. Cognition 113(3), 329–349 (2009)
Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind’’? Cognition 21(1), 37–46 (1985)
Birch, S.A.: When knowledge is a curse: children’s and adults’ reasoning about mental states. Curr. Dir. Psychol. Sci. 14(1), 25–29 (2005)
Bratman, M.: Intentions, Plans, and Practical Reason. Harvard University Press, Cambridge (1987)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
DeChurch, L.A., Mesmer-Magnus, J.R.: The cognitive underpinnings of effective teamwork: a meta-analysis. J. Appl. Psychol. 95(1), 32 (2010)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Fiore, S., Bracken, B., Demir, M., Freeman, J., Lewis, M., Huang, L.: Transdisciplinary team research to develop theory of mind in human-AI teams. In: Proceedings of the Sixty-Fourth Annual Conference of the Human Factors and Ergonomics Society (2020)
Guo, Y., Jena, R., Hughes, D., Lewis, M., Sycara, K.: Transfer learning for human navigation and triage strategies prediction in a simulated urban search and rescue task. In: 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pp. 784–791. IEEE (2021)
Jain, V., et al.: Predicting human strategies in simulated search and rescue task. In: Accepted at NeurIPS 2020; Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI+HADR 2020) (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Li, H., Zheng, K., Sycara, K., Lewis, M.: Human theory of mind inference in search and rescue tasks. In: Proceedings of the 65th Annual Meeting of the Human Factors and Ergonomics Society. Human Factors and Ergonomics Society (2021)
Liu, S., Ullman, T.D., Tenenbaum, J.B., Spelke, E.S.: Ten-month-old infants infer the value of goals from the costs of actions. Science 358(6366), 1038–1041 (2017)
Oguntola, I., Hughes, D., Sycara, K.: Deep interpretable models of theory of mind. In: 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pp. 657–664. IEEE (2021)
Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S.A., Botvinick, M.: Machine theory of mind. In: International Conference on Machine Learning, pp. 4218–4227. PMLR (2018)
Samson, D., Apperly, I.: There is more to mind reading than having theory of mind concepts: new directions in theory of mind research. Infant Child Dev. 19(5), 443–454 (2010)
Stich, S., Ravenscroft, I.: What is folk psychology? Cognition 50(1–3), 447–468 (1992)
Wimmer, H., Perner, J.: Beliefs about beliefs: representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition 13(1), 103–128 (1983)
Acknowledgement
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001120C0036 and by the AFRL/AFOSR award FA9550-18-1-0251. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H. et al. (2022). Sequential Theory of Mind Modeling in Team Search and Rescue Tasks. In: Gurney, N., Sukthankar, G. (eds) Computational Theory of Mind for Human-Machine Teams. AAAI-FSS 2021. Lecture Notes in Computer Science, vol 13775. Springer, Cham. https://doi.org/10.1007/978-3-031-21671-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-21671-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21670-1
Online ISBN: 978-3-031-21671-8
eBook Packages: Computer ScienceComputer Science (R0)