Abstract
This paper presents an AI-based co-creative system in which the interaction model focuses on emotional feedback, that is, the decisions about the creative contribution from the AI agent is based on the emotion detected in the human co-creator. In human-human collaboration, gestures, verbal communications, and emotional responses are among the general communication strategies used to shape the interactions between the collaborators and negotiate the contributions. Emotional feedback allows human collaborators to passively communicate their experience and their perception of the process without distracting the flow of the task. In human-human co-creative collaboration, participants interact and contribute to the task based on their perception of the collaboration over time. In designing human-AI co-creative collaboration, we address two challenges: (1) perceiving the user’s cognitive state to determine the dynamics of collaboration, such as whether the system should lead, follow, or wait, and (2) deciding what the agent should contribute to the artifact. This paper presents a model of an AI agent that addresses these challenges and the results of our study of participants that interact with the co-creative agent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ekman, P.: Basic emotions. Handb. Cogn. Emotion 98(45–60), 16 (1999)
Scherer, K.R.: What are emotions? and how can they be measured? Soc. Sci. Inf. 44(4), 695–729 (2005)
Sawyer, R.K.: Group Creativity: Music, Theater, Collaboration. Psychology Press (2014)
Ekman, P., Keltner, D.: Universal facial expressions of emotion. In: Segerstrale, U., Molnar, P. (eds.) Nonverbal Communication: Where Nature Meets Culture, pp. 27–46 (1997)
Jack, R.E., Garrod, O.G., Yu, H., Caldara, R., Schyns, P.G.: Facial expressions of emotion are not culturally universal. Proc. Natl. Acad. Sci. 109(19), 7241–7244 (2012)
Guo, K., Calver, L., Soornack, Y., Bourke, P.: Valence-dependent disruption in processing of facial expressions of emotion in early visual cortex—a transcranial magnetic stimulation study. J. Cognitive Neurosci. 32, 1–12 (2020)
Hudlicka, E.: To feel or not to feel: the role of affect in human–computer interaction. Int. J. Hum Comput. Stud. 59(1–2), 1–32 (2003)
Gutwin, C., Greenberg, S., Roseman, M.: Workspace awareness in real-time distributed groupware: Framework, widgets, and evaluation. In: Sasse, M.A., Cunningham, R.J., Winder, R.L. (eds.) People and Computers XI, pp. 281–298. Springer, London (1996). https://doi.org/10.1007/978-1-4471-3588-3_18
Jaques, N., Engel, J., Ha, D., Bertsch, F., Picard, R., Eck, D.: Learning via social awareness: improving sketch representations with facial feedback (2018)
Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. (TOG) 31(4), 1–10 (2012)
Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences. In: Advances in Neural Information Processing Systems, pp. 4299–4307 (2017)
Knox, W.B., Stone, P.: Interactively shaping agents via human reinforcement: the TAMER framework. In: Proceedings of the Fifth International Conference on Knowledge Capture, pp. 9–16 (2009)
Kellas, J.K., Trees, A.R.: Rating interactional sense-making in the process of joint storytelling. The sourcebook of nonverbal measures: Going beyond words, p. 281 (2005)
De Jaegher, H., Di Paolo, E.: Participatory sense-making. Phenomenol. Cognitive Sci. 6(4), 485–507 (2007)
DiPaola, S., McCaig, G.: Using artificial intelligence techniques to emulate the creativity of a portrait painter. In: Electronic Visualisation and the Arts, pp. 158–165 (2016)
Eligio, U.X., Ainsworth, S.E., Crook, C.K.: Emotion understanding and performance during computer-supported collaboration. Comput. Hum. Behav. 28(6), 2046–2054 (2012)
Picard, R.W., Daily, S.B.: Evaluating affective interactions: alternatives to asking what users feel. In: CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches 10(1056808.1057115), pp. 2119–2122 (2005)
Kapoor, A., Picard, R.W.: A real-time head nod and shake detector. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, pp. 1–5 (2001)
Affectiva Homepage. https://www.affectiva.com/. Accessed 11 June 2020
iMotions Homepage. https://imotions.com/. Accessed 11 June 2020
McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., Kaliouby, R.E.: AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3723–3726 (2016)
Bown, O.: Empirically grounding the evaluation of creative systems: incorporating interaction design. In: Proceedings of the Fifth International Conference on Computational Creativity, pp. 112–119 (2014)
Bown, O.: Player responses to a live algorithm: conceptualising computational creativity without recourse to human comparisons? In: International Conference on Computational Creativity, pp. 126–133 (2015)
Fuller, D., Magerko, B.: Shared mental models in improvisational performance. In: Proceedings of the Intelligent Narrative Technologies III Workshop, p. 15 (2010)
Teamviewer Homepage. https://www.teamviewer.com/en-us/. Accessed 11 June 2020
Mikolov, T., Chen, K., Corrado, G, Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Rehurek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Citeseer (2010)
Ziteboard Homepage. https://www.ziteboard.com/. Accessed 14 June 2020
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Abdellahi, S., Maher, M.L., Siddiqui, S., Rezwana, J., Almadan, A. (2020). Arny: A Study of a Co-creative Interaction Model Focused on Emotion Feedback. In: Stephanidis, C., Kurosu, M., Degen, H., Reinerman-Jones, L. (eds) HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence. HCII 2020. Lecture Notes in Computer Science(), vol 12424. Springer, Cham. https://doi.org/10.1007/978-3-030-60117-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-60117-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60116-4
Online ISBN: 978-3-030-60117-1
eBook Packages: Computer ScienceComputer Science (R0)