More Web Proxy on the site http://driver.im/

research-article

Open access

Perceptions of Interaction Dynamics in Co-Creative AI: A Comparative Study of Interaction Modalities in Drawcto

Authors:

Manoj Deshpande,

Brian MagerkoAuthors Info & Claims

C&C '24: Proceedings of the 16th Conference on Creativity & Cognition

Pages 102 - 116

https://doi.org/10.1145/3635636.3656202

Published: 23 June 2024 Publication History

All formats PDF

Abstract

This paper explores how different interaction modalities with AI agents affect human perception of the co-creative process. We utilize the Wizard of Oz methodology within Drawcto, a co-creative drawing system, to examine co-creativity across three scenarios: human-human, human-robot, and human-software interactions. Using a mixed-methods approach, we combined insights using the Observable Creative Sensemaking (OCSM) method with data from structured interviews. The study involved 20 participants engaging in a collaborative drawing task under each interaction scenario. Key findings reveal an average OCSM curve indicative of typical human-human interactions, varied themes in human-AI collaboration, and a notable influence of AI embodiment on participant perceptions, with the robot interface resembling human-human collaboration more closely than the software interface. Overall, this research offers valuable insights into how different interaction modalities influence the perceived role of AI in the co-creative process and provides design considerations for enhancing human-AI co-creative interactions.

References

[1]

Sarah Abdellahi, Mary Lou Maher, Safat Siddiqui, Jeba Rezwana, and Ali Almadan. 2020. Arny: A Study of a Co-creative Interaction Model Focused on Emotion Feedback. In HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence, Constantine Stephanidis, Masaaki Kurosu, Helmut Degen, and Lauren Reinerman-Jones (Eds.). Vol. 12424. Springer International Publishing, Cham, 377–396. https://doi.org/10.1007/978-3-030-60117-1_28 Series Title: Lecture Notes in Computer Science.

Digital Library

[2]

Nono Martinez Alonso. 2017. Suggestive Drawing Among Human and Artificial Intelligences. Master’s thesis. Harvard University Graduate School of Design. https://nono.ma/suggestive-drawing

[3]

Rudolf Arnheim. 1957. Art and visual perception: A psychology of the creative eye. Univ of California Press.

[4]

Siddhartha Asthana, Pushpendra Singh, and Parul Gupta. 2015. Survival Analysis: Objective assessment of Wait Time in HCI. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). Association for Computing Machinery, New York, NY, USA, 367–376. https://doi.org/10.1145/2702123.2702428

Digital Library

[5]

Margaret S. Barrett, Andrea Creech, and Katie Zhukov. 2021. Creative collaboration and collaborative creativity: a systematic literature review. Frontiers in Psychology 12 (2021), 713445. https://www.frontiersin.org/articles/10.3389/fpsyg.2021.713445/full Publisher: Frontiers Media SA.

[6]

Virginia Braun and Victoria Clarke. 2021. One size fits all? What counts as quality practice in (reflexive) thematic analysis?Qualitative Research in Psychology 18, 3 (July 2021), 328–352. https://doi.org/10.1080/14780887.2020.1769238

[7]

Jacob T. Browne. 2019. Wizard of Oz Prototyping for Machine Learning Experiences. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–6. https://doi.org/10.1145/3290607.3312877

Digital Library

[8]

Linda Candy and Ernest Edmonds. 2002. Modeling Co-Creativity in Art and Technology. https://doi.org/10.1145/581710.581731 Journal Abbreviation: Proceedings of the Fourth Creativity and Cognition Conference Pages: 141 Publication Title: Proceedings of the Fourth Creativity and Cognition Conference.

Digital Library

[9]

Li-Yuan Chiou, Peng-Kai Hung, Rung-Huei Liang, and Chun-Teng Wang. 2023. Designing with AI: An Exploration of Co-Ideation with Image Generators. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. ACM, Pittsburgh PA USA, 1941–1954. https://doi.org/10.1145/3563657.3596001

Digital Library

[10]

Sougwen Chung. 2022. Sketching Symbiosis: Towards the Development of Relational Systems. In The Language of Creative AI, Craig Vear and Fabrizio Poltronieri (Eds.). Springer International Publishing, Cham, 259–276. https://doi.org/10.1007/978-3-031-10960-7_15 Series Title: Springer Series on Cultural Computing.

[11]

John W Creswell and Vicki L Plano Clark. 2017. Designing and conducting mixed methods research. Sage publications.

[12]

Nicholas Davis, Chih-PIn Hsiao, Kunwar Yashraj Singh, Lisa Li, Sanat Moningi, and Brian Magerko. 2015. Drawing Apprentice: An Enactive Co-Creative Agent for Artistic Collaboration. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition(C&C ’15). Association for Computing Machinery, New York, NY, USA, 185–186. https://doi.org/10.1145/2757226.2764555

Digital Library

[13]

Nicholas Davis, Chih-Pin Hsiao, Kunwar Yashraj Singh, Brenda Lin, and Brian Magerko. 2017. Creative sense-making: Quantifying interaction dynamics in co-creation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 356–366.

Digital Library

[14]

Nicholas Davis, Chih-PIn Hsiao, Kunwar Yashraj Singh, Lisa Li, and Brian Magerko. 2016. Empirically Studying Participatory Sense-Making in Abstract Drawing with a Co-Creative Cognitive Agent. In Proceedings of the 21st International Conference on Intelligent User Interfaces - IUI ’16. ACM Press, Sonoma, California, USA, 196–207. https://doi.org/10.1145/2856767.2856795

Digital Library

[15]

Hanne De Jaegher and Ezequiel Di Paolo. 2007. Participatory sense-making. Phenomenology and the cognitive sciences 6, 4 (2007), 485–507. Publisher: Springer.

[16]

Nathaniel Dennler, Changxiao Ruan, Jessica Hadiwijoyo, Brenna Chen, Stefanos Nikolaidis, and Maja Matarić. 2023. Design Metaphors for Understanding User Expectations of Socially Interactive Robot Embodiments. ACM Transactions on Human-Robot Interaction 12, 2 (June 2023), 1–41. https://doi.org/10.1145/3550489

Digital Library

[17]

Manoj Deshpande and Brian Magerko. 2021. Drawcto: A Multi-Agent Co-Creative AI for Collaborative Non-Representational Art. In Joint Proceedings of the AIIDE 2021 Workshops(CEUR Workshop Proceedings, Vol. 3217), Brent Harrison (Ed.). CEUR, Virtual Event, Lexington. https://ceur-ws.org/Vol-3217/#paper7ISSN: 1613-0073.

[18]

Manoj Deshpande, Milka Trajkova, Andrea Knowlton, and Brian Magerko. 2023. Observable Creative Sense-Making (OCSM): A Method For Quantifying Improvisational Co-Creative Interaction. In Creativity and Cognition. ACM, Virtual Event USA, 103–115. https://doi.org/10.1145/3591196.3593514

Digital Library

[19]

Sebastian Deterding, Jonathan Hook, Rebecca Fiebrink, Marco Gillies, Jeremy Gow, Memo Akten, Gillian Smith, Antonios Liapis, and Kate Compton. 2017. Mixed-initiative creative interfaces. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 628–635.

Digital Library

[20]

Paul Dourish. 2001. Where the action is. MIT press Cambridge.

[21]

Ozgur Eris, Nikolas Martelaro, and Petra Badke-Schaub. 2014. A comparative analysis of multimodal communication during design sketching in co-located and distributed environments. Design Studies 35, 6 (2014), 559–592. https://www.sciencedirect.com/science/article/pii/S0142694X14000337 Publisher: Elsevier.

[22]

Judith E. Fan, Monica Dinculescu, and David Ha. 2019. collabdraw: An Environment for Collaborative Sketching with an Artificial Agent. In Proceedings of the 2019 Conference on Creativity and Cognition(C&C ’19). Association for Computing Machinery, New York, NY, USA, 556–561. https://doi.org/10.1145/3325480.3326578

Digital Library

[23]

Jerry YH Fuh and W. D. Li. 2005. Advances in collaborative CAD: the-state-of-the art. Computer-Aided Design 37, 5 (2005), 571–581. https://www.sciencedirect.com/science/article/pii/S0010448504001575 Publisher: Elsevier.

Digital Library

[24]

Songwei Ge, Vedanuj Goswami, C. Lawrence Zitnick, and Devi Parikh. 2021. Creative Sketch Generation. arXiv:2011.10039 [cs] (March 2021). http://arxiv.org/abs/2011.10039 arXiv:2011.10039.

[25]

Vlad Petre Glaveanu. 2010. Paradigms in the study of creativity: Introducing the perspective of cultural psychology. New ideas in psychology 28, 1 (2010), 79–93. Publisher: Elsevier.

[26]

Matthew Guzdial and Mark Riedl. 2019. An Interaction Framework for Studying Co-Creative AI. arXiv:1903.09709 [cs] (March 2019). http://arxiv.org/abs/1903.09709 arXiv:1903.09709.

[27]

Onur Yüce Gün. 2017. Computing with Watercolor Shapes. In Computer-Aided Architectural Design. Future Trajectories, Gülen Çağdaş, Mine Özkar, Leman Figen Gül, and Ethem Gürer (Eds.). Vol. 724. Springer Singapore, Singapore, 252–269. https://doi.org/10.1007/978-981-10-5197-5_14 Series Title: Communications in Computer and Information Science.

[28]

David Ha and Douglas Eck. 2017. A Neural Representation of Sketch Drawings. arXiv:1704.03477 [cs, stat] (May 2017). http://arxiv.org/abs/1704.03477 arXiv:1704.03477.

[29]

Guy Hoffman and Gil Weinberg. 2010. Shimon: an interactive improvisational robotic marimba player. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. 3097–3102.

[30]

Mikhail Jacob and Brian Magerko. 2015. Viewpoints AI. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, Glasgow United Kingdom, 361–362. https://doi.org/10.1145/2757226.2757400

Digital Library

[31]

Vera John-Steiner. 2006. Creative collaboration. Vol. 2. Oxford University Press. https://books.google.com/books?hl=en&lr=&id=fEFRDAAAQBAJ&oi=fnd&pg=PR19&dq=john-Steiner, +V.+(2000).+Creative+Collaboration&ots=JKKsuLnE3x&sig=W4gKSGWEIyrgjqoTq8UWj2uYqTY

[32]

Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. 2016. Quick, Draw! The Data. https://quickdraw.withgoogle.com/

[33]

Anna Kantosalo and Hannu Toivonen. 2016. Modes for creative human-computer collaboration: Alternating and task-divided co-creativity. In Proceedings of the seventh international conference on computational creativity. 77–84.

[34]

Pegah Karimi, Kazjon Grace, Mary Lou Maher, and Nicholas Davis. 2018. Evaluating creativity in computational co-creative systems. arXiv preprint arXiv:1807.09886 (2018).

[35]

Pegah Karimi, Jeba Rezwana, Safat Siddiqui, Mary Lou Maher, and Nasrin Dehbozorgi. 2020. Creative sketching partner: an analysis of human-AI co-creativity. In Proceedings of the 25th International Conference on Intelligent User Interfaces(IUI ’20). Association for Computing Machinery, New York, NY, USA, 221–230. https://doi.org/10.1145/3377325.3377522

Digital Library

[36]

Martin Kemp. 2007. Leonardo da Vinci: the marvellous works of nature and man. Oxford University Press. https://books.google.com/books?hl=en&lr=&id=oJwVDAAAQBAJ&oi=fnd&pg=PR5&dq=Leonardo+da+Vinci:+The+Marvellous+Works+of+Nature+and+Man&ots=UdkDKkjUP-&sig=7Q5M4xx4zKUDoAhojcKGLPRsao4

[37]

C.D. Kidd and C. Breazeal. 2004. Effect of a robot on user perceptions. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), Vol. 4. 3559–3564 vol.4. https://doi.org/10.1109/IROS.2004.1389967

[38]

Kanta Kochhar-Lindgren, Davis Schneiderman, and Tom Denlinger. 2009. The Exquisite Corpse: Chance and Collaboration in Surrealism’s Parlor Game. U of Nebraska Press. https://books.google.com/books?hl=en&lr=&id=IdeR6vrFBeEC&oi=fnd&pg=PR3&dq=dada+and+surrealism+Exquisite+Corpse&ots=XPqsDsrxu7&sig=Hoq8_mr3BrHhVAchx_4RSRcTDWM

[39]

Carine Lallemand and Guillaume Gronier. 2012. Enhancing User eXperience during waiting time in HCI: contributions of cognitive psychology. In Proceedings of the Designing Interactive Systems Conference(DIS ’12). Association for Computing Machinery, New York, NY, USA, 751–760. https://doi.org/10.1145/2317956.2318069

Digital Library

[40]

Matthew V. Law, JiHyun Jeong, Amritansh Kwatra, Malte F. Jung, and Guy Hoffman. 2019. Negotiating the Creative Space in Human-Robot Collaborative Design. In Proceedings of the 2019 on Designing Interactive Systems Conference(DIS ’19). Association for Computing Machinery, New York, NY, USA, 645–657. https://doi.org/10.1145/3322276.3322343

Digital Library

[41]

Tomas Lawton, Kazjon Grace, and Francisco J Ibarrola. 2023. When is a tool a tool? user perceptions of system agency in human–ai co-creative drawing. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. 1978–1996.

Digital Library

[42]

Kwan Min Lee, Younbo Jung, Jaywoo Kim, and Sang Ryong Kim. 2006. Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people’s loneliness in human–robot interaction. International Journal of Human-Computer Studies 64, 10 (Oct. 2006), 962–973. https://doi.org/10.1016/j.ijhcs.2006.05.002

Digital Library

[43]

Zhuying Li, Yan Wang, Wei Wang, Stefan Greuter, and Florian ’Floyd’ Mueller. 2020. Empowering a Creative City: Engage Citizens in Creating Street Art through Human-AI Collaboration. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems(CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3382976

Digital Library

[44]

Yuyu Lin, Jiahao Guo, Yang Chen, Cheng Yao, and Fangtian Ying. 2020. It Is Your Turn: Collaborative Ideation With a Co-Creative Robot through Sketch. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376258

Digital Library

[45]

Fang Liu, Xiaoming Deng, Yu-Kun Lai, Yong-Jin Liu, Cuixia Ma, and Hongan Wang. 2019. SketchGAN: Joint Sketch Completion and Recognition With Generative Adversarial Network. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Long Beach, CA, USA, 5823–5832. https://doi.org/10.1109/CVPR.2019.00598

[46]

Michael Muller, Justin D. Weisz, and Werner Geyer. 2020. Mixed initiative generative AI interfaces: An analytic framework for generative AI applications. In Proceedings of the Workshop The Future of Co-Creative Systems-A Workshop on Human-Computer Co-Creativity of the 11th International Conference on Computational Creativity (ICCC 2020). https://computationalcreativity.net/workshops/cocreative-iccc20/papers/Future_of_co-creative_systems_185.pdf

[47]

Fiona Fui-Hoon Nah. 2004. A study on tolerable waiting time: how long are Web users willing to wait?Behaviour & Information Technology 23, 3 (May 2004), 153–163. https://doi.org/10.1080/01449290410001669914

[48]

Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1–13. https://doi.org/10.1145/3173574.3174223

Digital Library

[49]

Devi Parikh and C. Lawrence Zitnick. 2020. Exploring Crowd Co-creation Scenarios for Sketches. http://arxiv.org/abs/2005.07328 arXiv:2005.07328 [cs].

[50]

Jeba Rezwana and Mary Lou Maher. 2022. Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems. ACM Transactions on Computer-Human Interaction (2022). Publisher: ACM New York, NY.

[51]

R. Keith Sawyer. 2014. Group creativity: Music, theater, collaboration. Psychology Press. https://books.google.com/books?hl=en&lr=&id=JCR5AgAAQBAJ&oi=fnd&pg=PP1&dq=group+creativity+sawyer&ots=C9nzmqk-15&sig=mDol4P78FwUpwz7tdfHwheUcVGM

[52]

Angie Spoto, Natalia Oleynik, Sebastian Deterding, and Jon Hook. 2017. Library of Mixed-Initiative Creative Interfaces. http://mici.codingconduct.cc/

[53]

George Stiny. 2006. Shape: talking about seeing and doing. MIt Press.

[54]

Lucy Suchman and Lucy A. Suchman. 2007. Human-machine reconfigurations: Plans and situated actions. Cambridge university press.

Digital Library

[55]

Lingyun Sun, Pei Chen, Wei Xiang, Peng Chen, Wei-yue Gao, and Ke-jun Zhang. 2019. SmartPaint: a co-creative drawing system based on generative adversarial networks. Frontiers of Information Technology & Electronic Engineering 20, 12 (Dec. 2019), 1644–1656. https://doi.org/10.1631/FITEE.1900386

[56]

Milka Trajkova, Manoj Deshpande, Andrea Knowlton, Cassandra Monden, Duri Long, and Brian Magerko. 2023. AI Meets Holographic Pepper’s Ghost: A Co-Creative Public Dance Experience. In Companion Publication of the 2023 ACM Designing Interactive Systems Conference. 274–278.

[57]

Florent Vinchon, Todd Lubart, Sabrina Bartolotta, Valentin Gironnay, Marion Botella, Samira Bourgeois-Bougrine, Jean‐Marie Burkhardt, Nathalie Bonnardel, Giovanni Emanuele Corazza, Vlad Glăveanu, Michael Hanchett Hanson, Zorana Ivcevic, Maciej Karwowski, James C. Kaufman, Takeshi Okada, Roni Reiter‐Palmon, and Andrea Gaggioli. 2023. Artificial Intelligence & Creativity: A Manifesto for Collaboration. The Journal of Creative Behavior (June 2023), jocb.597. https://doi.org/10.1002/jocb.597

[58]

Miaomiao Wen, Nancy Baym, Omer Tamuz, Jaime Teevan, Susan T. Dumais, and Adam Kalai. 2015. OMG UR Funny! Computer-Aided Humor with an Application to Chat. In ICCC. 86–93. https://www.computationalcreativity.net/iccc2015/proceedings/4_3Wen.pdf

[59]

Kai Chi Yam, Yochanan E. Bigman, Pok Man Tang, Remus Ilies, David De Cremer, Harold Soh, and Kurt Gray. 2021. Robots at work: People prefer—and forgive—service robots with perceived feelings.Journal of Applied Psychology 106, 10 (2021), 1557. https://psycnet.apa.org/record/2020-74093-001 Publisher: American Psychological Association.

[60]

Georgios N. Yannakakis, Antonios Liapis, and Constantine Alexopoulos. 2014. Mixed-initiative co-creativity. (2014). Publisher: Foundations of Digital Games.

[61]

Chao Zhang, Cheng Yao, Jianhui Liu, Zili Zhou, Weilin Zhang, Lijuan Liu, Fangtian Ying, Yijun Zhao, and Guanyun Wang. 2021. StoryDrawer: A Co-Creative Agent Supporting Children’s Storytelling through Collaborative Drawing. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems(CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3411763.3451785

Digital Library

Cited By

Li HXue TZhang ALuo XKong LHuang G(2024)The application and impact of artificial intelligence technology in graphic design: A critical interpretive synthesisHeliyon10.1016/j.heliyon.2024.e4003710:21(e40037)Online publication date: Nov-2024
https://doi.org/10.1016/j.heliyon.2024.e40037

Index Terms

Perceptions of Interaction Dynamics in Co-Creative AI: A Comparative Study of Interaction Modalities in Drawcto
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Collaborative content creation
  2. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Toward New Ecologies of Cyberphysical Representational Forms, Scales, and Interaction Modalities
TEI '19: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction

Research on tangible user interfaces commonly focuses on tangible interfaces acting alone or in comparison with multi-touch or graphical interfaces. In contrast, hybrid approaches can be seen as the norm for established "mainstream'' interaction ...
Comparing Tangible and Multi-touch Interaction for Interactive Data Visualization Tasks
TEI '16: Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction

Interactive visualization plays a key role in the analysis of large datasets. It can help users to explore data, investigate hypotheses and find patterns. The easier and more tangible the interaction, the more likely it is to enhance understanding. This ...
Mobile-based tangible interaction techniques for shared displays
MobileHCI '14: Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services

This tutorial explores the possibility of using touchscreen-based mobile devices as active tangibles on an interactive tabletop surface. The tutorial starts with an open discussion about various aspects of tangible interaction, including an overview of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

C&C '24: Proceedings of the 16th Conference on Creativity & Cognition

June 2024

718 pages

ISBN:9798400704857

DOI:10.1145/3635636

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2024

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

C&C '24

Sponsor:

SIGCHI

C&C '24: Creativity and Cognition

June 23 - 26, 2024

IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 108 of 371 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
471
Total Downloads

Downloads (Last 12 months)471
Downloads (Last 6 weeks)108

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li HXue TZhang ALuo XKong LHuang G(2024)The application and impact of artificial intelligence technology in graphic design: A critical interpretive synthesisHeliyon10.1016/j.heliyon.2024.e4003710:21(e40037)Online publication date: Nov-2024
https://doi.org/10.1016/j.heliyon.2024.e40037

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten