More Web Proxy on the site http://driver.im/

research-article

Open access

Exploring the Learnability of Program Synthesizers by Novice Programmers

Authors:

Dhanya Jayagopal,

Sarah E. ChasinsAuthors Info & Claims

UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

Article No.: 64, Pages 1 - 15

https://doi.org/10.1145/3526113.3545659

Published: 28 October 2022 Publication History

All formats PDF

Abstract

Modern program synthesizers are increasingly delivering on their promise of lightening the burden of programming by automatically generating code, but little research has addressed how we can make such systems learnable to all. In this work, we ask: What aspects of program synthesizers contribute to and detract from their learnability by novice programmers? We conducted a thematic analysis of 22 observations of novice programmers, during which novices worked with existing program synthesizers, then participated in semi-structured interviews. Our findings shed light on how their specific points in the synthesizer design space affect these tools’ learnability by novice programmers, including the type of specification the synthesizer requires, the method of invoking synthesis and receiving feedback, and the size of the specification. We also describe common misconceptions about what constitutes meaningful progress and useful specifications for the synthesizers, as well as participants’ common behaviors and strategies for using these tools. From this analysis, we offer a set of design opportunities to inform the design of future program synthesizers that strive to be learnable by novice programmers. This work serves as a first step toward understanding how we can make program synthesizers more learnable by novices, which opens up the possibility of using program synthesizers in educational settings as well as developer tooling oriented toward novice programmers.

References

[1]

D. Alan Allport, Elizabeth A. Styles, and Shulan Hsieh. 1994. Shifting Intentional Set: Exploring the Dynamic Control of Tasks. In Attention and Performance 15: Conscious and Nonconscious Information Processing. The MIT Press, 421–452.

[2]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, 1–13. https://doi.org/10.1145/3290605.3300233

Digital Library

[3]

Matej Balog, Alexander Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2017. DeepCoder: Learning to Write Programs. In Proceedings of ICLR’17.

[4]

Osbert Bastani, Xin Zhang, and Armando Solar-Lezama. 2019. Synthesizing Queries via Interactive Sketching. CoRR abs/1912.12659(2019). arXiv:1912.12659http://arxiv.org/abs/1912.12659

[5]

Victoria Bellotti and Keith Edwards. 2001. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human–Computer Interaction 16, 2-4 (Dec. 2001), 193–212. https://doi.org/10.1207/S15327051HCI16234_05

Digital Library

[6]

Francisco Enrique Vicente Castro and Kathi Fisler. 2016. On the Interplay Between Bottom-Up and Datatype-Driven Program Design. In SIGCSE Technical Symposium. https://doi.org/10.1145/2839509.2844574

Digital Library

[7]

Francisco Enrique Vicente Castro and Kathi Fisler. 2020. Qualitative Analyses of Movements Between Task-Level and Code-Level Thinking of Novice Programmers. Association for Computing Machinery, 487–493. https://doi.org/10.1145/3328778.3366847

Digital Library

[8]

Sarah E. Chasins, Maria Mueller, and Rastislav Bodik. 2018. Rousillon: Scraping Distributed Hierarchical Web Data. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology(UIST ’18). Association for Computing Machinery, 963–975. https://doi.org/10.1145/3242587.3242661

Digital Library

[9]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. (2021). arxiv:2107.03374 [cs.LG]

[10]

Codecademy. 2022. Learn to Code - for Free. https://www.codecademy.com. Accessed: 2022-07-26.

[11]

Simon P. Davies and Adrian M. Castell. 1994. From Individuals to Groups Through Artifacts: The Changing Semantics of Design in Software Development. In User-Centred Requirements for Software Engineering Environments, David J. Gilmore, Russel L. Winder, and Françoise Détienne (Eds.). https://doi.org/10.1007/978-3-662-03035-6_2

[12]

Asaf Degani. 1996. Modeling Human-Machine Systems: On Modes, Error, and Patterns of Interaction. Ph.D. Dissertation. Georgia Institute of Technology.

[13]

Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. Association for Computing Machinery, 1–12. https://doi.org/10.1145/3313831.3376442

Digital Library

[14]

Kasra Ferdowsifard, Shraddha Barke, Hila Peleg, Sorin Lerner, and Nadia Polikarpova. 2021. LooPy: Interactive Program Synthesis with Control Structures. Proc. ACM Program. Lang. 5, OOPSLA, Article 153 (Oct. 2021), 29 pages. https://doi.org/10.1145/3485530

Digital Library

[15]

Kasra Ferdowsifard, Allen Ordookhanians, Hila Peleg, Sorin Lerner, and Nadia Polikarpova. 2020. Small-Step Live Programming by Example. Association for Computing Machinery, 614–626. https://doi.org/10.1145/3379337.3415869

Digital Library

[16]

John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-Output Examples. ACM SIGPLAN Notices 50, 6 (June 2015), 229–239. https://doi.org/10.1145/2813885.2737977

Digital Library

[17]

Leah Findlater and Joanna McGrenere. 2004. A Comparison of Static, Adaptive, and Adaptable Menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’04). Association for Computing Machinery, 89–96. https://doi.org/10.1145/985692.985704

Digital Library

[18]

Kathi Fisler and Francisco Enrique Vicente Castro. 2017. Sometimes, Rainfall Accumulates: Talk-Alouds with Novice Functional Programmers. In International Conference on Computing Education Research(ICER). https://doi.org/10.1145/3105726.3106183

Digital Library

[19]

Nat Friedman. 2021. Introducing GitHub Copilot: your AI pair programmer. https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/. Accessed: 2022-04-04.

[20]

Joel Galenson, Philip Reames, Rastislav Bodik, Björn Hartmann, and Koushik Sen. 2014. CodeHint: Dynamic and Interactive Synthesis of Code Snippets. In Proceedings of the 36th International Conference on Software Engineering(ICSE 2014). Association for Computing Machinery, 653–663. https://doi.org/10.1145/2568225.2568250

Digital Library

[21]

Ivan Gavran, Eva Darulova, and Rupak Majumdar. 2020. Interactive Synthesis of Temporal Specifications from Examples and Natural Language. Proc. ACM Program. Lang. 4, OOPSLA, Article 201 (Nov. 2020), 26 pages. https://doi.org/10.1145/3428269

Digital Library

[22]

GitHub Inc.2021. GitHub Copilot: Your AI pair programmer. https://copilot.github.com/. Accessed: 2022-03-31.

[23]

Sumit Gulwani. 2011. Automating String Processing in Spreadsheets Using Input-Output Examples. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages(POPL ’11). Association for Computing Machinery, 317–330. https://doi.org/10.1145/1926385.1926423

Digital Library

[24]

Sumit Gulwani, Susmit Jha, Ashish Tiwari, and Ramarathnam Venkatesan. 2011. Synthesis of Loop-Free Programs. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation(PLDI ’11). Association for Computing Machinery, 62–73. https://doi.org/10.1145/1993498.1993506

Digital Library

[25]

Sumit Gulwani and Ramarathnam Venkatesan. 2009. Component Based Synthesis Applied to Bitvector Circuits. Technical Report MSR-TR-2010-12. Microsoft Research.

[26]

Brian Hempel and Ravi Chugh. 2016. Semi-Automated SVG Programming via Direct Manipulation. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 379–390. https://doi.org/10.1145/2984511.2984575

Digital Library

[27]

Austin Z. Henley, Julian Ball, Benjamin Klein, Aiden Rutter, and Dylan Lee. 2021. An Inquisitive Code Editor for Addressing Novice Programmers’ Misconceptions of Program Behavior. IEEE Press, 165–170. https://doi.org/10.1109/ICSE-SEET52601.2021.00026

Digital Library

[28]

K. Höök. 2000. Steps to Take before Intelligent User Interfaces Become Real. Interacting with Computers 12, 4 (2000), 409–426. https://doi.org/10.1016/S0953-5438(99)00006-5

[29]

Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of CHI ’99, ACM SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA, ACM Press.159–166.

Digital Library

[30]

Thibaud Hottelier, Ras Bodik, and Kimiko Ryokai. 2014. Programming by Manipulation for Layout. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology(UIST ’14). Association for Computing Machinery, 231–241. https://doi.org/10.1145/2642918.2647378

Digital Library

[31]

Jingmei Hu, Priyan Vaithilingam, Stephen Chong, Margo Seltzer, and Elena L. Glassman. 2021. Assuage: Assembly Synthesis Using A Guided Exploration. In The 34th Annual ACM Symposium on User Interface Software and Technology(UIST ’21). Association for Computing Machinery, 134–148. https://doi.org/10.1145/3472749.3474740

Digital Library

[32]

A. T. Jersild. 1927. Mental Set and Shift. Archives of Psychology 14, 89 (1927).

[33]

Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-Guided Component-Based Program Synthesis. In 2010 ACM/IEEE 32nd International Conference on Software Engineering, Vol. 1. 215–224. https://doi.org/10.1145/1806799.1806833

Digital Library

[34]

Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J. Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 386, 19 pages. https://doi.org/10.1145/3491102.3501870

Digital Library

[35]

Etienne Kneuss, Ivan Kuraj, Viktor Kuncak, and Philippe Suter. 2013. Synthesis modulo Recursive Functions. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications(OOPSLA ’13). Association for Computing Machinery, 407–426. https://doi.org/10.1145/2509136.2509555

Digital Library

[36]

Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces(IUI ’15). Association for Computing Machinery, 126–137. https://doi.org/10.1145/2678025.2701399

Digital Library

[37]

Tessa Lau. 2009. Why Programming-By-Demonstration Systems Fail: Lessons Learned for Usable AI. AI Magazine 30, 4 (Oct. 2009), 65–65. https://doi.org/10.1609/aimag.v30i4.2262

Digital Library

[38]

Sorin Lerner. 2020. Projection Boxes: On-the-Fly Reconfigurable Visualization for Live Programming. Association for Computing Machinery, 1–7. https://doi.org/10.1145/3313831.3376494

Digital Library

[39]

Colleen M. Lewis. 2012. The Importance of Students’ Attention to Program State: A Case Study of Debugging Behavior. In Proceedings of the Ninth Annual International Conference on International Computing Education Research(ICER ’12). Association for Computing Machinery, 127–134. https://doi.org/10.1145/2361276.2361301

Digital Library

[40]

Toby Jia-Jun Li, Amos Azaria, and Brad A. Myers. 2017. SUGILITE: Creating Multimodal Smartphone Automation by Demonstration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, 6038–6049. https://doi.org/10.1145/3025453.3025483

Digital Library

[41]

Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M. Mitchell, and Brad A. Myers. 2019. PUMICE: A Multi-Modal Agent That Learns Concepts and Conditionals from Natural Language and Demonstrations. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). Association for Computing Machinery, 577–589. https://doi.org/10.1145/3332165.3347899

Digital Library

[42]

Brian Y. Lim and Anind K. Dey. 2009. Assessing Demand for Intelligibility in Context-Aware Applications. In Proceedings of the 11th International Conference on Ubiquitous Computing(UbiComp ’09). Association for Computing Machinery, 195–204. https://doi.org/10.1145/1620545.1620576

Digital Library

[43]

Justin Lubin, Nick Collins, Cyrus Omar, and Ravi Chugh. 2020. Program Sketching with Live Bidirectional Evaluation. Proceedings of the ACM on Programming Languages 4, ICFP (Aug. 2020), 109:1–109:29. https://doi.org/10.1145/3408991

Digital Library

[44]

Ewa Luger and Abigail Sellen. 2016. “Like Having a Really Bad PA”: The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, 5286–5297. https://doi.org/10.1145/2858036.2858288

Digital Library

[45]

David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid Mining: Helping to Navigate the API Jungle. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation(PLDI ’05). Association for Computing Machinery, 48–61. https://doi.org/10.1145/1065010.1065018

Digital Library

[46]

Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011. Mind Your Language: On Novices’ Interactions with Error Messages. In Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Portland, Oregon, USA) (Onward! 2011). Association for Computing Machinery, New York, NY, USA, 3–18. https://doi.org/10.1145/2048237.2048241

Digital Library

[47]

Mikaël Mayer, Gustavo Soares, Maxim Grechkin, Vu Le, Mark Marron, Oleksandr Polozov, Rishabh Singh, Benjamin Zorn, and Sumit Gulwani. 2015. User Interaction Models for Disambiguation in Programming by Example. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology(UIST ’15). Association for Computing Machinery, 291–301. https://doi.org/10.1145/2807442.2807459

Digital Library

[48]

Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, and Abhishek Udupa. 2019. On the Fly Synthesis of Edit Suggestions. Proc. ACM Program. Lang. 3, OOPSLA, Article 143 (Oct. 2019), 29 pages. https://doi.org/10.1145/3360569

Digital Library

[49]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, 1–7. https://doi.org/10.1145/3173574.3173580

Digital Library

[50]

Sujata Narayana. 2020. Power BI Desktop August 2020 Feature Summary: Text/CSV By Example (preview). https://powerbi.microsoft.com/en-us/blog/power-bi-desktop-august-2020-feature-summary/#_text_csv. Accessed: 2022-04-05.

[51]

Greg L. Nelson, Benjamin Xie, and Amy J. Ko. 2017. Comprehension First: Evaluating a Novel Pedagogy and Tutoring System for Program Tracing in CS1. In Proceedings of the 2017 ACM Conference on International Computing Education Research(ICER ’17). Association for Computing Machinery, 2–11. https://doi.org/10.1145/3105726.3106178

Digital Library

[52]

Donald A. Norman. 1994. How Might People Interact with Agents. Commun. ACM 37, 7 (July 1994), 68–71. https://doi.org/10.1145/176789.176796

Digital Library

[53]

Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-Example-Directed Program Synthesis. SIGPLAN Not. 50, 6 (June 2015), 619–630. https://doi.org/10.1145/2813885.2738007

Digital Library

[54]

Hila Peleg, Shachar Itzhaky, and Sharon Shoham. 2018. Abstraction-Based Interaction Model for Synthesis. In Verification, Model Checking, and Abstract Interpretation, Isil Dillig and Jens Palsberg (Eds.). Springer International Publishing, 382–405. https://doi.org/10.1007/978-3-319-73721-8_18

[55]

Hila Peleg, Sharon Shoham, and Eran Yahav. 2018. Programming Not Only by Example. In Proceedings of the 40th International Conference on Software Engineering(ICSE ’18). Association for Computing Machinery, 1114–1124. https://doi.org/10.1145/3180155.3180189

Digital Library

[56]

Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program Synthesis from Polymorphic Refinement Types. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation(PLDI ’16). Association for Computing Machinery, 522–538. https://doi.org/10.1145/2908080.2908093

Digital Library

[57]

Jef Raskin. 2000. The Humane Interface: New Directions for Designing Interactive Systems. Addison-Wesley Professional.

Digital Library

[58]

Ruth Ravichandran, Sang-Wha Sien, Shwetak N. Patel, Julie A. Kientz, and Laura R. Pina. 2017. Making Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, 6864–6875. https://doi.org/10.1145/3025453.3025557

Digital Library

[59]

Robert D. Rogers and Stephen Monsell. 1995. Costs of a Predictible Switch between Simple Cognitive Tasks. Journal of Experimental Psychology: General 124, 2 (1995), 207–231. https://doi.org/10.1037/0096-3445.124.2.207

[60]

Chad Rothschiller. 2012. Flash Fill. https://www.microsoft.com/en-us/microsoft-365/blog/2012/08/09/flash-fill/. Accessed: 2022-04-04.

[61]

Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic Superoptimization. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems(ASPLOS ’13). Association for Computing Machinery, 305–316. https://doi.org/10.1145/2451116.2451150

Digital Library

[62]

Steven C. Shaffer. 2005. Ludwig: An Online Programming Tutoring and Assessment System. SIGCSE Bull. 37, 2 (June 2005), 56–60. https://doi.org/10.1145/1083431.1083464

Digital Library

[63]

Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. SIGARCH Comput. Archit. News 34, 5 (Oct. 2006), 404–415. https://doi.org/10.1145/1168919.1168907

Digital Library

[64]

Elliot Soloway and Kate Ehrlich. 1986. Empirical Studies of Programming Knowledge. In Readings in Artificial Intelligence and Software Engineering, Charles Richand Richard C. Waters (Eds.). https://doi.org/10.1016/B978-0-934613-12-5.50042-2

[65]

James C. Spohrer and Elliot Soloway. 1986. Novice Mistakes: Are the Folk Wisdoms Correct?Communications of the ACM (July 1986). https://doi.org/10.1145/6138.6145

Digital Library

[66]

Amanda Swearngin, Chenglong Wang, Alannah Oleson, James Fogarty, and Amy J. Ko. 2020. Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints. Association for Computing Machinery, 1–13. https://doi.org/10.1145/3313831.3376593

Digital Library

[67]

Nava Tintarev and Judith Masthoff. 2015. Explaining Recommendations: Design and Evaluation. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, 353–382. https://doi.org/10.1007/978-1-4899-7637-6_10

[68]

Priyan Vaithilingam and Philip J. Guo. 2019. Bespoke: Interactively Synthesizing Custom GUIs from Command-Line Applications By Demonstration. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). Association for Computing Machinery, 563–576. https://doi.org/10.1145/3332165.3347944

Digital Library

[69]

Chenglong Wang, Yu Feng, Rastislav Bodik, Alvin Cheung, and Isil Dillig. 2019. Visualization by Example. Proceedings of the ACM on Programming Languages 4, POPL (Dec. 2019), 49:1–49:28. https://doi.org/10.1145/3371117

Digital Library

[70]

Chenglong Wang, Yu Feng, Rastislav Bodik, Isil Dillig, Alvin Cheung, and Amy J Ko. 2021. Falx: Synthesis-Powered Visualization Authoring. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, Article 106, 15 pages. https://doi.org/10.1145/3411764.3445249

Digital Library

[71]

Jacqueline Whalley and Nadia Kasto. 2014. A Qualitative Think-Aloud Study of Novice Programmers’ Code Writing Strategies. In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education(ITiCSE ’14). Association for Computing Machinery, 279–284. https://doi.org/10.1145/2591708.2591762

Digital Library

[72]

Brian Whitworth. 2005. Polite Computing. Behaviour & Information Technology 24, 5 (Sept. 2005), 353–363. https://doi.org/10.1080/01449290512331333700

[73]

Mark Wilson-Thomas. 2019. Refactoring made easy with IntelliCode!https://devblogs.microsoft.com/visualstudio/refactoring-made-easy-with-intellicode/. Accessed: 2022-04-04.

[74]

Rayoung Yang, Eunice Shin, Mark W. Newman, and Mark S. Ackerman. 2015. When Fitness Trackers Don’t ’Fit’: End-User Difficulties in the Assessment of Personal Tracking Device Accuracy. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing(UbiComp ’15). Association for Computing Machinery, 623–634. https://doi.org/10.1145/2750858.2804269

Digital Library

[75]

Tianyi Zhang, Zhiyang Chen, Yuanli Zhu, Priyan Vaithilingam, Xinyu Wang, and Elena L. Glassman. 2021. Interpretable Program Synthesis. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, Article 105, 16 pages. https://doi.org/10.1145/3411764.3445646

Digital Library

[76]

Tianyi Zhang, London Lowmanstone, Xinyu Wang, and Elena L. Glassman. 2020. Interactive Program Synthesis by Augmented Examples. Association for Computing Machinery, 627–648. https://doi.org/10.1145/3379337.3415900

Digital Library

Cited By

Liu JLi SDong Q(2024)Collaboration with Generative Artificial Intelligence: An Exploratory Study Based on Learning AnalyticsJournal of Educational Computing Research10.1177/0735633124124244162:5(1234-1266)Online publication date: 1-Apr-2024
https://doi.org/10.1177/07356331241242441
Kiesler NScholz IAlbrecht JStappert FWienkop U(2024)Novice Learners of Programming and Generative AI - Prior Knowledge MattersProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699580(1-2)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699580
Yu Cd'Amorim M(2024)Unlocking the Full Potential of AI Chatbots: A Guide to Maximizing Your Digital CompanionsCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3664457(680-682)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3664457
Show More Cited By

Recommendations

Program visualization and explanation for novice C programmers
ACE '14: Proceedings of the Sixteenth Australasian Computing Education Conference - Volume 148

Program visualization and natural language explanations of program behaviour have been shown to assist novice programmers with improving their programming knowledge, correcting misunderstandings, and debugging programs. These techniques have been used ...
Experience Helps, but It Isn't Everything: Exploring Causes of Affective State in Novice Programmers
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2

Affective state, referring to an individual's feeling, can impact students' confidence and retention in CS, particularly for novice programmers. However, little research has been conducted to examine how moments that occur during programming impact ...
Exploring Novice Programming Behavior over Time
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 2

This work focuses on the effect that programming time has on novice programmers' interaction with two versions of the BlueJ programming environment that differ in compilation mechanism and error message presentation. We utilize programming process data ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

October 2022

1363 pages

ISBN:9781450393201

DOI:10.1145/3526113

Editors:
Maneesh Agrawala
Stanford University, USA
,
Jacob O. Wobbrock
University of Washington, USA
,
Eytan Adar
University of Michigan, USA
,
Vidya Setlur
Tableau Research, USA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

UIST '22

Sponsor:

UIST '22: The 35th Annual ACM Symposium on User Interface Software and Technology

October 29 - November 2, 2022

OR, Bend, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,581
Total Downloads

Downloads (Last 12 months)752
Downloads (Last 6 weeks)101

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu JLi SDong Q(2024)Collaboration with Generative Artificial Intelligence: An Exploratory Study Based on Learning AnalyticsJournal of Educational Computing Research10.1177/0735633124124244162:5(1234-1266)Online publication date: 1-Apr-2024
https://doi.org/10.1177/07356331241242441
Kiesler NScholz IAlbrecht JStappert FWienkop U(2024)Novice Learners of Programming and Generative AI - Prior Knowledge MattersProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699580(1-2)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699580
Yu Cd'Amorim M(2024)Unlocking the Full Potential of AI Chatbots: A Guide to Maximizing Your Digital CompanionsCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3664457(680-682)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3664457
Tian YKummerfeld JLi TZhang T(2024)SQLucid: Grounding Natural Language Database Queries with Interactive ExplanationsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676368(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676368
Tankelevitch LKewenig VSimkute AScott ASarkar ASellen ARintel S(2024)The Metacognitive Demands and Opportunities of Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642902(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642902
Gu KGrunde-McLaughlin MMcNutt AHeer JAlthoff T(2024)How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz StudyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641891(1-22)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641891
Simkute ATankelevitch LKewenig VScott ASellen ARintel S(2024)Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI InteractionInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2405782(1-22)Online publication date: 15-Oct-2024
https://doi.org/10.1080/10447318.2024.2405782
Sergeyuk AGolubev YBryksin TAhmed I(2024)Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forwardInformation and Software Technology10.1016/j.infsof.2024.107610(107610)Online publication date: Oct-2024
https://doi.org/10.1016/j.infsof.2024.107610
Naik AYin JKamath AMa QWu SMurray CBogart CSakr MRose C(2024)Generating Situated Reflection Triggers About Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative LearningArtificial Intelligence in Education10.1007/978-3-031-64302-6_4(46-59)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-64302-6_4
Prather JReeves BDenny PBecker BLeinonen JLuxton-Reilly APowell GFinnie-Ansley JSantos E(2023)“It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice ProgrammersACM Transactions on Computer-Human Interaction10.1145/361736731:1(1-31)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1145/3617367
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents