[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3313831.3376382acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Enabling Data-Driven API Design with Community Usage Data: A Need-Finding Study

Published: 23 April 2020 Publication History

Abstract

APIs are becoming the fundamental building block of modern software and their usability is crucial to programming efficiency and software quality. Yet API designers find it hard to gather and interpret user feedback on their APIs. To close the gap, we interviewed 23 API designers from 6 companies and 11 open-source projects to understand their practices and needs. The primary way of gathering user feedback is through bug reports and peer reviews, as formal usability testing is prohibitively expensive to conduct in practice. Participants expressed a strong desire to gather real-world use cases and understand users' mental models, but there was a lack of tool support for such needs. In particular, participants were curious about where users got stuck, their workarounds, common mistakes, and unanticipated corner cases. We highlight several opportunities to address those unmet needs, including developing new mechanisms that systematically elicit users' mental models, building mining frameworks that identify recurring patterns beyond shallow statistics about API usage, and exploring alternative design choices made in similar libraries.

Supplementary Material

ZIP File (paper255aux.zip)
The auxiliary material includes (1) the interview guide and consent form and (2) a complete list of codes and themes in six meta categories.
MP4 File (a255-zhang-presentation.mp4)

References

[1]
2017. Google API design guide. (2017). https://cloud.google.com/apis/design/ Accessed: 2019-08--29.
[2]
2019. Apigee: A Cross-Cloud API Management Platform. (2019). https://cloud.google.com/apigee/ Accessed: 2019-08--29.
[3]
Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01989 (2016).
[4]
Christopher Bogart, Christian Kästner, James Herbsleb, and Ferdian Thung. 2016. How to break an API: cost negotiation and community values in three software ecosystems. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 109--120.
[5]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.
[6]
Casey Casalnuovo, Bogdan Vasilescu, Premkumar Devanbu, and Vladimir Filkov. 2015. Developer onboarding in GitHub: the role of prior social links and language experience. In Proceedings of the 2015 10th joint meeting on foundations of software engineering. ACM, 817--828.
[7]
Chunyang Chen, Zhenchang Xing, and Yang Liu. 2019a. What's spain's paris? mining analogical libraries from q&a discussions. Empirical Software Engineering 24, 3 (2019), 1155--1194.
[8]
Chunyang Chen, Zhenchang Xing, Yang Liu, and Kent Long Xiong Ong. 2019b. Mining likely analogical apis across third-party libraries via large-scale unsupervised api semantics embedding. IEEE Transactions on Software Engineering (2019).
[9]
Fuxiang Chen and Sunghun Kim. 2015. Crowd debugging. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 320--332.
[10]
Steven Clarke. 2005. Describing and measuring API usability with the cognitive dimensions. In Cognitive Dimensions of Notations 10th Anniversary Workshop. Citeseer, 131.
[11]
Laura Dabbish, Colleen Stuart, Jason Tsay, and Jim Herbsleb. 2012. Social coding in GitHub: transparency and collaboration in an open software repository. In Proceedings of the ACM 2012 conference on computer supported cooperative work. ACM, 1277--1286.
[12]
Barthélémy Dagenais and Martin P Robillard. 2010. Creating and evolving developer documentation: understanding the decisions of open source contributors. In Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering. ACM, 127--136.
[13]
Barthélémy Dagenais and Martin P Robillard. 2012. Recovering traceability links between an API and its learning resources. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 47--57.
[14]
Norman K Denzin and Yvonna S Lincoln. 2011. The Sage handbook of qualitative research. Sage.
[15]
Ekwa Duala-Ekoko and Martin P Robillard. 2012. Asking and answering questions about unfamiliar APIs: An exploratory study. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 266--276.
[16]
Manuel Egele, David Brumley, Yanick Fratantonio, and Christopher Kruegel. 2013. An empirical study of cryptographic misuse in android applications. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security. ACM, 73--84.
[17]
Brian Ellis, Jeffrey Stylos, and Brad Myers. 2007. The factory pattern in API design: A usability evaluation. In Proceedings of the 29th international conference on Software Engineering. IEEE Computer Society, 302--312.
[18]
Sascha Fahl, Marian Harbach, Thomas Muders, Lars Baumgärtner, Bernd Freisleben, and Matthew Smith. 2012. Why Eve and Mallory love Android: An analysis of Android SSL (in) security. In Proceedings of the 2012 ACM conference on Computer and communications security. ACM, 50--61.
[19]
Umer Farooq, Leon Welicki, and Dieter Zirkler. 2010. API usability peer reviews: a method for evaluating the usability of application programming interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2327--2336.
[20]
Qing Gao, Hansheng Zhang, Jie Wang, Yingfei Xiong, Lu Zhang, and Hong Mei. 2015. Fixing recurring crash bugs via analyzing q&a sites (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 307--318.
[21]
Martin Georgiev, Subodh Iyengar, Suman Jana, Rishita Anubhai, Dan Boneh, and Vitaly Shmatikov. 2012. The most dangerous code in the world: validating SSL certificates in non-browser software. In Proceedings of the 2012 ACM conference on Computer and communications security. ACM, 38--49.
[22]
Elena L Glassman, Tianyi Zhang, Björn Hartmann, and Miryung Kim. 2018. Visualizing API usage examples at scale. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 580.
[23]
Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual programming environments: a 'cognitive dimensions' framework. Journal of Visual Languages & Computing 7, 2 (1996), 131--174.
[24]
Natalie Gruska, Andrzej Wasylkowski, and Andreas Zeller. 2010. Learning from 6,000 projects: lightweight cross-project anomaly detection. In Proceedings of the 19th international symposium on Software testing and analysis. ACM, 119--130.
[25]
Xiaodong Gu, Hongyu Zhang, Dongmei Zhang, and Sunghun Kim. 2016. Deep API learning. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 631--642.
[26]
Amber Horvath, Mariann Nagy, Finn Voichick, Mary Beth Kery, and Brad A Myers. 2019. Methods for Investigating Mental Models for Learners of APIs. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW0158.
[27]
Andrew J Ko, Brad A Myers, and Htet Htet Aung. 2004. Six learning barriers in end-user programming systems. In Visual Languages and Human Centric Computing, 2004 IEEE Symposium on. IEEE, 199--206.
[28]
Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1--10.
[29]
Hongwei Li, Sirui Li, Jiamou Sun, Zhenchang Xing, Xin Peng, Mingwei Liu, and Xuejiao Zhao. 2018. Improving api caveats accessibility by mining api caveats knowledge graph. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 183--193.
[30]
Robyn Longhurst. 2003. Semi-structured interviews and focus groups. Key methods in geography 3 (2003), 143--156.
[31]
Andrew Macvean, Luke Church, John Daughtry, and Craig Citro. 2016a. API Usability at Scale. In Psychology of Programming Interest Group. 26.
[32]
Andrew Macvean, Martin Maly, and John Daughtry. 2016b. API design reviews at scale. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 849--858.
[33]
Jennifer Marlow, Laura Dabbish, and Jim Herbsleb. 2013. Impression formation in online peer production: activity traces and personal profiles in github. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 117--128.
[34]
Phil McMinn, Muzammil Shahbaz, and Mark Stevenson. 2012. Search-based test input generation for string data types using the results of web queries. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. IEEE, 141--150.
[35]
Evan Moritz, Mario Linares-Vásquez, Denys Poshyvanyk, Mark Grechanik, Collin McMillan, and Malcom Gethers. 2013. Export: Detecting and visualizing api usages in large source code repositories. In Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 646--651.
[36]
Manish Motwani and Yuriy Brun. 2019. Automatically generating precise Oracles from structured natural language specifications. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 188--199.
[37]
Lauren Murphy, Mary Beth Kery, Oluwatosin Alliyu, Andrew Macvean, and Brad A Myers. 2018. API Designers in the Field: Design Practices and Challenges for Creating Usable APIs. In 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 249--258.
[38]
Emerson Murphy-Hill, Caitlin Sadowski, Andrew Head, John Daughtry, Andrew Macvean, Ciera Jaspan, and Collin Winter. 2018. Discovering API usability problems at scale. In Proceedings of the 2nd International Workshop on API Usage and Evolution. ACM, 14--17.
[39]
Brad A. Myers. 2017. Human-centered Methods for Improving API Usability. In Proceedings of the 1st International Workshop on API Usage and Evolution (WAPI '17). IEEE Press, Piscataway, NJ, USA, 2--2.
[40]
Brad A Myers and Jeffrey Stylos. 2016. Improving API usability. Commun. ACM 59, 6 (2016), 62--69.
[41]
Tung Thanh Nguyen, Hoan Anh Nguyen, Nam H. Pham, Jafar M. Al-Kofahi, and Tien N. Nguyen. 2009. Graph-based Mining of Multiple Object Usage Patterns. In Proceedings of the the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering (ESEC/FSE '09). ACM, NY, NY, USA, 383--392.
[42]
Raphael Pham, Leif Singer, Olga Liskin, Fernando Figueira Filho, and Kurt Schneider. 2013. Creating a shared understanding of testing culture on a social coding site. In Proceedings of the 2013 International Conference on Software Engineering. IEEE Press, 112--121.
[43]
Huilian Sophie Qiu, Yucen Lily Li, Susmita Padala, Anita Sarma, and Bogdan Vasilescu. 2019a. The Signals That Potential Contributors Look for When Choosing Open-Source Projects. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 122 (Nov. 2019), 29 pages.
[44]
Huilian Sophie Qiu, Alexander Nolte, Anita Brown, Alexander Serebrenik, and Bogdan Vasilescu. 2019b. Going farther together: The impact of social capital on sustained participation in open source. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 688--699.
[45]
Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Acm Sigplan Notices, Vol. 49. ACM, 419--428.
[46]
TJ Robertson, Shrinu Prabhakararao, Margaret Burnett, Curtis Cook, Joseph R Ruthruff, Laura Beckwith, and Amit Phalgune. 2004. Impact of interruption style on end-user debugging. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 287--294.
[47]
Martin P Robillard. 2009. What makes APIs hard to learn? Answers from developers. IEEE software 26, 6 (2009).
[48]
Martin P Robillard and Robert Deline. 2011. A field study of API learning obstacles. Empirical Software Engineering 16, 6 (2011), 703--732.
[49]
Caitlin Sadowski, Kathryn T Stolee, and Sebastian Elbaum. 2015. How developers search for code: a case study. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 191--201.
[50]
Susan Elliott Sim, Medha Umarji, Sukanya Ratanotayanon, and Cristina V Lopes. 2011. How well do search engines support code retrieval on the web? ACM Transactions on Software Engineering and Methodology (TOSEM) 21, 1 (2011), 4.
[51]
Simone Stumpf, Vidya Rajaram, Lida Li, Margaret Burnett, Thomas Dietterich, Erin Sullivan, Russell Drummond, and Jonathan Herlocker. 2007. Toward harnessing user feedback for machine learning. In Proceedings of the 12th international conference on Intelligent user interfaces. ACM, 82--91.
[52]
Jeffrey Stylos and Steven Clarke. 2007. Usability implications of requiring parameters in objects' constructors. In Proceedings of the 29th international conference on Software Engineering. IEEE Computer Society, 529--539.
[53]
Jeffrey Stylos, Benjamin Graf, Daniela K Busse, Carsten Ziegler, Ralf Ehret, and Jan Karstens. 2008. A case study of API redesign for improved usability. In 2008 IEEE Symposium on Visual Languages and Human-Centric Computing. IEEE, 189--192.
[54]
Jeffrey Stylos and Brad Myers. 2007. Mapping the space of API design decisions. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2007). IEEE, 50--60.
[55]
Jeffrey Stylos and Brad A Myers. 2008. The implications of method placement on API learnability. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. ACM, 105--112.
[56]
Christoph Treude and Martin P Robillard. 2016. Augmenting api documentation with insights from stack overflow. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 392--403.
[57]
Asher Trockman, Shurui Zhou, Christian Kästner, and Bogdan Vasilescu. 2018. Adding sparkle to social coding: an empirical study of repository badges in the npm ecosystem. In Proceedings of the 40th International Conference on Software Engineering. ACM, 511--522.
[58]
Joe Tullio, Anind K Dey, Jason Chalecki, and James Fogarty. 2007. How it works: a field study of non-technical users interacting with an intelligent system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 31--40.
[59]
Marat Valiev, Bogdan Vasilescu, and James Herbsleb. 2018. Ecosystem-level determinants of sustained activity in open-source projects: A case study of the PyPI ecosystem. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 644--655.
[60]
Jue Wang, Yingnong Dang, Hongyu Zhang, Kai Chen, Tao Xie, and Dongmei Zhang. 2013. Mining succinct and high-coverage API usage patterns from source code. In Proceedings of the 10th Working Conference on Mining Software Repositories. IEEE Press, 319--328.
[61]
Tianyi Zhang, Ganesha Upadhyaya, Anastasia Reinhardt, Hridesh Rajan, and Miryung Kim. 2018. Are code examples on an online Q&A forum reliable?: a study of API misuse on stack overflow. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 886--896.
[62]
Tianyi Zhang, Di Yang, Crista Lopes, and Miryung Kim. 2019. Analyzing and supporting adaptation of online code examples. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 316--327.
[63]
Hao Zhong, Tao Xie, Lu Zhang, Jian Pei, and Hong Mei. 2009. MAPO: Mining and recommending API usage patterns. In European Conference on Object-Oriented Programming. Springer, 318--343.

Cited By

View all
  • (2024)XR Composition in the Wild: The Impact of User Environments on Creativity, UX and Flow during Music Production in Augmented RealityProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678314(152-161)Online publication date: 18-Sep-2024
  • (2024)Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud ServicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642721(1-17)Online publication date: 11-May-2024
  • (2024)Scaling Code Pattern Inference with Interactive What-If AnalysisProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639193(1-12)Online publication date: 20-May-2024
  • Show More Cited By

Index Terms

  1. Enabling Data-Driven API Design with Community Usage Data: A Need-Finding Study

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
      April 2020
      10688 pages
      ISBN:9781450367080
      DOI:10.1145/3313831
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 23 April 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. api design
      2. community
      3. information needs
      4. tool support

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      CHI '20
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Upcoming Conference

      CHI 2025
      ACM CHI Conference on Human Factors in Computing Systems
      April 26 - May 1, 2025
      Yokohama , Japan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)199
      • Downloads (Last 6 weeks)29
      Reflects downloads up to 10 Dec 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)XR Composition in the Wild: The Impact of User Environments on Creativity, UX and Flow during Music Production in Augmented RealityProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678314(152-161)Online publication date: 18-Sep-2024
      • (2024)Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud ServicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642721(1-17)Online publication date: 11-May-2024
      • (2024)Scaling Code Pattern Inference with Interactive What-If AnalysisProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639193(1-12)Online publication date: 20-May-2024
      • (2024)Using an LLM to Help With Code UnderstandingProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639187(1-13)Online publication date: 20-May-2024
      • (2024)API usage templates via structural generalizationJournal of Systems and Software10.1016/j.jss.2024.111974210:COnline publication date: 1-Apr-2024
      • (2023)ASMRcade: Interactive Audio Triggers for an Autonomous Sensory Meridian ResponseProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614155(70-78)Online publication date: 9-Oct-2023
      • (2023)Wish You Were Here: Mental and Physiological Effects of Remote Music Collaboration in Mixed RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581162(1-16)Online publication date: 19-Apr-2023
      • (2022)Concept-Annotated Examples for Library ComparisonProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545647(1-16)Online publication date: 29-Oct-2022
      • (2022)Mining the usage of reactive programming APIsProceedings of the 19th International Conference on Mining Software Repositories10.1145/3524842.3527966(203-214)Online publication date: 23-May-2022
      • (2022)BreakBotProceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results10.1145/3510455.3512783(26-30)Online publication date: 21-May-2022
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media