More Web Proxy on the site http://driver.im/

research-article

Sensemaking Practices in the Everyday Work of AI/ML Software Engineering

Authors:

Christine T. Wolf,

Drew PaineAuthors Info & Claims

ICSEW'20: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops

Pages 86 - 92

https://doi.org/10.1145/3387940.3391496

Published: 25 September 2020 Publication History

Abstract

This paper considers sensemaking as it relates to everyday software engineering (SE) work practices and draws on a multi-year ethnographic study of SE projects at a large, global technology company building digital services infused with artificial intelligence (AI) and machine learning (ML) capabilities. Our findings highlight the breadth of sensemaking practices in AI/ML projects, noting developers' efforts to make sense of AI/ML environments (e.g., algorithms/methods and libraries), of AI/ML model ecosystems (e.g., pre-trained models and "upstream" models), and of business-AI relations (e.g., how the AI/ML service relates to the domain context and business problem at hand). This paper builds on recent scholarship drawing attention to the integral role of sensemaking in everyday SE practices by empirically investigating how and in what ways AI/ML projects present software teams with emergent sensemaking requirements and opportunities.

References

[1]

S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi and T. Zimmermann. 2019. Software Engineering for Machine Learning: A Case Study, in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 291--300.10.1109/ICSE-SEIP.2019.00042

Digital Library

[2]

A. Begel. 2019. Best Practices for Engineering AI-Infused Applications: Lessons Learned from Microsoft Teams, in 2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Enqineerinq Research and Industrial Practice (SER&IP), 1--1. 10.1109/CESSER-IP.2019.00008

[3]

Matthias Book and André van der Hoek. 2018. Sketching with a purpose: moving from supporting modeling to supporting software engineering activities. in Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering, Gothenburg, Sweden, Association for Computing Machinery, 93--96. 10.1145/3195836.3195854

[4]

Kathy Charmaz. 2014. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. Sage.

[5]

Souti Chattopadhyay, Nicholas Nelson, Thien Nam, McKenzie Calvert and Anita Sarma. 2018. Context in programming: an investigation of how programmers create context. in Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering, Gothenburg, Sweden, Association for Computing Machinery, 33--36.10.1145/3195836.3195861

Digital Library

[6]

Marisa Leavitt Cohn. 2016. Convivial Decay: Entangled Lifetimes in a Geriatric Infrastructure. in Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, San Francisco, California, USA, ACM, 1511--1523.10.1145/2818048.2820077

Digital Library

[7]

Kobus Ehlers. 2011. Agile software development as managed sensemaking, Stellenbosch: University of Stellenbosch.

[8]

Rob Fuller. 2019. Functional organization of software groups considered harmful. in Proceedings of the International Conference on Software and System Processes, Montreal, Quebec, Canada, IEEE Press, 120--124. 10.1109/icssp.2019.00024

Digital Library

[9]

Valentina Grigoreanu, Margaret Burnett, Susan Wiedenbeck, Jill Cao, Kyle Rector and Irwin Kwan. 2012. End-user debugging strategies: A sensemaking perspective. 19 (1). Article 5. 10.1145/2147783.2147788

[10]

F. Ishikawa and N. Yoshioka. 2019. How Do Engineers Perceive Difficulties in Engineering of Machine-Learning Systems? - Questionnaire Survey. in 2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial Practice (SER&IP), 2--9. 10.1109/CESSER-IP.2019.00009

[11]

Sami Jantunen, Rex Dumdum and Donald C. Gause. 2019. Towards new requirements engineering competencies. in Proceedings of the 12th International Workshop on Cooperative and Human Aspects of Software Engineering, Montreal, Quebec, Canada, IEEE Press, 131--134. 10.1109/chase.2019.00038

[12]

F. Khomh, B. Adams, J. Cheng, M. Fokaefs and G. Antoniol. 2018. Software Engineering for Machine-Learning Applications: The Road Ahead. IEEE Software, 35 (5). 81--84. 10.1109/MS.2018.3571224

[13]

Charlotte P. Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA): A Conceptual Framework of and for CSCW. in Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada, ACM, 179--194. 10.1145/2675133.2675161

Digital Library

[14]

Lucy Ellen Lwakatare, Aiswarya Raj, Jan Bosch, Helena Holmström Olsson and Ivica Crnkovic. 2019. A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation. in, Cham, Springer International Publishing, 227--243.

[15]

Drew Paine and Charlotte P. Lee. 2020. Coordinative Entities: Forms of Organizing in Data Intensive Science. Computer Supported Cooperative Work (CSCW). 10.1007/s10606-020-09372-2

[16]

Paul Ralph. 2015. Developing and evaluating software engineering process theories. in Proceedings of the 37th International Conference on Software Engineering - Volume 1, Florence, Italy, IEEE Press, 20--31.

[17]

Paul Ralph. 2015. The Sensemaking-Coevolution-Implementation Theory of software design. Science of Computer Programming, 101. 21--41. https://doi.Org/10.1016/j.scico.2014.ll.007

[18]

Paul Ralph and Rahul Mohanani. 2015. Is requirements engineering inherently counterproductive? in Proceedings of the Fifth International Workshop on Twin Peaks of Requirements and Architecture, Florence, Italy, IEEE Press, 20--23.

Digital Library

[19]

Walt Scacchi. 2002. Process Models in Software Engineering. in Marciniak, J.J. ed. Encyclopedia of Software Engineering.

[20]

Todd Sedano, Paul Ralph and Cécile Péraire. 2019. The product backlog. in Proceedings of the 41st International Conference on Software Engineering, Montreal, Quebec, Canada, IEEE Press, 200--211.10.1109/icse.2019.00036

Digital Library

[21]

H. Sharp, Y. Dittrich and C. R. B. de Souza. 2016. The Role of Ethnographic Studies in Empirical Software Engineering. IEEE Transactions on Software Engineering, 42 (8]. 786--804. 10.1109/TSE.2016.2519887

[22]

Ben Shreeve, Paul Ralph, Pete Sawyer and Patrick Stacey. 2015. Geographically distributed sensemaking: developing understanding in forum-based software development teams. in Proceedings of the Eighth International Workshop on Cooperative and Human Aspects of Software Engineering, Florence, Italy, IEEE Press, 36--42.

Digital Library

[23]

H. Washizaki, H. Uchida, F. Khomh and Y. Guéhéneuc. 2019. Studying Software Engineering Patterns for Designing Machine Learning Systems. in 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP), 49--495.10.1109/IWESEP49350.2019.00017

[24]

Karl E. Weick. 2009. Sensemaking in organizations. Sage, Thousand Oaks, CA.

[25]

Christine T. Wolf. 2020. AI Models and Their Worlds: Investigating Data-Driven, AI/ML Ecosystems Through a Work Practices Lens. in, Cham, Springer International Publishing, 651--664.

[26]

Christine T. Wolf and Jeanette L. Blomberg. 2020. Making Sense of Enterprise Apps in Everyday Work Practices. Computer Supported Cooperative Work (CSCW), 29 (1-2). 1--27. 10.1007/s10606-019-09363-y

Cited By

Souza FSouza AAmorim BCordeiro T(2024)Towards Role Definition in Agile AI-based System Development: Perspectives and ReflectionsProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701661(220-230)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3701625.3701661
Dolata MCrowston K(2024)Making Sense of AI Systems DevelopmentIEEE Transactions on Software Engineering10.1109/TSE.2023.333885750:1(123-140)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3338857
Husen JWashizaki HRunpakprakun JYoshioka NTun HFukazawa YTakeuchi H(2024)Integrated multi-view modeling for reliable machine learning-intensive software engineeringSoftware Quality Journal10.1007/s11219-024-09687-z32:3(1239-1285)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09687-z
Show More Cited By

Index Terms

Sensemaking Practices in the Everyday Work of AI/ML Software Engineering

Recommendations

Bridging Work Practice and System Design: Integrating Systemic Analysis, Appreciative Intervention and Practitioner Participation

This article discusses the integration of work practice and system design. By scrutinising the unfolding discourse of workshop participants the co-construction of work practice issues as relevant design considerations is described. Through a mutual ...
Using ethnographic methods in software engineering research
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2

This tutorial provides an overview of the role of ethnography in Software Engineering research. It describes the use of ethnographic methods as a means to provide an in-depth understanding of the socio-technological realities surrounding everyday ...
Wild food practices: understanding the wider implications for design and HCI
UbiComp '13 Adjunct: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication

Ethnographic studies in their many forms have played a major role in informing the design and development of a multitude of systems, from pervasive games to ubiquitous systems that support market traders. This paper presents an alternative response to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSEW'20: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops

June 2020

831 pages

ISBN:9781450379632

DOI:10.1145/3387940

Copyright © 2020 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICSE '20

Sponsor:

SIGSOFT
KIISE

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, Republic of Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
353
Total Downloads

Downloads (Last 12 months)86
Downloads (Last 6 weeks)9

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Souza FSouza AAmorim BCordeiro T(2024)Towards Role Definition in Agile AI-based System Development: Perspectives and ReflectionsProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701661(220-230)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3701625.3701661
Dolata MCrowston K(2024)Making Sense of AI Systems DevelopmentIEEE Transactions on Software Engineering10.1109/TSE.2023.333885750:1(123-140)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3338857
Husen JWashizaki HRunpakprakun JYoshioka NTun HFukazawa YTakeuchi H(2024)Integrated multi-view modeling for reliable machine learning-intensive software engineeringSoftware Quality Journal10.1007/s11219-024-09687-z32:3(1239-1285)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09687-z
Hamer SQuesada-López CJenkins M(2023)Students’ perceptions of integrating a contribution measurement tool in software engineering projects2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET58097.2023.00013(21-30)Online publication date: Aug-2023
https://doi.org/10.1109/CSEET58097.2023.00013
Martínez-Fernández SBogner JFranch XOriol MSiebert JTrendowicz AVollmer AWagner S(2022)Software Engineering for AI-Based Systems: A SurveyACM Transactions on Software Engineering and Methodology10.1145/348704331:2(1-59)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1145/3487043
Giray G(2022)A software engineering perspective on engineering machine learning systemsJournal of Systems and Software10.1016/j.jss.2021.111031180:COnline publication date: 22-Apr-2022
https://dl.acm.org/doi/10.1016/j.jss.2021.111031
Sirén‐Heikel SKjellman MLindén C(2022) At the crossroads of logics: Automating newswork with artificial intelligence — (Re)defining journalistic logics from the perspective of technologists Journal of the Association for Information Science and Technology10.1002/asi.2465674:3(354-366)Online publication date: 20-May-2022
https://doi.org/10.1002/asi.24656
Garg SPundir PRathee GGupta PGarg SAhlawat S(2021)On Continuous Integration / Continuous Delivery for Automated Deployment of Machine Learning Models using MLOps2021 IEEE Fourth International Conference on Artificial Intelligence and Knowledge Engineering (AIKE)10.1109/AIKE52691.2021.00010(25-28)Online publication date: Dec-2021
https://doi.org/10.1109/AIKE52691.2021.00010

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents