Agility in Software 2.0 – Notebook Interfaces and MLOps with Buttresses and Rebars

1. Giray, G.: A software engineering perspective on engineering machine learning systems: state of the art and challenges. J. Syst. Softw. 180, 111031 (2021)

   Article  Google Scholar 

2. Ebert, C., Gallardo, G., Hernantes, J., Serrano, N.: DevOps. IEEE Softw. 33(3), 94–100 (2016)

   Article  Google Scholar 

3. Diebold, P., Theobald, S.: How is agile development currently being used in regulated embedded domains? J. Softw. Evol. Process 30(8), e1935 (2018)

   Article  Google Scholar 

4. Hanssen, G.K., Stålhane, T., Myklebust, T.: SafeScrum®-Agile Development of Safety-Critical Software. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99334-8

   Book  Google Scholar 

5. Kim, M., Zimmermann, T., DeLine, R., Begel, A.: The emerging role of data scientists on software development teams. In: Proceedings of the 38th International Conference on Software Engineering, pp. 96–107 (2016)

   Google Scholar 

6. Sletholt, M.T., Hannay, J.E., Pfahl, D., Langtangen, H.P.: What do we know about scientific software development’s agile practices? Comput. Sci. Eng. 14(2), 24–37 (2011)

   Article  Google Scholar 

7. Sculley, D., et al.: Hidden technical debt in machine learning systems. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, pp. 2503–2511 (2015)

   Google Scholar 

8. Amershi, S., et al.: Software engineering for machine learning: a case study. In: Proceedings of the 41st International Conference on Software Engineering, pp. 291–300 (2019)

   Google Scholar 

9. Wan, Z., Xia, X., Lo, D., Murphy, G.C.: How does machine learning change software development practices? IEEE Trans. Software Eng. 47(9), 1857–1871 (2021)

   Google Scholar 

10. Vogelsang, A., Borg, M.: Requirements engineering for machine learning: perspectives from data scientists. In: Proceedings of the 27th International Requirements Engineering Conference Workshops, pp. 245–251 (2019)

    Google Scholar 

11. Chattopadhyay, S., Prasad, I., Henley, A.Z., Sarma, A., Barik, T.: What’s wrong with computational notebooks? Pain points, needs, and design opportunities. In: Human Factors in Computing Systems, pp. 1–12 (2020)

    Google Scholar 

12. Jakobsson, M., Henriksson, J.: Sharing local files with Kubernetes clusters. MSc thesis, Lund University (2021). http://lup.lub.lu.se/student-papers/record/9066685/file/9066686.pdf

13. Knuth, D.E.: Literate programming. Comput. J. 27(2), 97–111 (1984)

    Article  MATH  Google Scholar 

14. Knuth, D.E.: Literate Programming. Center for the Study of Language and Information, Stanford, US (1992)

    Google Scholar 

15. Hannay, J.E., MacLeod, C., Singer, J., Langtangen, H.P., Pfahl, D., Wilson, G.: How do scientists develop and use scientific software? In: Proceedings of the ICSE Workshop on Software Engineering for Computational Science and Engineering, pp. 1–8. IEEE (2009)

    Google Scholar 

16. Vognstrup Fog, B., Nylandsted Klokmose, C.: Mapping the landscape of literate computing. In: Proceedings of the 30th Annual Workshop of the Psychology of Programming Interest Group (2019)

    Google Scholar 

17. Kery, M.B., John, B.E., O’Flaherty, P., Horvath, A., Myers, B.A.: Towards effective foraging by data scientists to find past analysis choices. In: Human Factors in Computing Systems, pp. 1–13 (2019)

    Google Scholar 

18. Kery, M.B., Myers, B.A.: Interactions for untangling messy history in a computational notebook. In: Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 147–155 (2018)

    Google Scholar 

19. Head, A., Hohman, F., Barik, T., Drucker, S.M., DeLine, R.: Managing messes in computational notebooks. In: Human Factors in Computing Systems, pp. 1–12 (2019)

    Google Scholar 

20. Singer, J.: Notes on notebooks: is Jupyter the bringer of jollity? In: Proceedings of the ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, pp. 180–186 (2020)

    Google Scholar 

21. Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers, Burlington (1993)

    Book  MATH  Google Scholar 

22. Kuusela, H., Paul, P.: A comparison of concurrent and retrospective verbal protocol analysis. Am. J. Psychol. 113(3), 387–404 (2000)

    Article  Google Scholar 

23. Bosch, J., Holmström Olsson, H., Crnkovic, I.: Engineering AI systems: a research agenda. In: Artificial Intelligence Paradigms for Smart Cyber-Physical Systems, pp. 1–19. IGI Global (2021)

    Google Scholar 

24. Tapia, P., Palacios, E., Noël, L., et al.: Implementing Operational AI in telecom environments. Tupl White Paper 7 (2018)

    Google Scholar 

25. Fitzgerald, B., Stol, K.J.: Continuous software engineering: a roadmap and agenda. J. Syst. Softw. 123, 176–189 (2017)

    Article  Google Scholar 

26. Jabbari, R., Ali, N., Petersen, K., Tanveer, B.: Towards a benefits dependency network for DevOps based on a systematic literature review. J. Softw. Evol. Process 30(11), e1957 (2018)

    Article  Google Scholar 

27. Treveil, M., et al.: Introducing MLOps. O’Reilly Media Inc., Sebastopol (2020)

    Google Scholar 

28. Hummer, W., et al.: ModelOps: cloud-based lifecycle management for reliable and trusted AI. In: Proceedings of the International Conference on Cloud Engineering, pp. 113–120 (2019)

    Google Scholar 

29. Borg, M., et al.: Safely entering the deep: a review of verification and validation for machine learning and a challenge elicitation in the automotive industry. J. Automot. Softw. Eng. 1(1), 1–19 (2019)

    Article  Google Scholar 

30. Falcini, F., Lami, G., Costanza, A.M.: Deep learning in automotive software. IEEE Softw. 34(3), 56–63 (2017)

    Article  Google Scholar 

31. Jiang, F., et al.: Artificial intelligence in healthcare: past, present and future. Stroke Vasc. Neurol. 2(4), 230–243 (2017)

    Article  Google Scholar 

32. Vidot, G., Gabreau, C., Ober, I., Ober, I.: Certification of embedded systems based on machine learning: a survey. arXiv preprint arXiv:2106.07221 (2021)

33. High-Level Expert Group on Artificial Intelligence: Ethics guidelines for trustworthy artificial intelligence. Technical report, European Commission, Brussels, Belgium (2019)

    Google Scholar 

34. Borg, M., Englund, C., Duran, B.: Traceability and deep learning-safety-critical systems with traces ending in deep neural networks. In: Proceedings of the Grand Challenges of Traceability: The Next Ten Years, pp. 48–49 (2017)

    Google Scholar 

35. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: Proceedings of the 5th International Conference on Data Science and Advanced Analytics, pp. 80–89 (2018)

    Google Scholar 

36. European Commission: Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts, 21 April 2021. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX

37. European Commission: Regulation (EU) 2016/679 of the European Parliament and of the Council on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/ec (General Data Protection Regulation). Off. J. Eur. Union 119, 1–88 (2016)

    Google Scholar 

38. Borg, M., Jabangwe, R., Åberg, S., Ekblom, A., Hedlund, L., Lidfeldt, A.: Test automation with Grad-CAM heatmaps - a future pipe segment in MLOps for vision AI? In: Proceedings of the 14th International Conference on Software Testing, Verification and Validation Workshops, pp. 175–181 (2021)

    Google Scholar 

39. D’Amour, A., et al.: Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 (2020)

40. Ahmad, K., Bano, M., Abdelrazek, M., Arora, C., Grundy, J.: What’s up with requirements engineering for artificial intelligence systems? In: Proceedings of the 29th International Requirements Engineering Conference, pp. 1–12 (2021)

    Google Scholar 

41. Habibullah, K.M., Horkoff, J.: Non-functional requirements for machine learning: understanding current use and challenges in industry. In: Proceedings of the 29th International Requirements Engineering Conference, pp. 13–23 (2021)

    Google Scholar 

42. Siebert, J., et al.: Construction of a quality model for machine learning systems. Softw. Qual. J., 1–29 (2021)

    Google Scholar 

43. Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: survey, landscapes and horizons. IEEE Trans. Softw. Eng. (2020)

    Google Scholar 

44. Riccio, V., Jahangirova, G., Stocco, A., Humbatova, N., Weiss, M., Tonella, P.: Testing machine learning based systems: a systematic mapping. Empir. Softw. Eng. 25(6), 5193–5254 (2020)

    Article  Google Scholar 

45. Xiang, W., et al.: Verification for machine learning, autonomy, and neural networks survey. arXiv preprint arXiv:1810.01989 (2018)

46. Bjarnason, E., et al.: Challenges and practices in aligning requirements with verification and validation: a case study of six companies. Empir. Softw. Eng. 19(6), 1809–1855 (2014)

    Article  Google Scholar 

47. Bjarnason, E.: Integrated Requirements Engineering - Understanding and Bridging Gaps in Software Development. Lund University, Sweden (2013). https://lucris.lub.lu.se/ws/portalfiles/portal/3427902/4117182.pdf

48. Cleland-Huang, J., Marrero, W., Berenbach, B.: Goal-centric traceability: using virtual plumblines to maintain critical systemic qualities. IEEE Trans. Software Eng. 34(5), 685–699 (2008)

    Article  Google Scholar 

Download references