[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3597926.3605233acmconferencesArticle/Chapter ViewAbstractPublication PagesisstaConference Proceedingsconference-collections
research-article
Open access

Harnessing Large Language Models for Simulink Toolchain Testing and Developing Diverse Open-Source Corpora of Simulink Models for Metric and Evolution Analysis

Published: 13 July 2023 Publication History

Abstract

MATLAB/Simulink is a de-facto standard tool in several safety-critical industries such as automotive, aerospace, healthcare, and industrial automation for system modeling and analysis, compiling models to code, and deploying code to embedded hardware. On one hand, testing cyber-physical system (CPS) development tools such as MathWorks’ Simulink is important as a bug in the toolchain may propagate to the artifacts they produce. On the other hand, it is equally important to understand modeling practices and model evolution to support engineers and scientists as they are widely used in design, simulation, and verification of CPS models. Existing work in this area is limited by two main factors, i.e., (1) inefficiencies of state-of-the-art testing schemes in finding critical tool-chain bugs and (2) the lack of a reusable corpus of public Simulink models. In my thesis, I propose to (1) curate a large reusable corpus of Simulink models to help understand modeling practices and model evolution and (2) leverage such a corpus with deep-learning based language models to test the toolchain.

References

[1]
Vincent Bertram, Shahar Maoz, Jan Oliver Ringert, Bernhard Rumpe, and Michael von Wenckstern. 2017. Component and Connector Views in Practice: An Experience Report. In MODELS. IEEE Computer Society, 167–177.
[2]
Alexander Boll, Florian Brokhausen, Tiago Amorim, Timo Kehrer, and Andreas Vogelsang. 2021. Characteristics, potentials, and limitations of open-source Simulink projects for empirical research. SoSyM, 1–20. https://doi.org/10.1007/s10270-021-00883-0
[3]
Yang Chen, Alex Groce, Chaoqiang Zhang, Weng-Keen Wong, Xiaoli Z. Fern, Eric Eide, and John Regehr. 2013. Taming compiler fuzzers. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, Seattle, WA, USA, June 16-19, 2013. 197–208. https://doi.org/10.1145/2491956.2462173
[4]
Shafiul Azam Chowdhury, Soumik Mohian, Sidharth Mehra, Siddhant Gawsane, Taylor T. Johnson, and Christoph Csallner. 2018. Automatically finding bugs in a commercial cyber-physical system development tool chain with SLforge. In Proc. 40th ACM/IEEE International Conference on Software Engineering (ICSE). ACM.
[5]
Shafiul Azam Chowdhury, Soumik Mohian, Sidharth Mehra, Siddhant Gawsane, Taylor T. Johnson, and Christoph Csallner. 2018. Automatically finding bugs in a commercial cyber-physical system development tool chain with SLforge. In ICSE. ACM, 981–992. https://doi.org/10.1145/3180155.3180231
[6]
Shafiul Azam Chowdhury, Sohil Lal Shrestha, Taylor T. Johnson, and Christoph Csallner. 2020. SLEMI: Equivalence modulo input (EMI) based mutation of CPS models for finding compiler bugs in Simulink. In ICSE. ACM, 335–346. https://doi.org/10.1145/3377811.3380381
[7]
Shafiul Azam Chowdhury, Sohil Lal Shrestha, Taylor T. Johnson, and Christoph Csallner. 2020. SLEMI: Finding Simulink Compiler Bugs through Equivalence Modulo Input (EMI). In Proc. 42nd International Conference on Software Engineering (ICSE), Companion Volume. ACM, 1–4. https://doi.org/10.1145/3377812.3382147
[8]
Shafiul Azam Chowdhury, Lina Sera Varghese, Soumik Mohian, Taylor T. Johnson, and Christoph Csallner. 2018. A curated corpus of Simulink models for model-based empirical studies. In SEsCPS. ACM, 45–48. https://doi.org/10.1145/3196478.3196484
[9]
Zhenying Jiang, Xiao Wu, Zeqian Dong, and Ming Mu. 2017. Optimal Test Case Generation for Simulink Models Using Slicing. In QRS-C. 363–369. https://doi.org/10.1109/QRS-C.2017.67
[10]
Reza Matinnejad, Shiva Nejati, Lionel C. Briand, and Thomas Bruckmann. 2015. Effective test suites for mixed discrete-continuous stateflow controllers. In ESEC/FSE. ACM, 84–95. https://doi.org/10.1145/2786805.2786818
[11]
A. Chakrapani Rao, A. Raouf, Gunwant Dhadyalla, and V. Pasupuleti. 2017. Mutation Testing Based Evaluation of Formal Verification Tools. In DSA. IEEE, 1–7. https://doi.org/10.1109/DSA.2017.10
[12]
Sohil Lal Shrestha. 2020. Automatic generation of Simulink models to find bugs in a cyber-physical system tool chain using deep learning. In Proc. 42nd International Conference on Software Engineering (ICSE), Companion Volume. ACM, 110–112. https://doi.org/10.1145/3377812.3382163
[13]
Sohil Lal Shrestha, Shafiul Azam Chowdhury, and Christoph Csallner. 2020. DeepFuzzSL: Generating models with deep learning to find bugs in the Simulink toolchain. In DeepTest 2020. ACM.
[14]
Sohil Lal Shrestha, Shafiul Azam Chowdhury, and Christoph Csallner. 2022. SLNET: A Redistributable Corpus of 3rd-party Simulink Models. In 19th IEEE/ACM International Conference on Mining Software Repositories, MSR 2022, Pittsburgh, PA, USA, May 23-24, 2022. ACM, 1–5. https://doi.org/10.1145/3524842.3528001
[15]
Sohil Lal Shrestha and Christoph Csallner. 2021. SLGPT: Using Transfer Learning to Directly Generate Simulink Model Files and Find Bugs in the Simulink Toolchain. In EASE 2021: Evaluation and Assessment in Software Engineering, Trondheim, Norway, June 21-24, 2021. ACM, 260–265. https://doi.org/10.1145/3463274.3463806
[16]
Sohil Lal Shrestha, Saroj Panda, and Christoph Csallner. 2018. Complementing Machine Learning Classifiers via Dynamic Symbolic Execution: "Human vs. Bot Generated" Tweets. In 6th IEEE/ACM International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, RAISE@ICSE 2018, Gothenburg, Sweden, May 27, 2018, Walter F. Tichy and Leandro L. Minku (Eds.). ACM, 15–20. https://doi.org/10.1145/3194104.3194111

Cited By

View all
  • (2023)ScoutSL: An Open-Source Simulink Search Engine2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00022(70-74)Online publication date: 1-Oct-2023
  • (2023)PhyFu: Fuzzing Modern Physics Simulation EnginesProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00054(1579-1591)Online publication date: 11-Nov-2023

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
July 2023
1554 pages
ISBN:9798400702211
DOI:10.1145/3597926
This work is licensed under a Creative Commons Attribution 4.0 International License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Cyber-physical system development
  2. GPT-2
  3. Simulink
  4. deep learning
  5. mining software repositories
  6. model evolution
  7. open-source
  8. programming language modeling
  9. tool chain bugs

Qualifiers

  • Research-article

Funding Sources

Conference

ISSTA '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)305
  • Downloads (Last 6 weeks)29
Reflects downloads up to 01 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)ScoutSL: An Open-Source Simulink Search Engine2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00022(70-74)Online publication date: 1-Oct-2023
  • (2023)PhyFu: Fuzzing Modern Physics Simulation EnginesProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00054(1579-1591)Online publication date: 11-Nov-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media