More Web Proxy on the site http://driver.im/

research-article

Open access

Harnessing Large Language Models for Simulink Toolchain Testing and Developing Diverse Open-Source Corpora of Simulink Models for Metric and Evolution Analysis

Author:

Sohil Lal ShresthaAuthors Info & Claims

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 1541 - 1545

https://doi.org/10.1145/3597926.3605233

Published: 13 July 2023 Publication History

Abstract

MATLAB/Simulink is a de-facto standard tool in several safety-critical industries such as automotive, aerospace, healthcare, and industrial automation for system modeling and analysis, compiling models to code, and deploying code to embedded hardware. On one hand, testing cyber-physical system (CPS) development tools such as MathWorks’ Simulink is important as a bug in the toolchain may propagate to the artifacts they produce. On the other hand, it is equally important to understand modeling practices and model evolution to support engineers and scientists as they are widely used in design, simulation, and verification of CPS models. Existing work in this area is limited by two main factors, i.e., (1) inefficiencies of state-of-the-art testing schemes in finding critical tool-chain bugs and (2) the lack of a reusable corpus of public Simulink models. In my thesis, I propose to (1) curate a large reusable corpus of Simulink models to help understand modeling practices and model evolution and (2) leverage such a corpus with deep-learning based language models to test the toolchain.

References

[1]

Vincent Bertram, Shahar Maoz, Jan Oliver Ringert, Bernhard Rumpe, and Michael von Wenckstern. 2017. Component and Connector Views in Practice: An Experience Report. In MODELS. IEEE Computer Society, 167–177.

[2]

Alexander Boll, Florian Brokhausen, Tiago Amorim, Timo Kehrer, and Andreas Vogelsang. 2021. Characteristics, potentials, and limitations of open-source Simulink projects for empirical research. SoSyM, 1–20. https://doi.org/10.1007/s10270-021-00883-0

Digital Library

[3]

Yang Chen, Alex Groce, Chaoqiang Zhang, Weng-Keen Wong, Xiaoli Z. Fern, Eric Eide, and John Regehr. 2013. Taming compiler fuzzers. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, Seattle, WA, USA, June 16-19, 2013. 197–208. https://doi.org/10.1145/2491956.2462173

Digital Library

[4]

Shafiul Azam Chowdhury, Soumik Mohian, Sidharth Mehra, Siddhant Gawsane, Taylor T. Johnson, and Christoph Csallner. 2018. Automatically finding bugs in a commercial cyber-physical system development tool chain with SLforge. In Proc. 40th ACM/IEEE International Conference on Software Engineering (ICSE). ACM.

Digital Library

[5]

Shafiul Azam Chowdhury, Soumik Mohian, Sidharth Mehra, Siddhant Gawsane, Taylor T. Johnson, and Christoph Csallner. 2018. Automatically finding bugs in a commercial cyber-physical system development tool chain with SLforge. In ICSE. ACM, 981–992. https://doi.org/10.1145/3180155.3180231

Digital Library

[6]

Shafiul Azam Chowdhury, Sohil Lal Shrestha, Taylor T. Johnson, and Christoph Csallner. 2020. SLEMI: Equivalence modulo input (EMI) based mutation of CPS models for finding compiler bugs in Simulink. In ICSE. ACM, 335–346. https://doi.org/10.1145/3377811.3380381

Digital Library

[7]

Shafiul Azam Chowdhury, Sohil Lal Shrestha, Taylor T. Johnson, and Christoph Csallner. 2020. SLEMI: Finding Simulink Compiler Bugs through Equivalence Modulo Input (EMI). In Proc. 42nd International Conference on Software Engineering (ICSE), Companion Volume. ACM, 1–4. https://doi.org/10.1145/3377812.3382147

Digital Library

[8]

Shafiul Azam Chowdhury, Lina Sera Varghese, Soumik Mohian, Taylor T. Johnson, and Christoph Csallner. 2018. A curated corpus of Simulink models for model-based empirical studies. In SEsCPS. ACM, 45–48. https://doi.org/10.1145/3196478.3196484

Digital Library

[9]

Zhenying Jiang, Xiao Wu, Zeqian Dong, and Ming Mu. 2017. Optimal Test Case Generation for Simulink Models Using Slicing. In QRS-C. 363–369. https://doi.org/10.1109/QRS-C.2017.67

[10]

Reza Matinnejad, Shiva Nejati, Lionel C. Briand, and Thomas Bruckmann. 2015. Effective test suites for mixed discrete-continuous stateflow controllers. In ESEC/FSE. ACM, 84–95. https://doi.org/10.1145/2786805.2786818

Digital Library

[11]

A. Chakrapani Rao, A. Raouf, Gunwant Dhadyalla, and V. Pasupuleti. 2017. Mutation Testing Based Evaluation of Formal Verification Tools. In DSA. IEEE, 1–7. https://doi.org/10.1109/DSA.2017.10

[12]

Sohil Lal Shrestha. 2020. Automatic generation of Simulink models to find bugs in a cyber-physical system tool chain using deep learning. In Proc. 42nd International Conference on Software Engineering (ICSE), Companion Volume. ACM, 110–112. https://doi.org/10.1145/3377812.3382163

Digital Library

[13]

Sohil Lal Shrestha, Shafiul Azam Chowdhury, and Christoph Csallner. 2020. DeepFuzzSL: Generating models with deep learning to find bugs in the Simulink toolchain. In DeepTest 2020. ACM.

[14]

Sohil Lal Shrestha, Shafiul Azam Chowdhury, and Christoph Csallner. 2022. SLNET: A Redistributable Corpus of 3rd-party Simulink Models. In 19th IEEE/ACM International Conference on Mining Software Repositories, MSR 2022, Pittsburgh, PA, USA, May 23-24, 2022. ACM, 1–5. https://doi.org/10.1145/3524842.3528001

Digital Library

[15]

Sohil Lal Shrestha and Christoph Csallner. 2021. SLGPT: Using Transfer Learning to Directly Generate Simulink Model Files and Find Bugs in the Simulink Toolchain. In EASE 2021: Evaluation and Assessment in Software Engineering, Trondheim, Norway, June 21-24, 2021. ACM, 260–265. https://doi.org/10.1145/3463274.3463806

Digital Library

[16]

Sohil Lal Shrestha, Saroj Panda, and Christoph Csallner. 2018. Complementing Machine Learning Classifiers via Dynamic Symbolic Execution: "Human vs. Bot Generated" Tweets. In 6th IEEE/ACM International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, RAISE@ICSE 2018, Gothenburg, Sweden, May 27, 2018, Walter F. Tichy and Leandro L. Minku (Eds.). ACM, 15–20. https://doi.org/10.1145/3194104.3194111

Digital Library

Cited By

Di Rocco JDi Ruscio DDi Sipio CNguyen PRubei R(2025)On the use of large language models in model-driven engineeringSoftware and Systems Modeling10.1007/s10270-025-01263-8Online publication date: 31-Jan-2025
https://doi.org/10.1007/s10270-025-01263-8
Shrestha SBoll AKehrer TCsallner C(2023)ScoutSL: An Open-Source Simulink Search Engine2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00022(70-74)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00022
Xiao DLiu ZWang SBissyandé TKlein JBird CSarro F(2023)PhyFu: Fuzzing Modern Physics Simulation EnginesProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00054(1579-1591)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1109/ASE56229.2023.00054

Index Terms

Harnessing Large Language Models for Simulink Toolchain Testing and Developing Diverse Open-Source Corpora of Simulink Models for Metric and Evolution Analysis

Recommendations

SLGPT: Using Transfer Learning to Directly Generate Simulink Model Files and Find Bugs in the Simulink Toolchain
EASE '21: Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering

Finding bugs in a commercial cyber-physical system (CPS) development tool such as Simulink is hard as its codebase contains millions of lines of code and complete formal language specifications are not available. While deep learning techniques promise ...
A Simulink Implementation of the Cerebellum Model Articulation Controller
ACT '10: Proceedings of the 2010 Second International Conference on Advances in Computing, Control, and Telecommunication Technologies

This paper presents a Simulink implementation of the CMAC controller which was proposed by J.S. Albus. The controller was implemented using the MatlAB S-function so that it can be used by a control engineer working on the Simulink platform to study the ...
Pattern Language for the Modeling Practices of MATLAB and Simulink
PLoP '23: Proceedings of the 30th Conference on Pattern Languages of Programs

Model-based development (MBD) is a powerful method for designing control algorithms. MATLAB and Simulink are adopted as a standard tool for MBD. However, it could be an obstacle especially for beginners to handle the tools, which could make it difficult ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2023

1554 pages

ISBN:9798400702211

DOI:10.1145/3597926

General Chair:
René Just
University of Washington, USA
,
Program Chair:
Gordon Fraser
University of Passau, Germany

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution 4.0 International License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ISSTA '23

Sponsor:

SIGSOFT

ISSTA '23: 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 17 - 21, 2023

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
541
Total Downloads

Downloads (Last 12 months)319
Downloads (Last 6 weeks)39

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Di Rocco JDi Ruscio DDi Sipio CNguyen PRubei R(2025)On the use of large language models in model-driven engineeringSoftware and Systems Modeling10.1007/s10270-025-01263-8Online publication date: 31-Jan-2025
https://doi.org/10.1007/s10270-025-01263-8
Shrestha SBoll AKehrer TCsallner C(2023)ScoutSL: An Open-Source Simulink Search Engine2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00022(70-74)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00022
Xiao DLiu ZWang SBissyandé TKlein JBird CSarro F(2023)PhyFu: Fuzzing Modern Physics Simulation EnginesProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00054(1579-1591)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1109/ASE56229.2023.00054

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten