More Web Proxy on the site http://driver.im/

research-article

Embedded-check a Code Quality Tool for Automatic Firmware Verification

Authors:

Rafael Corsi Ferrão,

Igor dos Santos Montagner,

Rodolfo AzevedoAuthors Info & Claims

ITiCSE 2024: Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1

Pages 66 - 72

https://doi.org/10.1145/3649217.3653577

Published: 03 July 2024 Publication History

Abstract

Developing embedded microcontroller code is a complex task, especially for undergrad students new to this area. These students often make high-level conceptual mistakes beyond the scope of commercial standards like MISRA-C. These conceptual errors need to be checked manually through code feedback, a process that is time-consuming, error-prone, and does not scale well with an increasing number of students and/or assignments. In this paper, we present an embedded-check an automated tool that can detect common and critical errors students make when learning to code firmware. A set of 13 rules (baremetal and FreeRTOS) was devised based on our experience from several years of teaching Embedded systems. To validate our tool, we compared its results with manual code review of N=99 projects from the last 3 course offerings. We furthered our analysis by running our tool on N=1132 coding lab submissions that did not receive manual feedback and were used as part of classroom activities. We found that the top-3 errors flagged in the projects were already present when students completed the lab activities. We found that (i) our tool also identified all issues discovered during manual code feedback, (ii) our tool detected issues in 86% of student submissions, whereas manual code feedback only flagged 28% of the submissions as problematic, and (iii) 94.3% of students made some code quality error on individual assignments. Within this results, we believe that our tool can have a significant impact when used both as an formative assessment tool to support learning and as a learning analytics tool to improve teaching.

References

[1]

Sara Abbaspour Asadollah, Daniel Sundmark, Sigrid Eldh, and Hans Hansson. 2018. A runtime verification tool for detecting concurrency bugs in freertos embedded software. In 2018 17th International Symposium on Parallel and Distributed Computing (ISPDC). IEEE, 172--179.

[2]

Roberto Bagnara, Michael Barr, and Patricia M. Hill. 2020. BARR-C:2018 and MISRA C:2012: Synergy Between the Two Most Widely Used C Coding Standards. https://doi.org/10.48550/arXiv.2003.06893 arxiv: 2003.06893 [cs]

[3]

Michael Barr. 2009. Embedded C Coding Standard. Netrino.

[4]

Gabriele Bavota and Barbara Russo. 2015. Four eyes are better than two: On the impact of code reviews on software quality. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 81--90.

Digital Library

[5]

International Electrotechnical Commission et al. 2010. IEC 61508: 2010-functional safety of electrical/electronical/programmable electronic safety-related systems. (2010).

[6]

David M Cummings. 2016. Embedded software under the courtroom microscope: A case study of the Toyota unintended acceleration trial. IEEE Technology and Society Magazine, Vol. 35, 4 (2016), 76--84.

[7]

Tomche Delev and Dejan Gjorgjevikj. 2017. Static analysis of source code written by novice programmers. In 2017 IEEE Global Engineering Education Conference (EDUCON). IEEE, 825--830.

[8]

Stephen H Edwards, Nischel Kandru, and Mukund BM Rajagopal. 2017. Investigating static analysis errors in student Java programs. In Proceedings of the 2017 ACM Conference on International Computing Education Research. 65--73.

Digital Library

[9]

Anum Fatima, Shazia Bibi, and Rida Hanif. 2018. Comparative study on static code analysis tools for c/c. In 2018 15th International Bhurban Conference on Applied Sciences and Technology (IBCAST). IEEE, 465--469.

[10]

Daniel Feitosa, Apostolos Ampatzoglou, Paris Avgeriou, and Elisa Yumi Nakagawa. 2015. Investigating Quality Trade-offs in Open Source Critical Embedded Systems. In Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software Architectures (New York, NY, USA, 2015-05-04) (QoSA '15). Association for Computing Machinery, 113--122. https://doi.org/10.1145/2737182.2737190

Digital Library

[11]

Rafael Corsi Ferrao, Igor Dos Santos Montagner, Ricardo Caceffo, and Rodolfo Azevedo. 2022. How much C can students learn in one week? Experiences teaching C in advanced CS courses. In 2022 IEEE Frontiers in Education Conference (FIE). IEEE, 1--8.

[12]

Les Hatton. 2004. Safer language subsets: an overview and a case history, MISRA C., Vol. 46, 7 (2004), 465--472. https://doi.org/10.1016/j.infsof.2003.09.016

[13]

Nannan He and Han-Way Huang. 2014. Use of freeRTOS in teaching real-time embedded systems design course. In 2014 ASEE Annual Conference & Exposition. 24--1307.

[14]

Center for Devices and Radiological Health. 2020. General Principles of Software Validation. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/general-principles-software-validation Publisher: FDA.

[15]

ISO. 2018. ISO 26262--1:2018. https://www.iso.org/standard/68383.html

[16]

JC Jensen, EA Lee, and SA Seshia. 2012. Teaching Embedded Systems the Berkeley Way. In Workshop on Embedded Systems Education (in conjunction with ESWeek), Tampere, Finland.

[17]

Bjarne Johansson, Alessandro V Papadopoulos, and Thomas Nolte. 2019. Concurrency defect localization in embedded systems using static code analysis: An evaluation. In 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 7--12.

[18]

Diana Kirk, Tyne Crow, Andrew Luxton-Reilly, and Ewan Tempero. 2020. On assuring learning about code quality. In Proceedings of the Twenty-Second Australasian Computing Education Conference. 86--94.

Digital Library

[19]

Oleksii Kononenko, Olga Baysal, Latifa Guerrouj, Yaxin Cao, and Michael W Godfrey. 2015. Investigating code review quality: Do people and participation matter?. In 2015 IEEE international conference on software maintenance and evolution (ICSME). IEEE, 111--120.

Digital Library

[20]

Akash Kumar, Shakith Fernando, and Rajesh C Panicker. 2013. Project-Based Learning in Embedded Systems Education Using an FPGA Platform. IEEE Transactions on Education, Vol. 56, 4 (2013), 407--415. https://doi.org/10.1109/TE.2013.2246568

Digital Library

[21]

Alois Mayr, Reinhold Plösch, Michael Kläs, Constanza Lampasona, and Matthias Saft. 2012. A comprehensive code-based quality model for embedded systems: systematic development and validation by industrial projects. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. IEEE, 281--290.

Digital Library

[22]

Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering, Vol. 21 (2016), 2146--2189.

Digital Library

[23]

MISRA. 2023. Misra C:2023: Guidelines for the use of the C language in critical systems 3rd, 2nd revision ed.). https://misra.org.uk

[24]

Arthur-Jozsef Molnar, Simona Motogna, and Cristina Vlad. 2020. Using static analysis tools to assist student project evaluation. In Proceedings of the 2nd ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence. 7--12.

Digital Library

[25]

Igor S Montagner, Rafael Corsi Ferr ao, Eduardo Marossi, and Fábio J Ayres. 2019. Teaching C programming in context: a joint effort between the Computer Systems, Embedded Computing and Programming Challenges courses. In 2019 IEEE Frontiers in Education Conference (FIE). IEEE, 1--9.

Digital Library

[26]

Simona Motogna, Andreea Vescan, and Camelia ?erban. 2023. Empirical investigation in embedded systems: Quality attributes in general, maintainability in particular., Vol. 201 (2023), 111678. https://doi.org/10.1016/j.jss.2023.111678

Digital Library

[27]

Lazaros Papadopoulos, Charalampos Marantos, Georgios Digkas, Apostolos Ampatzoglou, Alexander Chatzigeorgiou, and Dimitrios Soudris. 2018. Interrelations between Software Quality Metrics, Performance and Energy Consumption in Embedded Applications. In Proceedings of the 21st International Workshop on Software and Compilers for Embedded Systems (New York, NY, USA, 2018-05--28) (SCOPES '18). Association for Computing Machinery, 62--65. https://doi.org/10.1145/3207719.3207736

Digital Library

[28]

Sudeep Pasricha. 2022. Embedded systems education in the 2020s: Challenges, reflections, and future directions. In Proceedings of the Great Lakes Symposium on VLSI 2022. 519--524.

[29]

Caitlin Sadowski, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. Modern code review: a case study at google. In Proceedings of the 40th international conference on software engineering: Software engineering in practice. 181--190.

Digital Library

[30]

RTCA SC-205. 2011. DO-178C - Software Considerations in Airborne Systems and Equipment Certification. https://my.rtca.org/productdetails?id=a1B36000001IcmqEAC

[31]

Philipp Dominik Schubert, Paul Gazzillo, Zach Patterson, Julian Braha, Fabian Schiebel, Ben Hermann, Shiyi Wei, and Eric Bodden. 2022. Static data-flow analysis for software product lines in C: Revoking the preprocessor's special role. Automated Software Engineering, Vol. 29, 1 (2022), 35.

Digital Library

[32]

Junji Shimagaki, Yasutaka Kamei, Shane McIntosh, Ahmed E Hassan, and Naoyasu Ubayashi. 2016. A study of the quality-impacting practices of modern code review at sony mobile. In Proceedings of the 38th International Conference on Software Engineering Companion. 212--221.

Digital Library

[33]

André Sanches Fonseca Sobrinho. 2020. An Embedded Systems Remote Course. Journal of Online Engineering Education, Vol. 11, 2 (2020), 01--07.

[34]

Christopher Thompson and David Wagner. 2017. A large-scale study of modern code review and security in open source projects. In Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering. 83--92.

Digital Library

[35]

Antonio Vetro, Luca Ardito, Giuseppe Procaccianti, Maurizio Morisio, and others. 2013. Definition, implementation and validation of energy code smells: an exploratory study on an embedded system. In Proceedings of ENERGY 2013: The Third International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies. 34--39.

[36]

Jiang Zheng, Laurie Williams, Nachiappan Nagappan, Will Snipes, John P Hudepohl, and Mladen A Vouk. 2006. On the value of static analysis for fault detection in software. IEEE transactions on software engineering, Vol. 32, 4 (2006), 240--253.

Digital Library

Cited By

Corsi Ferrao R(2024)Teaching Code Quality In Embedded Systems (Doctoral Consortium)Proceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699579(1-2)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699579

Index Terms

Embedded-check a Code Quality Tool for Automatic Firmware Verification

Recommendations

Debugging protocol for remote cross development environment
RTCSA '00: Proceedings of the Seventh International Conference on Real-Time Systems and Applications

To develop the applications for Internet appliances, embedded software development tools need various host tools such as cross compilers and binary utilities as well as a remote debugger, an interactive shell, and real time resource monitors. The heavy ...
Teaching Code Quality In Embedded Systems (Doctoral Consortium)
Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research
The teaching of code quality is a prominent topic among CSed researchers; however, most studies focus on elementary courses, with very few exploring the subject from the perspective of embedded systems education. This topic is of real interest because ...
A Survey of WCET Analysis of Real-Time Operating Systems
ICESS '09: Proceedings of the 2009 International Conference on Embedded Software and Systems

Timing correctness of hard real-time systems is guaranteed by schedulability analysis and worst-case execution time (WCET) analysis of programs. Traditional WCET analysis mainly deals with application programs and has achieved success in industry. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE 2024: Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1

July 2024

776 pages

ISBN:9798400706004

DOI:10.1145/3649217

General Chairs:
Mattia Monga
University of Milan, Italy
,
Violetta Lonati
University of Milan, Italy
,
Erik Barendsen
Radboud University, The Netherlands
,
Program Chairs:
Judithe Sheard
Monash University, Australia
,
James Paterson
Glasgow Caledonian University, Scotland

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE 2024

Sponsor:

SIGCSE

ITiCSE 2024: Innovation and Technology in Computer Science Education

July 8 - 10, 2024

Milan, Italy

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
35
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)6

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Corsi Ferrao R(2024)Teaching Code Quality In Embedded Systems (Doctoral Consortium)Proceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699579(1-2)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699579

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents