More Web Proxy on the site http://driver.im/

research-article

Exploring compiler optimization space for control flow obfuscation

Authors:

Muhammad Faraz Hyder,

Muhammad Fahim ul Haque,

Paulo Cesar SantosAuthors Info & Claims

Volume 139, Issue C

https://doi.org/10.1016/j.cose.2024.103704

Published: 01 April 2024 Publication History

Abstract

Code obfuscation is a promising technique for securing software and protecting it from adversaries. The objective is to harden the exploitation of security vulnerabilities for the attacker as well as launching of successful attacks. Obfuscation can be classified into layout, data, and control flow obfuscation. Control flow obfuscation impedes the understanding of the application logic by making it complicated to determine the actual control flows. Although numerous control flow methods exist in the literature, the role of existing compiler optimizations has just been discovered. This paper is the first one that explores the existing optimization space of LLVM compiler for obfuscating code. Our techniques optimally explore the native compiler's optimizations to improve the original code performance and reduce memory space with no disruptive efforts, tools, or extra costs. In the CBench benchmark suite, our work is able to improve 246%, 143%, and 468% in cyclomatic complexity, program length, and implementation effort, respectively, compared to unobfuscated code. Therefore, instead of inventing new obfuscation tools, the existing compiler optimizations can easily be used to obfuscate control flows, saving the overall cost and efforts.

References

[1]

H. Ahmed, M.A. Ismail, Reducer: elimination of repetitive codes for accelerated iterative compilation, Comput. Inform. 40 (3) (2021) 543–574.

[2]

H. Ahmed, M.A. Ismail, Toward a novel engine for compiler optimization space exploration of big data workloads, Softw. Pract. Exp. 52 (5) (2022) 1262–1293.

[3]

A.V. Aho, Compilers: Principles, Techniques and Tools (for Anna University), 2 ed., 2003.

[4]

A.H. Ashouri, G. Mariani, G. Palermo, E. Park, J. Cavazos, C. Silvano, Cobayn: compiler autotuning framework using Bayesian networks, ACM Trans. Archit. Code Optim. 13 (2) (2016) 1–25.

[5]

A.H. Ashouri, A. Bignoli, G. Palermo, C. Silvano, S. Kulkarni, J. Cavazos, Micomp: mitigating the compiler phase-ordering problem using optimization sub-sequences and machine learning, ACM Trans. Archit. Code Optim. 14 (3) (2017) 1–28.

[6]

A.H. Ashouri, W. Killian, J. Cavazos, G. Palermo, C. Silvano, A survey on compiler autotuning using machine learning, ACM Comput. Surv. 51 (5) (2018) 96.

[7]

V. Balachandran, S. Emmanuel, Potent and stealthy control flow obfuscation by stack based self-modifying code, IEEE Trans. Inf. Forensics Secur. 8 (4) (2013) 669–681.

[8]

V. Balachandran, D.J. Tan, V.L. Thing, et al., Control flow obfuscation for Android applications, Comput. Secur. 61 (2016) 72–93.

[9]

Banescu, S. (2016): Obfuscation benchmarks. https://github.com/tum-i4/obfuscation-benchmarks/.

[10]

C.K. Behera, D.L. Bhaskari, Different obfuscation techniques for code protection, Proc. Comput. Sci. 70 (2015) 757–763.

[11]

C.-F. Chen, T. Petsios, M. Pomonis, A. Tang, Confuse: Llvm-based code obfuscation, Technical Report, 2013.–15 s 2013.

[12]

C. Collberg, C. Thomborson, D. Low, A taxonomy of obfuscating transformations, Citeseer, 1997.

[13]

K. Cooper, L. Torczon, Engineering a Compiler, Elsevier, 2011.

[14]

K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: Nsga-ii, IEEE Trans. Evol. Comput. 6 (2) (2002) 182–197.

Digital Library

[15]

S.A. Ebad, A.A. Darem, J.H. Abawajy, Measuring software obfuscation quality–a systematic literature review, IEEE Access 9 (2021) 99024–99038.

[16]

Fursin, G. (2010): cBench benchmark. https://sourceforge.net/projects/cbenchmark/files/cBench/V1.1/.

[17]

G. Fursin, O. Temam, Collective optimization: a practical collaborative approach, ACM Trans. Archit. Code Optim. 7 (4) (2010) 20.

[18]

C. Gao, M. Cai, S. Yin, G. Huang, H. Li, W. Yuan, X. Luo, Obfuscation-resilient Android malware analysis based on complementary features, IEEE Trans. Inf. Forensics Secur. (2023).

[19]

J. Garcia, M. Hammad, S. Malek, Lightweight, obfuscation-resilient detection and family identification of Android malware, ACM Trans. Softw. Eng. Methodol. 26 (3) (2018) 1–29.

[20]

M.R. Guthaus, J.S. Ringenberg, D. Ernst, T.M. Austin, T. Mudge, R.B. Brown, Mibench: a free, commercially representative embedded benchmark suite, in: Proceedings of the Fourth Annual IEEE International Workshop on Workload Characterization. WWC-4 (Cat. No. 01EX538), IEEE, 2001, pp. 3–14.

[21]

M.H. Halstead, Elements of Software Science (Operating and Programming Systems Series), Elsevier Science Inc., 1977.

Digital Library

[22]

W.A. Harrison, K.I. Magel, A complexity measure based on nesting level, ACM SIGPLAN Not. 16 (3) (1981) 63–74.

[23]

S. Hosseinzadeh, S. Rauti, S. Laurén, J.-M. Mäkelä, J. Holvitie, S. Hyrynsalmi, V. Leppänen, Diversification and obfuscation techniques for software security: a systematic literature review, Inf. Softw. Technol. 104 (2018) 72–93.

[24]

K. Hoste, L. Eeckhout, Cole: compiler optimization level exploration, in: Proceedings of the 6th Annual IEEE/ACM International Symposium on Code Generation and Optimization, ACM, 2008, pp. 165–174.

[25]

C. Lattner, V. Adve, Llvm: a compilation framework for lifelong program analysis & transformation, in: International Symposium on Code Generation and Optimization, in: CGO 2004, vol. 2004., IEEE, 2004, pp. 75–86.

[26]

A. Majumdar, C. Thomborson, S. Drape, A survey of control-flow obfuscations, in: Information Systems Security: Second International Conference, ICISS 2006, in: Proceedings, vol. 2, Kolkata, India, December 19–21, 2006, 2006, pp. 353–356. A survey of control-flow obfuscations, Springer, 2006.

[27]

T.J. McCabe, A complexity measure, IEEE Trans. Softw. Eng. 4 (1976) 308–320.

Digital Library

[28]

R. Omar, A. El-Mahdy, E. Rohou, Arbitrary control-flow embedding into multiple threads for obfuscation: a preliminary complexity and performance analysis, in: Proceedings of the 2nd International Workshop on Security in Cloud Computing, 2014, pp. 51–58.

[29]

Y. Peng, G. Su, B. Tian, M. Sun, Q. Li, Control flow obfuscation based protection method for Android applications, China Commun. 14 (11) (2017) 247–259.

[30]

S. Schrittwieser, S. Katzenbeisser, Code obfuscation against static and dynamic reverse engineering, in: Information Hiding: 13th International Conference, IH 2011, Prague, Czech Republic, May 18–20, 2011, Springer, 2011, pp. 270–284. Revised Selected Papers 13.

[31]

S. Schrittwieser, S. Katzenbeisser, J. Kinder, G. Merzdovnik, E. Weippl, Protecting software through obfuscation: can it keep pace with progress in code analysis?, ACM Comput. Surv. 49 (1) (2016) 1–37.

[32]

F. Scrinzi, Behavioral analysis of obfuscated code, Master's thesis University of Twente, 2015.

[33]

M.T. Shirazi, Analysis of obfuscation transformations on binary code, PhD thesis Université Grenoble Alpes, 2019.

[34]

M. Styugin, V. Zolotarev, A. Prokhorov, R. Gorbil, New approach to software code diversification in interpreted languages based on the moving target technology, in: 2016 IEEE 10th International Conference on Application of Information and Communication Technologies (AICT), IEEE, 2016, pp. 1–5.

[35]

A.J. Suresh, S. Sankaran, A framework for evaluation of software obfuscation tools for embedded devices, in: International Conference on Applications and Techniques in Information Security, Springer, 2020, pp. 1–13.

[36]

J.C. Torre, J.M. Aragó-Jurado, J. Jareño, S. Varrette, B. Dorronsoro, Obfuscating llvm intermediate representation source code with nsga-ii, in: 15th Intl. Conf. on Computational Intelligence in Security for Information Systems (CISIS'22), Springer, 2022.

[37]

S. Verma, M. Pant, V. Snasel, A comprehensive review on nsga-ii for multi-objective combinatorial optimization problems, IEEE Access 9 (2021) 57757–57791.

[38]

Werkelin Ahlin O., 2013. Implementation and evaluation of some platformindependent obfuscating transformations.

[39]

H. Xu, Y. Zhou, J. Ming, M. Lyu, Layered obfuscation: a taxonomy of software obfuscation techniques for layered security, Cybersecurity 3 (1) (2020) 1–18.

[40]

I. You, K. Yim, Malware obfuscation techniques: a brief survey, in: 2010 International Conference on Broadband, Wireless Computing, Communication and Applications, IEEE, 2010, pp. 297–300.

[41]

Y. Yusoff, M.S. Ngadiman, A.M. Zain, Overview of nsga-ii for optimizing machining process parameters, Proc. Eng. 15 (2011) 3978–3983.

[42]

L. Zhang, H. Meng, V.L. Thing, Progressive control flow obfuscation for Android applications, in: TENCON 2018-2018 IEEE Region 10 Conference, IEEE, 2018, pp. 1075–1079.

Recommendations

Obfuscation: The Hidden Malware

A cyberwar exists between malware writers and antimalware researchers. At this war's heart rages a weapons race that originated in the 80s with the first computer virus. Obfuscation is one of the latest strategies to camouflage the telltale signs of ...
Lightweight Dispatcher Constructions for Control Flow Flattening
SSPREW-7: Proceedings of the 7th Software Security, Protection, and Reverse Engineering / Software Security and Protection Workshop

The objective of control flow obfuscation is to protect the program control flow from analysis. A technique called control flow flattening addresses static analysis by hiding edges between basic blocks in a program and introduces a dispatcher block that ...
Control flow based obfuscation
DRM '05: Proceedings of the 5th ACM workshop on Digital rights management

A software obfuscator is a program O to transform a source program P for protection against malicious reverse engineering. O should be correct (O(P) has same functionality with P), resilient (O(P) is resilient against attacks), and effective (O(P) is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Security

Computers and Security Volume 139, Issue C

Apr 2024

874 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Advanced Technology Publications

United Kingdom

Publication History

Published: 01 April 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents