[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3447786.3456255acmconferencesArticle/Chapter ViewAbstractPublication PageseurosysConference Proceedingsconference-collections
research-article

rkt-io: a direct I/O stack for shielded execution

Published: 21 April 2021 Publication History

Abstract

The shielding of applications using trusted execution environments (TEEs) can provide strong security guarantees in untrusted cloud environments. When executing I/O operations, today's shielded execution frameworks, however, exhibit performance and security limitations: they assign resources to the I/O path inefficiently, perform redundant data copies, use untrusted host I/O stacks with security risks and performance overheads. This prevents TEEs from running modern I/O-intensive applications that require high-performance networking and storage.
We describe rkt-io (pronounced "rocket I/O"), a direct user-space network and storage I/O stack specifically designed for TEEs that combines high-performance, POSIX compatibility and security. rkt-io achieves high I/O performance by employing direct userspace I/O libraries (DPDK and SPDK) inside the TEE for kernel-bypass I/O. For efficiency, rkt-io polls for I/O events directly, by interacting with the hardware instead of relying on interrupts, and it avoids data copies by mapping DMA regions in the untrusted host memory. To maintain full Linux ABI compatibility, the userspace I/O libraries are integrated with userspace versions of the Linux VFS and network stacks inside the TEE. Since it omits the host OS from the I/O path, does not suffer from host interface/Iago attacks. Our evaluation with Intel SGX TEEs shows that rkt-io is 9×faster for networking and 7× faster for storage compared to host- (Scone) and LibOS-based (SGX-LKL) I/O approaches.

References

[1]
Nix package manager. https://nixos.org/download.html. Last accessed: March, 2021.
[2]
A. Ahmad, K. Kim, M. I. Sarfaraz, and B. Lee. OBLIVIATE: A data oblivious filesystem for intel SGX. In 25th Annual Network and Distributed System Security Symposium (NDSS), 2018.
[3]
AMD. AMD Secure Encrypted Virtualization (SEV). https://developer.amd.com/sev/. Last accessed: Mar, 2021.
[4]
ARM. Building a secure system using trustzone technology. http://infocenter.arm.com/help/topic/com.arm.doc.prd29-genc-009492c/PRD29-GENC-009492C_trustzone_security_whitepaper.pdf. Last accessed: Mar, 2021.
[5]
S. Arnautov, B. Trach, F. Gregor, T. Knauth, A. Martin, C. Priebe, J. Lind, D. Muthukumaran, D. O'Keeffe, M. L. Stillwell, D. Goltzsche, D. Eyers, R. Kapitza, P. Pietzuch, and C. Fetzer. SCONE: Secure Linux Containers with Intel SGX. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016.
[6]
M. Bailleu, J. Thalheim, P. Bhatotia, C. Fetzer, M. Honda, and K. Vaswani. SPEICHER: Securing lsm-based key-value stores using shielded execution. In 17th USENIX Conference on File and Storage Technologies (FAST), 2019.
[7]
A. Baumann, M. Peinado, and G. Hunt. Shielding Applications from an Untrusted Cloud with Haven. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2014.
[8]
A. Belay, G. Prekas, A. Klimovic, S. Grossman, C. Kozyrakis, and E. Bugnion. IX: A protected dataplane operating system for high throughput and low latency. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2014.
[9]
B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. E. Fiuczynski, D. Becker, C. Chambers, and S. Eggers. Extensibility safety and performance in the spin operating system. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (SOSP), 1995.
[10]
A. Biondo, M. Conti, L. Davi, T. Frassetto, and A.-R. Sadeghi. The guard's dilemma: Efficient code-reuse attacks against intel SGX. In 27th USENIX Security Symposium (USENIX Security 18), 2018.
[11]
BlobFS: Blobstore Filesystem. Last accessed: Mar, 2021.
[12]
F. Brasser, U. Müller, A. Dmitrienko, K. Kostiainen, S. Capkun, and A.-R. Sadeghi. Software grand exposure: SGX cache attacks are practical. In 11th USENIX Workshop on Offensive Technologies (WOOT 17), 2017.
[13]
Bufferbloat project. Last accessed: Mar, 2021.
[14]
D. Cash, P. Grubbs, J. Perry, and T. Ristenpart. Leakage-abuse attacks against searchable encryption. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015.
[15]
S. Checkoway and H. Shacham. Iago attacks: Why the system call api is a bad untrusted rpc interface. In Proceedings of the 18th International Conference on Architectural Support for Programming Languages and Operating Systems, 2013.
[16]
D. R. Cheriton and K. J. Duda. A caching model of operating system kernel functionality. In Proceedings of the 6th Workshop on ACM SIGOPS European Workshop, 1994.
[17]
A. Cloud. Alibaba Cloud's Next-Generation Security Makes Gartner's Report. https://www.alibabacloud.com/blog/alibaba-clouds-next-generation-security-makes-gartners-report_595367. Last accessed: Mar, 2021.
[18]
B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud computing (SoCC), 2010.
[19]
J. Corbet. The multiqueue block layer. https://lwn.net/Articles/552904/, 2013. Last accessed: Mar, 2021.
[20]
V. Costan and S. Devadas. Intel SGX Explained, 2016.
[21]
R. Darawsheh. mbuf external buffer and usage examples. In Proceedings of the DPDK Userspace, Dublin, 2018.
[22]
dm-crypt/device encryption. Last accessed: Mar, 2021.
[23]
J. A. Donenfeld. WireGuard: Next Generation Kernel Network Tunnel. https://www.wireguard.com/papers/wireguard.pdf. Last accessed: Mar, 2021.
[24]
Data plane development kit (DPDK). Last accessed: Mar, 2021.
[25]
E. Dumazet. Busy Polling: Past, Present, Future. In netdev 2.1 Montreal, 2017.
[26]
D. R. Engler, M. F. Kaashoek, and J. O'Toole. Exokernel: An operating system architecture for application-level resource management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (SOSP), 1995.
[27]
Introducing Google Cloud Confidential Computing with Confidential VMs. https://cloud.google.com/blog/products/identity-security/introducing-google-cloud-confidential-computing-with-confidential-vms. Last accessed: Mar, 2021.
[28]
J. Götzfried, M. Eckert, S. Schinzel, and T. Müller. Cache attacks on intel sgx. In Proceedings of the 10th European Workshop on Systems Security, 2017.
[29]
M. Hähnel, W. Cui, and M. Peinado. High-resolution side channels for untrusted operating systems. In Proceedings of the USENIX Annual Technical Conference (ATC), 2017.
[30]
S. Herwig, C. Garman, and D. Levin. Achieving keyless cdns with conclaves. In 29th USENIX Security Symposium (USENIX Security), 2020.
[31]
M. Honda, G. Lettieri, L. Eggert, and D. Santry. PASTE: A network programming interface for non-volatile main memory. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2018.
[32]
Tracking page weight over time. https://discuss.httparchive.org/t/tracking-page-weight-over-time/1049. Last accessed: Mar, 2021.
[33]
J. Hwang, Q. Cai, A. Tang, and R. Agarwal. TCP = RDMA: Cpu-efficient remote storage access with i10. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2020.
[34]
Product page of Intel SSD DC P4600 Series. https://ark.intel.com/content/www/us/en/ark/products/series/96947/intel-ssd-dc-p4600-series.html. Last accessed: Mar, 2021.
[35]
Intel Software Guard Extensions (Intel SGX). https://software.intel.com/en-us/sgx. Last accessed: Mar, 2021.
[36]
Intel SGX SDK. Last accessed: Mar, 2021.
[37]
Intel Storage Performance Development Kit. http://www.spdk.io. Last accessed: Mar, 2021.
[38]
Product page of XL710 network card family. https://www.intel.com/content/www/us/en/products/docs/network-io/ethernet/network-adapters/ethernet-xl710-brief.html. Last accessed: Mar, 2021.
[39]
iPerf - The ultimate speed test tool for TCP, UDP and SCTP. https://iperf.fr/. Last accessed: Mar, 2021.
[40]
Jens Axboe. Flexible I/O Tester. https://github.com/axboe/fio. Last accessed: Dec, 2020.
[41]
E. Y. Jeong, S. Woo, M. Jamshed, H. Jeong, S. Ihm, D. Han, and K. Park. MTCP: A Highly Scalable User-Level TCP Stack for Multicore Systems. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementationi (NSDI), 2014.
[42]
S. Kanev, J. P. Darago, K. Hazelwood, P. Ranganathan, T. Moseley, G.-Y. Wei, and D. Brooks. Profiling a warehouse-scale computer. In Proceedings of the 42nd Annual International Symposium on Computer Architecture(ISCA), 2015.
[43]
A. Kaufmann, T. Stamler, S. Peter, N. K. Sharma, A. Krishnamurthy, and T. Anderson. Tas: Tcp acceleration as an os service. In Proceedings of the Fourteenth EuroSys Conference (EuroSys), 2019.
[44]
A. Klimovic, H. Litz, and C. Kozyrakis. Reflex: Remote flash ≡ local flash. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2017.
[45]
R. Krahn, B. Trach, A. Vahldiek-Oberwagner, T. Knauth, P. Bhatotia, and C. Fetzer. Pesos: Policy enhanced secure object store. In Proceedings of the Thirteenth EuroSys Conference (EuroSys), 2018.
[46]
R. Kunkel, D. L. Quoc, F. Gregor, S. Arnautov, P. Bhatotia, and C. Fetzer. Tensorscone: A secure tensorflow framework using intel SGX. CoRR, 2019.
[47]
D. Kuvaiskii, O. Oleksenko, S. Arnautov, B. Trach, P. Bhatotia, P. Felber, and C. Fetzer. SGXBOUNDS: Memory Safety for Shielded Execution. In Proceedings of the 12th ACM European Conference on Computer Systems (EuroSys), 2017.
[48]
A. Langley. rfc7539: ChaCha20 and Poly1305 for IETF Protocols. https://tools.ietf.org/html/rfc7539. Last accessed: Mar, 2021.
[49]
D. Lee, D. Kohlbrenner, S. Shinde, K. Asanović, and D. Song. Keystone: an open framework for architecting trusted execution environments. In Proceedings of the Fifteenth European Conference on Computer Systems (EuroSys), 2020.
[50]
J. Lee, J. Jang, Y. Jang, N. Kwak, Y. Choi, C. Choi, T. Kim, M. Peinado, and B. B. Kang. Hacking in darkness: Return-oriented programming against secure enclaves. In 26th USENIX Security Symposium (USENIX Security 17), 2017.
[51]
J. Lind, O. Naor, I. Eyal, F. Kelbert, E. G. Sirer, and P. Pietzuch. Teechain: A secure payment network with asynchronous blockchain access. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (SOSP), 2019.
[52]
I. Marinos, R. N. Watson, and M. Handley. Network stack specialization for performance. In Proceedings of the 2014 ACM Conference on SIGCOMM, 2014.
[53]
I. Marinos, R. N. Watson, M. Handley, and R. R. Stewart. Disk|crypt|net: Rethinking the stack for high-performance video streaming. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM), 2017.
[54]
Microsoft Azure. Azure confidential computing. https://azure.microsoft.com/en-us/solutions/confidential-compute/. Last accessed: Mar, 2021.
[55]
musl: an implementation of the C standard library. https://musl.libc.org/. Last accessed: Mar, 2021.
[56]
MySQL. https://www.mysql.com/. Last accessed: Mar, 2021.
[57]
M. Nanavati, J. Wires, and A. Warfield. Decibel: Isolation and sharing in disaggregated rack-scale storage. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017.
[58]
Nginx Web Server. https://www.nginx.com/. Last accessed: Mar, 2021.
[59]
O. Purdila and L. A. Grijincu and N. Tapus. Lkl: The linux kernel library. In 9th RoEduNet IEEE International Conference, 2010.
[60]
Off-CPU Flame Graphs. Last accessed: Mar, 2021.
[61]
O. Oleksenko, D. Kuvaiskii, P. Bhatotia, P. Felber, and C. Fetzer. Intel MPX Explained: A Cross-layer Analysis of the Intel MPX System Stack. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2018.
[62]
M. Orenbach, M. Minkin, P. Lifshits, and M. Silberstein. Eleos: ExitLess OS services for SGX enclaves. In Proceedings of the 12th ACM European ACM Conference in Computer Systems (EuroSys), 2017.
[63]
S. Peter, J. Li, I. Zhang, D. R. K. Ports, D. Woos, A. Krishnamurthy, T. Anderson, and T. Roscoe. Arrakis: The operating system is the control plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2014.
[64]
R. Poddar, C. Lan, R. A. Popa, and S. Ratnasamy. Safebricks: Shielding network functions in the cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2018.
[65]
D. E. Porter, S. Boyd-Wickizer, J. Howell, R. Olinsky, and G. C. Hunt. Rethinking the library os from the top down. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2011.
[66]
C. Priebe, D. Muthukumaran, J. Lind, H. Zhu, S. Cui, V. A. Sartakov, and P. Pietzuch. Sgx-lkl: Securing the host os interface for trusted execution, 2019.
[67]
C. Priebe, K. Vaswani, and M. Costa. EnclaveDB: A Secure Database using SGX (S&P). In IEEE Symposium on Security and Privacy, 2018.
[68]
Redis. https://redis.io/. Last accessed: Mar, 2021.
[69]
RISC-V. Keystone Open-source Secure Hardware Enclave. https://keystone-enclave.org/. Last accessed: Mar, 2021.
[70]
L. Rizzo. Revisiting network I/O APIs: The Netmap Framework. Communications of the ACM, 2012.
[71]
N. Santos, K. P. Gummadi, and R. Rodrigues. Towards Trusted Cloud Computing. In Proceedings of the 1st USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), 2009.
[72]
N. Santos, R. Rodrigues, and B. Ford. Enhancing the os against security threats in system administration. In Proceedings of the 13th International Middleware Conference (Middleware), 2012.
[73]
F. Schuster, M. Costa, C. Gkantsidis, M. Peinado, G. Mainar-ruiz, and M. Russinovich. VC3: Trustworthy Data Analytics in the Cloud using SGX. In Proceedings of the 36th IEEE Symposium on Security and Privacy (Oakland), 2015.
[74]
Scone file protection. https://sconedocs.github.io/SCONE_Fileshield/. Last accessed: Mar, 2021.
[75]
S. Shinde, D. Le Tien, S. Tople, and P. Saxena. PANOPLY: Low-TCB Linux Applications with SGX Enclaves. In Proceedings of the Network and Distributed System Security Symposium (NDSS), 2017.
[76]
L. Soares and M. Stumm. FlexSC: Flexible System Call Scheduling with Exception-less System Calls. In Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2010.
[77]
speedtest. https://www.sqlite.org/speed.html. Last accessed: Mar, 2021.
[78]
SQLite. https://www.sqlite.org/. Last accessed: Mar, 2021.
[79]
Supported Hardware. Last accessed: Feb, 2021.
[80]
sysbench. https://github.com/akopytov/sysbench. Last accessed: Mar, 2021.
[81]
R. Ta-Min, L. Litty, and D. Lie. Splitting interfaces: Making trust between applications and operating systems configurable. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), 2006.
[82]
B. Trach, R. Faqeh, O. Oleksenko, W. Ozga, P. Bhatotia, and C. Fetzer. T-lease: A trusted lease primitive for distributed systems. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC), 2020.
[83]
B. Trach, A. Krohmer, F. Gregor, S. Arnautov, P. Bhatotia, and C. Fetzer. ShieldBox: Secure Middleboxes using Shielded Execution. In Proceedings of the ACM SIGCOMM Symposium on SDN Research (SOSR), 2018.
[84]
B. Trach, O. Oleksenko, F. Gregor, P. Bhatotia, and C. Fetzer. Clemmys: Towards secure remote execution in faas. In Proceedings of the 12th ACM International Conference on Systems and Storage (SYSTOR), 2019.
[85]
C.-C. Tsai, D. E. Porter, and M. Vij. Graphene-SGX: A practical library OS for unmodified applications on SGX. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC), 2017.
[86]
J. Van Bulck, M. Minkin, O. Weisse, D. Genkin, B. Kasikci, F. Piessens, M. Silberstein, T. F. Wenisch, Y. Yarom, and R. Strackx. Foreshadow: Extracting the keys to the Intel SGX kingdom with transient out-of-order execution. In Proceedings of the 27th USENIX Security Symposium (USENIX Security), 2018.
[87]
Vlad Krasnov. How "expensive" is crypto anyway. https://blog.cloudflare.com/how-expensive-is-crypto-anyway, 2017. Last accessed: Mar, 2021.
[88]
Weichbrodt, Nico and Kurmus, Anil and Pietzuch, Peter and Kapitza, Rüdiger. AsyncShock: Exploiting Synchronisation Bugs in Intel SGX Enclaves. In Computer Security - ESORICS, 2016.
[89]
wrk. https://github.com/wg/wrk. Last accessed: Mar, 2021.
[90]
Y. Xu, W. Cui, and M. Peinado. Controlled-channel attacks: Deterministic side channels for untrusted operating systems. In Proceedings of the 36th IEEE Symposium on Security and Privacy (Oakland), 2015.
[91]
K. Yasukata, M. Honda, D. Santry, and L. Eggert. Stackmap: Low-latency networking with the OS stack and dedicated nics. In 2016 USENIX Annual Technical Conference (USENIX ATC), 2016.
[92]
T. Yates. Linux kernel: Introduction of hybrid polling in the blk-mq subsystem. https://lwn.net/Articles/735275, 2017. Last accessed: Mar, 2021.
[93]
I. Zhang, J. Liu, A. Austin, M. L. Roberts, and A. Badam. I'm not dead yet! the role of the operating system in a kernel-bypass era. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS), 2019.
[94]
W. Zheng, A. Dave, J. G. Beekman, R. A. Popa, J. E. Gonzalez, and I. Stoica. Opaque: An Oblivious and Encrypted Distributed Analytics Platform. In Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017.

Cited By

View all
  • (2024)Improving Virtualized I/O Performance by Expanding the Polled I/O Path of LinuxProceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems10.1145/3655038.3665944(31-37)Online publication date: 8-Jul-2024
  • (2024)Conditional Network Availability: Enhancing Connectivity Guarantees for TEE-Based Services2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00030(225-233)Online publication date: 8-Jul-2024
  • (2023)Anchor: A Library for Building Secure Persistent Memory SystemsProceedings of the ACM on Management of Data10.1145/36267181:4(1-31)Online publication date: 12-Dec-2023
  • Show More Cited By

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
EuroSys '21: Proceedings of the Sixteenth European Conference on Computer Systems
April 2021
631 pages
ISBN:9781450383349
DOI:10.1145/3447786
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2021

Permissions

Request permissions for this article.

Check for updates

Badges

Qualifiers

  • Research-article

Funding Sources

  • NCSC/GCHQ

Conference

EuroSys '21
Sponsor:
EuroSys '21: Sixteenth European Conference on Computer Systems
April 26 - 28, 2021
Online Event, United Kingdom

Acceptance Rates

EuroSys '21 Paper Acceptance Rate 38 of 181 submissions, 21%;
Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25
Twentieth European Conference on Computer Systems
March 30 - April 3, 2025
Rotterdam , Netherlands

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)144
  • Downloads (Last 6 weeks)12
Reflects downloads up to 12 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Improving Virtualized I/O Performance by Expanding the Polled I/O Path of LinuxProceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems10.1145/3655038.3665944(31-37)Online publication date: 8-Jul-2024
  • (2024)Conditional Network Availability: Enhancing Connectivity Guarantees for TEE-Based Services2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00030(225-233)Online publication date: 8-Jul-2024
  • (2023)Anchor: A Library for Building Secure Persistent Memory SystemsProceedings of the ACM on Management of Data10.1145/36267181:4(1-31)Online publication date: 12-Dec-2023
  • (2023)GuardBox: A High-Performance Middlebox Providing Confidentiality and Integrity for PacketsIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.326662918(2413-2426)Online publication date: 1-Jan-2023
  • (2023)GreenCoin: A Renewable Energy-Aware Cryptocurrency2023 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E59103.2023.00016(70-80)Online publication date: 25-Sep-2023
  • (2023)A Survey of Trusted Computing Solutions Using FPGAsIEEE Access10.1109/ACCESS.2023.326180211(31583-31593)Online publication date: 2023
  • (2022)Secure and Policy-Compliant Query Processing on Heterogeneous Computational Storage ArchitecturesProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517913(1462-1477)Online publication date: 10-Jun-2022
  • (2022)Treaty: Secure Distributed Transactions2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN53405.2022.00015(14-27)Online publication date: Jun-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media