More Web Proxy on the site http://driver.im/

research-article

Open access

Performance Characterization of Modern Storage Stacks: POSIX I/O, libaio, SPDK, and io_uring

Authors:

Animesh TrivediAuthors Info & Claims

CHEOPS '23: Proceedings of the 3rd Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems

Pages 35 - 45

https://doi.org/10.1145/3578353.3589545

Published: 08 May 2023 Publication History

Abstract

Linux storage stack offers a variety of storage I/O stacks and APIs such as POSIX I/O, asynchronous I/O (libaio), high-performance asynchronous I/O (emerging io_uring) or SPDK, the last of which completely bypasses the kernel. Despite their availability, there has not been a systematic study of their performance and overheads. In order to aid our understanding, in this work we systematically characterize performance, scalability and microarchitectural properties of popular Linux I/O APIs on high-performance storage hardware (Intel Optane SSDs). Our characterization reveals that: (1) at low I/O loads, all APIs perform competitively with each other, with polling helping the performance by 1.7X, but consuming 2.3X CPU instructions; (2) at high-loads and scale, io_uring is more than an order of magnitude slower than SPDK; (3) at high-loads and scale, the benchmarking tool (fio) itself becomes a bottleneck; (4) state-of-practice Linux block I/O schedulers (BFQ, mq-deadline, and Kyber) introduce significant (up to 50%) overheads, and their use of global locks hinder their scalability. All artifacts from this work are available at https://github.com/atlarge-research/Performance-Characterization-Storage-Stacks.

References

[1]

2023. Chapter 12. Setting the disk scheduler. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/monitoring_and_managing_system_status_and_performance/setting-the-disk-schedulermonitoring-and-managing-system-status-and-performance. Accessed: 31-03-2023.

[2]

Accessed: 2023-01-26. 10.39M Storage I/O Per Second From One Thread. https://spdk.io/news/2019/05/06/nvme/

[3]

Accessed: 2023-01-26. aio. https://man7.org/linux/man-pages/man7/aio.7.html

[4]

Accessed: 2023-01-26. BFQ Budget Fair Queueing Document. https://www.kernel.org/doc/html/latest/block/bfq-iosched.html

[5]

Accessed: 2023-01-26. Evaluate Performance for Storage Performance Development Kit (SPDK)-based NVMe^* SSDs. https://www.intel.com/content/www/us/en/developer/articles/technical/evaluate-performance-for-storage-performance-development-kit-spdk-based-nvme-ssd.html

[6]

Accessed: 2023-01-26. fio. https://github.com/axboe/fio

[7]

Accessed: 2023-01-26. Intel® Optane^TM SSD DC P5800X Series. https://ark.intel.com/content/www/us/en/ark/products/201859/intel-optane-ssd-dc-p5800x-series-1-6tb-2-5in-pcie-x4-3d-xpoint.html

[8]

Accessed: 2023-01-26. io_uring. https://man.archlinux.org/man/io_uring.7.en

[9]

Accessed: 2023-01-26. Kyber multiqueue I/O scheduler. https://lwn.net/Articles/720071/

[10]

Accessed: 2023-01-26. Linux I/O schedulers. https://wiki.ubuntu.com/Kernel/Reference/IOSchedulers

[11]

Accessed: 2023-01-26. Multi-Queue Block IO Queueing Mechanism (blk-mq). https://www.kernel.org/doc/html/latest/block/blk-mq.html

[12]

Accessed: 2023-01-26. pread. https://man7.org/linux/man-pages/man2/pread.2.html

[13]

Accessed: 2023-01-26. pwrite. https://man7.org/linux/man-pages/man3/pwrite.3p.html

[14]

Accessed: 2023-01-26. SPDK. https://spdk.io/

[15]

Accessed: 2023-01-26. SPDK NVMe perf tool. https://github.com/spdk/spdk/blob/master/examples/nvme/perf/perf.c

[16]

Accessed: 2023-01-26. SPDK perf code. https://github.com/spdk/spdk/tree/master/examples/nvme/perf

[17]

Accessed: 2023-01-26. Toshiba Memory Introduces XL-FLASH Storage Class Memory Solution. https://americas.kioxia.com/en-us/business/news/2019/memory-20190805-1.html

[18]

Jens Axboe. Accessed: 2023-01-26. Efficient IO with io_uring. https://kernel.dk/io_uring.pdf

[19]

Ashish Bijlani and Umakishore Ramachandran. 2019. Extension Framework for File Systems in User space. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 121--134. https://www.usenix.org/conference/atc19/presentation/bijlani

[20]

Matias Bjørling, Jens Axboe, David W. Nellans, and Philippe Bonnet. 2013. Linux block IO: introducing multi-queue SSD access on multi-core systems. In 6th Annual International Systems and Storage Conference, SYSTOR '13, Haifa, Israel - June 30 - July 02, 2013, Ronen I. Kat, Mary Baker, and Sivan Toledo (Eds.). ACM, 22:1--22:10.

Digital Library

[21]

Jonathan Corbet. Accessed: 2023-01-26. Fixing asynchronous I/O, again. https://lwn.net/Articles/671649/

[22]

Diego Didona, Jonas Pfefferle, Nikolas Ioannou, Bernard Metzler, and Animesh Trivedi. 2022. Understanding modern storage APIs: a systematic study of libaio, SPDK, and io_uring. In SYSTOR '22: The 15th ACM International Systems and Storage Conference, Haifa, Israel, June 13 - 15, 2022, Michal Malka, Hillel Kolodner, Frank Bellosa, and Moshe Gabel (Eds.). ACM, 120--127.

Digital Library

[23]

Krijn Doekemeijer and Animesh Trivedi. 2022. Key-Value Stores on Flash Storage Devices: A Survey.

[24]

Alberto Faria, Ricardo Macedo, José Pereira, and João Paulo. 2021. BDUS: implementing block devices in user space. In SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14--16, 2021, Bruno Wassermann, Michal Malka, Vijay Chidambaram, and Danny Raz (Eds.). ACM, 8:1--8:11.

Digital Library

[25]

Shashank Gugnani, Xiaoyi Lu, and Dhabaleswar K. Panda. 2018. Analyzing, Modeling, and Provisioning QoS for NVMe SSDs. In 11th IEEE/ACM International Conference on Utility and Cloud Computing, UCC 2018, Zurich, Switzerland, December 17--20, 2018, Alan Sill and Josef Spillner (Eds.). IEEE Computer Society, 247--256.

[26]

Sergej Hardock. 2020. Flash-aware Database Management Systems. Ph. D. Dissertation. Technical University of Darmstadt, Germany. http://tuprints.ulb.tu-darmstadt.de/14476/

[27]

Mohammad Hedayati, Kai Shen, Michael L. Scott, and Mike Marty. 2019. Multi-Queue Fair Queueing. In Proceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference (Renton, WA, USA) (USENIX ATC '19). USENIX Association, USA, 301--314.

[28]

Mohammad Hedayati, Kai Shen, Michael L. Scott, and Mike Marty. 2019. Multi-Queue Fair Queuing. In 2019 USENIX Annual Technical Conference, USENIX ATC 2019, Renton, WA, USA, July 10--12, 2019, Dahlia Malkhi and Dan Tsafrir (Eds.). USENIX Association, 301--314. https://www.usenix.org/conference/atc19/presentation/hedayati-queue

[29]

John L. Hennessy and David A. Patterson. 2019. A new golden age for computer architecture. Commun. ACM 62, 2 (2019), 48--60.

Digital Library

[30]

Jaehyun Hwang, Midhul Vuppalapati, Simon Peter, and Rachit Agarwal. 2021. Rearchitecting Linux Storage Stack for μs Latency and High Throughput. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14--16, 2021, Angela Demke Brown and Jay R. Lorch (Eds.). USENIX Association, 113--128. https://www.usenix.org/conference/osdi21/presentation/hwang

[31]

Intel®. Accessed:2022-04-26. SPDK In The News. https://spdk.io/news/.

[32]

Jonathan Corbet. 2017. Two new block I/O schedulers for 4.12. https://lwn.net/Articles/720675/. Accessed: 31-03-2023.

[33]

Myoungsoo Jung, Wonil Choi, Shekhar Srikantaiah, Joonhyuk Yoo, and Mahmut T. Kandemir. 2014. HIOS: A Host Interface I/O Scheduler for Solid State Disks. In Proceeding of the 41st Annual International Symposium on Computer Architecuture (Minneapolis, Minnesota, USA) (ISCA '14). IEEE Press, 289--300.

[34]

Hyeong-Jun Kim, Young-Sik Lee, and Jin-Soo Kim. 2016. NVMeDirect: A User-space I/O Framework for Application-specific Optimization on NVMe SSDs. In 8th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2016, Denver, CO, USA, June 20--21, 2016, Nitin Agrawal and Sam H. Noh (Eds.). USENIX Association. https://www.usenix.org/conference/hotstorage16/workshop-program/presentation/kim

[35]

Jaeho Kim, Yongseok Oh, Eunsam Kim, Jongmoo Choi, Donghee Lee, and Sam H. Noh. 2009. Disk Schedulers for Solid State Drivers. In Proceedings of the Seventh ACM International Conference on Embedded Software (Grenoble, France) (EMSOFT '09). Association for Computing Machinery, New York, NY, USA, 295--304.

Digital Library

[36]

Sunghwan Kim, Gyusun Lee, Jiwon Woo, and Jinkyu Jeong. 2021. Zero-Copying I/O Stack for Low-Latency SSDs. IEEE Comput. Archit. Lett. 20, 1 (2021), 50--53.

[37]

Sungjoon Koh, Junhyeok Jang, Changrim Lee, Miryeong Kwon, Jie Zhang, and Myoungsoo Jung. 2019. Faster than Flash: An In-Depth Study of System Challenges for Emerging Ultra-Low Latency SSDs. In IEEE International Symposium on Workload Characterization, IISWC 2019, Orlando, FL, USA, November 3--5, 2019. IEEE, 216--227.

[38]

Kornilios Kourtis, Nikolas Ioannou, and Ioannis Koltsidas. 2019. Reaping the performance of fast NVM storage with uDepot. In 17th USENIX Conference on File and Storage Technologies, FAST 2019, Boston, MA, February 25--28, 2019, Arif Merchant and Hakim Weatherspoon (Eds.). USENIX Association, 1--15. https://www.usenix.org/conference/fast19/presentation/kourtis

[39]

Gyusun Lee, Seokha Shin, and Jinkyu Jeong. 2022. Efficient hybrid polling for ultra-low latency storage devices. J. Syst. Archit. 122 (2022), 102338.

Digital Library

[40]

Gyusun Lee, Seokha Shin, Wonsuk Song, Tae Jun Ham, Jae W. Lee, and Jinkyu Jeong. 2019. Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs. In 2019 USENIX Annual Technical Conference, USENIX ATC 2019, Renton, WA, USA, July 10--12, 2019, Dahlia Malkhi and Dan Tsafrir (Eds.). USENIX Association, 603--616. https://www.usenix.org/conference/atc19/presentation/lee-gyusun

[41]

Jing Liu, Anthony Rebello, Yifan Dai, Chenhao Ye, Sudarsun Kannan, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2021. Scale and Performance in a Filesystem Semi-Microkernel. In 28th Symposium on Operating Systems Principles (SOSP 21). ACM, 819--835.

[42]

Mingzhe Liu, Haikun Liu, Chencheng Ye, Xiaofei Liao, Hai Jin, Yu Zhang, Ran Zheng, and Liting Hu. 2022. Towards low-latency I/O services for mixed workloads using ultra-low latency SSDs. In ICS '22: 2022 International Conference on Supercomputing, Virtual Event, June 28 - 30, 2022, Lawrence Rauchwerger, Kirk W. Cameron, Dimitrios S. Nikolopoulos, and Dionisios N. Pnevmatikatos (Eds.). ACM, 13:1--13:12.

Digital Library

[43]

Corne Lukken, Giulia Frascaria, and Animesh Trivedi. 2021. ZCSD: a Computational Storage Device over Zoned Namespaces (ZNS) SSDs. CoRR abs/2112.00142 (2021). arXiv:2112.00142 https://arxiv.org/abs/2112.00142

[44]

Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu. 2022. GenStore: A High-Performance in-Storage Processing System for Genome Sequence Analysis. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS '22). Association for Computing Machinery, New York, NY, USA, 635--654.

Digital Library

[45]

Till Miemietz, Maksym Planeta, and Viktor Laurin Reusch. 2021. New Mechanism for Fast System Calls. CoRR abs/2112.10106 (2021). arXiv:2112.10106 https://arxiv.org/abs/2112.10106

[46]

Jaehong Min, Ming Liu, Tapan Chugh, Chenxingyu Zhao, Andrew Wei, In Hwan Doh, and Arvind Krishnamurthy. 2021. Gimbal: Enabling Multi-Tenant Storage Disaggregation on SmartNIC JBOFs. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (Virtual Event, USA) (SIGCOMM '21). Association for Computing Machinery, New York, NY, USA, 106--122.

Digital Library

[47]

Jayashree Mohan, Amar Phanishayee, Ashish Raniwala, and Vijay Chidambaram. 2021. Analyzing and Mitigating Data Stalls in DNN Training. Proc. VLDB Endow. 14, 5 (mar 2021), 771--784.

Digital Library

[48]

Stan Park and Kai Shen. 2012. FIOS: a fair, efficient flash I/O scheduler. In Proceedings of the 10th USENIX conference on File and Storage Technologies, FAST 2012, San Jose, CA, USA, February 14--17, 2012, William J. Bolosky and Jason Flinn (Eds.). USENIX Association, 13. https://www.usenix.org/conference/fast12/fios-fair-efficient-flash-io-scheduler

[49]

Stan Park and Kai Shen. 2012. FIOS: A Fair, Efficient Flash I/O Scheduler. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (San Jose, CA) (FAST'12). USENIX Association, USA, 13.

[50]

Yeonhong Park, Sunhong Min, and Jae W. Lee. 2022. Ginex: SSD-Enabled Billion-Scale Graph Neural Network Training on a Single Machine via Provably Optimal in-Memory Caching. Proc. VLDB Endow. 15, 11 (sep 2022), 2626--2639.

Digital Library

[51]

Xiang (Jenny) Ren, Kirk Rodrigues, Luyuan Chen, Camilo Vega, Michael Stumm, and Ding Yuan. 2019. An Analysis of Performance Evolution of Linux's Core Operations. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (Huntsville, Ontario, Canada) (SOSP '19). Association for Computing Machinery, New York, NY, USA, 554--569.

Digital Library

[52]

Sherif Sakr, Angela Bonifati, Hannes Voigt, Alexandra Iosup, Khaled Ammar, Renzo Angles, Walid Aref, Marcelo Arenas, Maciej Besta, Peter A. Boncz, Khuzaima Daudjee, Emanuele Della Valle, Stefania Dumbrava, Olaf Hartig, Bernhard Haslhofer, Tim Hegeman, Jan Hidders, Katja Hose, Adriana Iamnitchi, Vasiliki Kalavri, Hugo Kapp, Wim Martens, M. Tamer Özsu, Eric Peukert, Stefan Plantikow, Mohamed Ragab, Matei R. Ripeanu, Semih Salihoglu, Christian Schulz, Petra Selmer, Juan F. Sequeda, Joshua Shinavier, Gábor Szárnyas, Riccardo Tommasini, Antonino Tumeo, Alexandru Uta, Ana Lucia Varbanescu, Hsiang-Yun Wu, Nikolay Yakovets, Da Yan, and Eiko Yoneki. 2021. The Future is Big Graphs: A Community View on Graph Processing Systems. Commun. ACM 64, 9 (aug 2021), 62--71.

Digital Library

[53]

Kai Shen and Stan Park. 2013. FlashFQ: A Fair Queueing I/O Scheduler for Flash-Based SSDs. In Proceedings of the 2013 USENIX Conference on Annual Technical Conference (San Jose, CA) (USENIX ATC'13). USENIX Association, USA, 67--78.

[54]

Woong Shin, Qichen Chen, Myoungwon Oh, Hyeonsang Eom, and Heon Y. Yeom. 2014. OS I/O Path Optimizations for Flash Solid-state Drives. In 2014 USENIX Annual Technical Conference, USENIX ATC '14, Philadelphia, PA, USA, June 19--20, 2014, Garth Gibson and Nickolai Zeldovich (Eds.). USENIX Association, 483--488. https://www.usenix.org/conference/atc14/technical-sessions/presentation/shin

[55]

Livio Soares and Michael Stumm. 2010. FlexSC: Flexible System Call Scheduling with Exception-Less System Calls. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10). USENIX Association, Vancouver, BC. https://www.usenix.org/conference/osdi10/flexsc-flexible-system-call-scheduling-exception-less-system-calls

[56]

Timothy Stamler, Deukyeon Hwang, Amanda Raybuck, Wei Zhang, and Simon Peter. 2022. zIO: Accelerating IO-Intensive Applications with Transparent Zero-Copy IO. In 16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022, Carlsbad, CA, USA, July 11--13, 2022, Marcos K. Aguilera and Hakim Weatherspoon (Eds.). USENIX Association, 431--445. https://www.usenix.org/conference/osdi22/presentation/stamler

[57]

Athanasios Stratikopoulos, Christos Kotselidis, John Goodacre, and Mikel Luján. 2018. FastPath: Towards Wire-Speed NVMe SSDs. In 28th International Conference on Field Programmable Logic and Applications, FPL 2018, Dublin, Ireland, August 27--31, 2018. IEEE Computer Society, 170--177.

[58]

Athanasios Stratikopoulos, Christos Kotselidis, John Goodacre, and Mikel Luján. 2020. FastPath_MP: Low Overhead & Energy-efficient FPGA-based Storage Multi-paths. ACM Trans. Archit. Code Optim. 17, 4 (2020), 37:1--37:23.

Digital Library

[59]

Amy Tai, Igor Smolyar, Michael Wei, and Dan Tsafrir. 2021. Optimizing Storage Performance with Calibrated Interrupts. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14--16, 2021, Angela Demke Brown and Jay R. Lorch (Eds.). USENIX Association, 129--145. https://www.usenix.org/conference/osdi21/presentation/tai

[60]

Arash Tavakkol, Mohammad Sadrosadati, Saugata Ghose, Jeremie S. Kim, Yixin Luo, Yaohua Wang, Nika Mansouri Ghiasi, Lois Orosa, Juan Gómez-Luna, and Onur Mutlu. 2018. FLIN: Enabling Fairness and Enhancing Performance in Modern NVMe Solid State Drives. In Proceedings of the 45th Annual International Symposium on Computer Architecture (Los Angeles, California) (ISCA '18). IEEE Press, 397--410.

Digital Library

[61]

Animesh Trivedi, Nikolas Ioannou, Bernard Metzler, Patrick Stuedi, Jonas Pfefferle, Kornilios Kourtis, Ioannis Koltsidas, and Thomas R. Gross. 2018. FlashNet: Flash/Network Stack Co-Design. ACM Trans. Storage 14, 4, Article 30 (dec 2018), 29 pages.

Digital Library

[62]

Animesh Trivedi, Patrick Stuedi, Bernard Metzler, Roman Pletka, Blake G. Fitch, and Thomas R. Gross. 2013. Unified High-Performance I/O: One Stack to Rule Them All. In Proceedings of the 14th USENIX Conference on Hot Topics in Operating Systems (Santa Ana Pueblo, New Mexcio) (HotOS'13). USENIX Association, USA, 4.

[63]

Paolo Valente and Mauro Andreolini. 2012. Improving Application Responsiveness with the BFQ Disk I/O Scheduler. In Proceedings of the 5th Annual International Systems and Storage Conference (Haifa, Israel) (SYSTOR '12). Association for Computing Machinery, New York, NY, USA, Article 6, 12 pages.

Digital Library

[64]

Jiwon Woo, Minwoo Ahn, Gyusun Lee, and Jinkyu Jeong. 2021. D2FQ: Device-Direct Fair Queueing for NVMe SSDs. In 19th USENIX Conference on File and Storage Technologies, FAST 2021, February 23--25, 2021, Marcos K. Aguilera and Gala Yadgar (Eds.). USENIX Association, 403--415. https://www.usenix.org/conference/fast21/presentation/woo

[65]

Kan Wu, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2019. Towards an Unwritten Contract of Intel Optane SSD. In 11th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage 2019, Renton, WA, USA, July 8--9, 2019, Daniel Peek and Gala Yadgar (Eds.). USENIX Association. https://www.usenix.org/conference/hotstorage19/presentation/wu-kan

[66]

Qiumin Xu, Huzefa Siyamwala, Mrinmoy Ghosh, Tameesh Suri, Manu Awasthi, Zvika Guz, Anahita Shayesteh, and Vijay Balakrishnan. 2015. Performance analysis of NVMe SSDs and their implication on real world databases. In Proceedings of the 8th ACM International Systems and Storage Conference, SYSTOR 2015, Haifa, Israel, May 26--28, 2015, Dalit Naor, Gernot Heiser, and Idit Keidar (Eds.). ACM, 6:1--6:11.

Digital Library

[67]

Shuai Xue, Shang Zhao, Quan Chen, Gang Deng, Zheng Liu, Jie Zhang, Zhuo Song, Tao Ma, Yong Yang, Yanbo Zhou, Keqiang Niu, Sijie Sun, and Minyi Guo. 2020. Spool: Reliable Virtualized NVMe Storage Pool in Public Cloud Infrastructure. In USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 97--110.

[68]

Jisoo Yang, Dave B. Minturn, and Frank Hady. 2012. When poll is better than interrupt. In Proceedings of the 10th USENIX conference on File and Storage Technologies, FAST 2012, San Jose, CA, USA, February 14--17, 2012, William J. Bolosky and Jason Flinn (Eds.). USENIX Association, 3. https://www.usenix.org/conference/fast12/when-poll-better-interrupt

[69]

Ziye Yang, James R. Harris, Benjamin Walker, Daniel Verkamp, Changpeng Liu, Cunyin Chang, Gang Cao, Jonathan Stern, Vishal Verma, and Luse E. Paul. 2017. SPDK: A Development Kit to Build High Performance Storage Applications. In IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2017, Hong Kong, December 11--14, 2017. IEEE Computer Society, 154--161.

[70]

Youngjin Yu, Dongin Shin, Woong Shin, Nae Young Song, Jae-Woo Choi, Hyeong Seog Kim, Hyeonsang Eom, and Heon Young Yeom. 2014. Optimizing the Block I/O Subsystem for Fast Storage Devices. ACM Trans. Comput. Syst. 32, 2 (2014), 6:1--6:48.

Digital Library

[71]

Xiantao Zhang, Xiao Zheng, Zhi Wang, Hang Yang, Yibin Shen, and Xin Long. 2020. High-density Multi-tenant Bare-metal Cloud. In 25th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 20). ACM, 483--495.

[72]

Yuhong Zhong, Haoyu Li, Yu Jian Wu, Ioannis Zarkadas, Jeffrey Tao, Evan Mesterhazy, Michael Makris, Junfeng Yang, Amy Tai, Ryan Stutsman, and Asaf Cidon. 2022. XRP: In-Kernel Storage Functions with eBPF. In 16th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2022, Carlsbad, CA, USA, July 11--13, 2022, Marcos K. Aguilera and Hakim Weatherspoon (Eds.). USENIX Association, 375--393. https://www.usenix.org/conference/osdi22/presentation/zhong

[73]

Yuhong Zhong, Hongyi Wang, Yu Jian Wu, Asaf Cidon, Ryan Stutsman, Amy Tai, and Junfeng Yang. 2021. BPF for storage: an exokernel-inspired approach. In HotOS '21: Workshop on Hot Topics in Operating Systems, Ann Arbor, Michigan, USA, June, 1--3, 2021, Sebastian Angel, Baris Kasikci, and Eddie Kohler (Eds.). ACM, 128--135.

Digital Library

[74]

Jinbin Zhu, Limin Xiao, Liang Wang, Guangjun Qin, Rui Zhang, Yuting Liu, and Zhonglin Liu. 2021. UPM-DMA: An Efficient Userspace DMA-Pinned Memory Management Strategy for NVMe SSDs. In Algorithms and Architectures for Parallel Processing - 21st International Conference, ICA3PP 2021, Virtual Event, December 3--5, 2021, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 13155), Yongxuan Lai, Tian Wang, Min Jiang, Guangquan Xu, Wei Liang, and Aniello Castiglione (Eds.). Springer, 257--270.

Digital Library

Cited By

An BSung HEom H(2024)A2L: User-Transparent Workload-Adaptive Asynchronous I/O Layer (industry track)Proceedings of the 25th International Middleware Conference Industrial Track10.1145/3700824.3701098(36-42)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3700824.3701098
Chen YYang HLu KHuang WWang JWan JZhou JWu FXie C(2024)PeakFS: An Ultra-High Performance Parallel File System via Computing-Network-Storage Co-Optimization for HPC ApplicationsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.348575435:12(2578-2595)Online publication date: Dec-2024
https://doi.org/10.1109/TPDS.2024.3485754
Gupta ADhakshinamoorthy DPaul A(2024)Studying the Effects of Asynchronous I/O on HPC I/O Patterns2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops)10.1109/CLUSTERWorkshops61563.2024.00025(109-112)Online publication date: 24-Sep-2024
https://doi.org/10.1109/CLUSTERWorkshops61563.2024.00025
Show More Cited By

Index Terms

Performance Characterization of Modern Storage Stacks: POSIX I/O, libaio, SPDK, and io_uring
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Secondary storage

Recommendations

I/O interface independence with xNVMe
SYSTOR '22: Proceedings of the 15th ACM International Conference on Systems and Storage

The tight coupling of data-intensive systems and I/O interface has been a problem for years. A database system, relying on an specific I/O backend for direct asynchronous I/Os such as libaio, inherits its limitations in terms of portability, ...
BFQ, Multiqueue-Deadline, or Kyber? Performance Characterization of Linux Storage Schedulers in the NVMe Era
ICPE '24: Proceedings of the 15th ACM/SPEC International Conference on Performance Engineering

Flash SSDs have become the de-facto choice to deliver high I/O performance to modern data-intensive workloads. These workloads are often deployed in the cloud, where multiple tenants share access to flash-based SSDs. Cloud providers use various ...
On the performance variation in modern storage stacks
FAST'17: Proceedings of the 15th Usenix Conference on File and Storage Technologies

Ensuring stable performance for storage stacks is important, especially with the growth in popularity of hosted services where customers expect QoS guarantees. The same requirement arises from benchmarking settings as well. One would expect that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHEOPS '23: Proceedings of the 3rd Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems

May 2023

51 pages

ISBN:9798400700811

DOI:10.1145/3578353

General Chair:
Jean-Thomas Acquaviva,
Program Chairs:
Shadi Ibrahim,
Suren Byna

Copyright © 2023 Owner/Author(s).

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Conference

CHEOPS '23

Sponsor:

SIGOPS

CHEOPS '23: 3rd Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems

May 8, 2023

Rome, Italy

Acceptance Rates

Overall Acceptance Rate 6 of 8 submissions, 75%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
2,233
Total Downloads

Downloads (Last 12 months)1,540
Downloads (Last 6 weeks)252

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

An BSung HEom H(2024)A2L: User-Transparent Workload-Adaptive Asynchronous I/O Layer (industry track)Proceedings of the 25th International Middleware Conference Industrial Track10.1145/3700824.3701098(36-42)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3700824.3701098
Chen YYang HLu KHuang WWang JWan JZhou JWu FXie C(2024)PeakFS: An Ultra-High Performance Parallel File System via Computing-Network-Storage Co-Optimization for HPC ApplicationsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.348575435:12(2578-2595)Online publication date: Dec-2024
https://doi.org/10.1109/TPDS.2024.3485754
Gupta ADhakshinamoorthy DPaul A(2024)Studying the Effects of Asynchronous I/O on HPC I/O Patterns2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops)10.1109/CLUSTERWorkshops61563.2024.00025(109-112)Online publication date: 24-Sep-2024
https://doi.org/10.1109/CLUSTERWorkshops61563.2024.00025
Trivedi AJansen MDoekemeijer KTalluri STehrany N(2024) Reviving Storage Systems Education in the 21 st Century — An experience report 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00074(616-625)Online publication date: 6-May-2024
https://doi.org/10.1109/CCGrid59990.2024.00074
Zhan LLu KXiong YWan JYang Z(2024)TrickleKV: A High-Performance Key-Value Store on Disaggregated Storage With Low Network TrafficIEEE Access10.1109/ACCESS.2024.349688012(167596-167612)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3496880
Doekemeijer KTehrany NChandrasekaran BBjørling MTrivedi A(2023)Performance Characterization of NVMe Flash Devices with Zoned Namespaces (ZNS)2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00018(118-131)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00018

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents