More Web Proxy on the site http://driver.im/

Article

Storage Over IP: When Does Hardware Support Help?

Authors:

Prasenjit Sarkar,

Sandeep Uttamchandani,

Kaladhar VorugantiAuthors Info & Claims

FAST '03: Proceedings of the 2nd USENIX Conference on File and Storage Technologies

Pages 231 - 244

Published: 31 March 2003 Publication History

Abstract

This paper explores the effect of the current generation of hardware support for IP storage area networks on application performance. In this regard, this paper presents a comprehensive analysis of three competing approaches to build an IP storage area network that differ in their level of hardware support: software, TOE (TCP Offload Engine) and HBA (Host Bus Adapter). The software approach is based on the unmodified TCP/IP stacks that are part of a standard operating system distribution. For the two hardware-based approaches (TOE, HBA), we experimented with a range of adapters and chose a representative adapter for the current generation of each of the hardware approaches.The micro-benchmark analysis reveals that while hardware support does reduce the CPU utilization for large block sizes, the hardware support can itself be a performance bottleneck that hurts throughput and latency with small block sizes. Furthermore, the macro-benchmark analysis demonstrates that while the current generation of the hardware approaches may have the potential to provide performance improvements in CPU-intensive applications, overall the analysis does not demonstrate any performance benefits in database, scientific and email benchmarks. The analysis in this paper points out that a disparity in the processing power between the host and the adapter is the primary cause of the performance bottleneck in the current generation of the hardware approaches. The paper aims to guide the designers of the next generation of hardware-assisted adapters to better leverage the increasing processing power in the host. In particular, future adapters should be capable of handling small operations at wire speed.

References

[1]

{Ang01} Boon S. Ang, "An Evaluation of an Attempt at Offloading TCP/IP Protocol Processing on to an i960RN-based iNIC", Hewlett-Packard Technical Report HPL-2001-8, 2001.

[2]

{Ansi90} HIPPI, ANSI Standard X3T9.3/90-043, 1990.

[3]

{Bailey02} S. Bailey, "The Architecture of Direct Data Placement (DDP) and Remote Direct Memory Access (RDMA) On Internet Protocols", Internet Draft, IETF 2002, Work in Progress.

[4]

{Benner96} A. Benner, "Fibre Channel: Gigabit Communications and I/O For Computer Networks", McGraw-Hill, 1996.

[5]

{Chase01} J. Chase, A. Gallatin and K. Yocum, "End-System Optimizations for High-Speed TCP", IEEE Communications, 39:4, pp. 68-74, 2001.

[6]

{Compaq97} Compaq, Intel, Microsoft, "Virtual Interface Architecture Specification", v1.0, December 1997.

[7]

{DeBergalis03} M. DeBergalis, P. Corbett, S. Kleiman, A. Lent, D. Noveck, T. Talpey, M. Wittle., "The Direct Access File System", FAST 2003.

[8]

{Gibson97} G. Gibson, D. Nagle, K. Amiri, J. Butler, F. Chang, E. Feinberg, H. Gobyoff, C. Lee, B. Ozceri, E. Riedel, D. Rochberg, J. Zelenka., "File Server Scaling with Network-attached Secure Disks", Proceedings of ACM International Conference on Measurement and Modeling of Computer Systems, June 1997.

[9]

{Hotz98} S. Hotz, R. Van Meter, and G. Finn, "Internet Protocols for Network Attached Peripherals", 6th IEEE/NASA Conference on Mass Storage Systems and Technologies, 1998.

[10]

{Intel02} Intel Corp., "Iometer Performance Analysis Tool", http://www.intel.com/design/servers/devtools/iometer

[11]

{Katcher97} J. Katcher, "PostMark: A New File System Benchmark", Technical Report TR3022, Network Appliance Inc, 1997.

[12]

{Keng96} Hsiao Keng, and J. Chu, "Zero-copy TCP in Solaris", USENIX 1996 Annual Technical Conference, pp. 253- 264, 1996.

[13]

{Kent98} S. Kent and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, 1998.

[14]

{Kronenberg86} N. Kronenberg, H. Levy and W. Stecker. "Vax-Clusters: A Loosely Coupled Distributed System", ACM Transactions on Computer Systems, 4:2, pp. 130-146, 1986.

[15]

{Magoutis02} K. Magoutis, S. Addetia, A. Federova, M. Seltzer, J. Chase, A. Gallatin, R. Kisley, R. Wickremsinghe, E. Gabber, "Structure and Performance of the Direct Access File System", USENIX Technical Conference, pp. 1-14, 2002.

[16]

{Magoutis03} K. Magoutis, S. Addetia, A. Federova, M. Seltzer, "Making the Most out of Direct-Access Network Attached Storage", FAST 2003.

[17]

{Ng02} Wee Teck Ng, H. Sun, B. Hillyer, E. Shriver, E. Gabber, B. Ozden, "Obtaining High Performance for Storage Outsourcing", FAST, pp. 145-158, 2002.

[18]

{Rajagopal02} M. Rajagopal, E. Rodriquez, R. Weber., "Fibre Channel Over TCP/IP", Internet Draft, IETF, 2002, Work in Progress.

[19]

{Sarkar02} P. Sarkar and K. Voruganti, "IP Storage: The Challenge Ahead", 19th IEEE Symposium on Mass Storage Systems, pp. 35-42, 2002.

[20]

{Satran02} J. Satran, K. Meth, C. Mallikarjun, C. Sapuntzakis, E. Zeidner, "iSCSI", Internet Draft, IETF, 2002, Work in Progress.

[21]

{Shanley02} T. Shanley and J. Winkley, "Infiniband Network Architecture", Addison-Wesley, 2002.

[22]

{TPC97} TPC Benchmark C Standard Revision 3.3.2, Transaction Processing Performance Council. 1997.

[23]

{VanMeter98} R. Van Meter, G. Finn, and S. Hotz, "VISA: Netstation's Virtual Internet SCSI Adapter", ASPLOS 8, pp. 71- 80, 1998.

[24]

{Vengroff96} D. E. Vengroff and J. S. Vitter, "I/O-Efficient Scientific Computation using TPIE", IEEE Conference on Mass Storage Systems and Technologies, pp. 553-570, 1996.

[25]

{Voruganti01} K. Voruganti and P. Sarkar, "An Analysis of Three Gigabit Storage Networking Protocols", pp. 259-265, IPCCC 2001.

Cited By

Shalev LSatran JBorovik EBen-Yehuda M(2010)IsoStackProceedings of the 2010 USENIX conference on USENIX annual technical conference10.5555/1855840.1855845(5-5)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.5555/1855840.1855845
Solá-Sloan JCouvertier-Reyes IWainer GShaffer CMcGraw RChinni M(2009)A TCP offload engine emulator for estimating the impact of removing protocol processing from a host running Apache HTTP serverProceedings of the 2009 Spring Simulation Multiconference10.5555/1639809.1639889(1-8)Online publication date: 22-Mar-2009
https://dl.acm.org/doi/10.5555/1639809.1639889
Deng Y(2009)Deconstructing Network Attached Storage systemsJournal of Network and Computer Applications10.1016/j.jnca.2009.02.00632:5(1064-1072)Online publication date: 1-Sep-2009
https://dl.acm.org/doi/10.1016/j.jnca.2009.02.006
Show More Cited By

Index Terms

Storage Over IP: When Does Hardware Support Help?

Recommendations

Storage over IP: when does hardware support help
FAST'03: Proceedings of the 2nd USENIX conference on File and storage technologies

This paper explores the effect of the current generation of hardware support for IP storage area networks on application performance. In this regard, this paper presents a comprehensive analysis of three competing approaches to build an IP storage area ...
PC-based Router Virtualization with Hardware Support
AINA '12: Proceedings of the 2012 IEEE 26th International Conference on Advanced Information Networking and Applications

In this paper we focus on how to use open source Linux software in combination with PC hardware to build high speed virtual routers. Router virtualization means that multiple virtual router instances will run in parallel on the same hardware. To enable ...
Hardware-Accelerated Software Library Drivers Generation for IP-Centric SoC Designs
GLSVLSI '16: Proceedings of the 26th edition on Great Lakes Symposium on VLSI

In recent years, the semiconductor industry has been witnessing an increasing reuse of hardware IPs for System-on-Chip (SoC) designs and embedded computing systems on FPGA platforms with hard-core processors. The IP-reuse comes with an increasing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

FAST '03: Proceedings of the 2nd USENIX Conference on File and Storage Technologies

March 2003

273 pages

Publisher

USENIX Association

United States

Publication History

Published: 31 March 2003

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
129
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shalev LSatran JBorovik EBen-Yehuda M(2010)IsoStackProceedings of the 2010 USENIX conference on USENIX annual technical conference10.5555/1855840.1855845(5-5)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.5555/1855840.1855845
Solá-Sloan JCouvertier-Reyes IWainer GShaffer CMcGraw RChinni M(2009)A TCP offload engine emulator for estimating the impact of removing protocol processing from a host running Apache HTTP serverProceedings of the 2009 Spring Simulation Multiconference10.5555/1639809.1639889(1-8)Online publication date: 22-Mar-2009
https://dl.acm.org/doi/10.5555/1639809.1639889
Deng Y(2009)Deconstructing Network Attached Storage systemsJournal of Network and Computer Applications10.1016/j.jnca.2009.02.00632:5(1064-1072)Online publication date: 1-Sep-2009
https://dl.acm.org/doi/10.1016/j.jnca.2009.02.006
Traeger AZadok EJoukov NWright C(2008)A nine year study of file system and storage benchmarkingACM Transactions on Storage10.1145/1367829.13678314:2(1-56)Online publication date: 28-May-2008
https://dl.acm.org/doi/10.1145/1367829.1367831
Brecht TJanakiraman GLynn BSaletore VTurner Y(2006)Evaluating network processing efficiency with processor partitioning and asynchronous I/OACM SIGOPS Operating Systems Review10.1145/1218063.121796140:4(265-278)Online publication date: 18-Apr-2006
https://dl.acm.org/doi/10.1145/1218063.1217961
Brecht TJanakiraman GLynn BSaletore VTurner YBerbers YZwaenepoel W(2006)Evaluating network processing efficiency with processor partitioning and asynchronous I/OProceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 200610.1145/1217935.1217961(265-278)Online publication date: 18-Apr-2006
https://dl.acm.org/doi/10.1145/1217935.1217961
Wang WYeo HZhu YChong TChai TZhou LBitwas J(2006)Design and development of Ethernet-based storage area network protocolComputer Communications10.1016/j.comcom.2005.10.00429:9(1271-1283)Online publication date: 1-May-2006
https://dl.acm.org/doi/10.1016/j.comcom.2005.10.004
Joglekar AKounavis MBerry F(2005)A scalable and high performance software iSCSI implementationProceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 410.5555/1251028.1251048(20-20)Online publication date: 13-Dec-2005
https://dl.acm.org/doi/10.5555/1251028.1251048
Saletore VStillwell Jr PWiegert JCayton PGray JRegnier G(2005)Efficient Direct User Level Sockets for an Intel® Xeon" Processor Based TCP On-Load EngineProceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 9 - Volume 1010.1109/IPDPS.2005.191Online publication date: 4-Apr-2005
https://dl.acm.org/doi/10.1109/IPDPS.2005.191
Tomonori FMasanori OJiang HYang Q(2004)Analysis of iSCSI target softwareProceedings of the international workshop on Storage network architecture and parallel I/Os10.1145/1162628.1162632(25-32)Online publication date: 30-Sep-2004
https://dl.acm.org/doi/10.1145/1162628.1162632
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents