More Web Proxy on the site http://driver.im/

research-article

A nine year study of file system and storage benchmarking

Authors:

Avishay Traeger,

Nikolai Joukov,

Charles P. WrightAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 4, Issue 2

Article No.: 5, Pages 1 - 56

https://doi.org/10.1145/1367829.1367831

Published: 28 May 2008 Publication History

Abstract

Benchmarking is critical when evaluating performance, but is especially difficult for file and storage systems. Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is rather difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The large variety of workloads that these systems experience in the real world also adds to this difficulty.

In this article we survey 415 file system and storage benchmarks from 106 recent papers. We found that most popular benchmarks are flawed and many research papers do not provide a clear indication of true performance. We provide guidelines that we hope will improve future performance evaluations. To show how some widely used benchmarks can conceal or overemphasize overheads, we conducted a set of experiments. As a specific example, slowing down read operations on ext2 by a factor of 32 resulted in only a 2--5% wall-clock slowdown in a popular compile benchmark. Finally, we discuss future work to improve file system and storage benchmarking.

References

[1]

&ast;Abd-El-Malek, M., II, W. V. C., Cranor, C., Ganger, G., Hendricks, J., Klosterman, A. J., Mesnier, M., Prasad, M., Salmon, B., Sambasivan, R. R., Sinnamohideen, S., Strunk, J. D., Thereska, E., Wachs, M., and Wylie, J. J. 2005.Ursa Minor: Versatile cluster-based storage. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 59--72.

Digital Library

[2]

&ast;Adya, A., Bolosky, W. J., Castro, M., Cermak, G., Chaiken, R., Douceur, J. R., Howell, J., Lorch, J. R., Theimer, M., and Wattenhofer, R. P. 2002. FARSITE: Federated, available, and reliable storage for an incompletely trusted environment. In Proceedings of the 5th Symposium on Operating System Design and Implementation, Boston, MA, 1--14.

Digital Library

[3]

Agrawal, N., Bolosky, W. J., Douceur, J. R., and Lorch, J. R. 2007. A five-year study of file-system metadata. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 31--45.

Digital Library

[4]

&ast;Aguilera, M. K., Ji, M., Lillibridge, M., MacCormick, J., Oertli, E., Andersen, D., Burrows, M., Mann, T., and Thekkath, C. A. 2003. Block-Level security for network-attached disks. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 159--174.

Digital Library

[5]

Akkerman, W. 2002. Strace software home page. www.liacs.nl/~wichert/strace/.

[6]

&ast;Anderson, D. C., Chase, J. S., and Vahdat, A. M. 2000. Interposed request routing for scalable network storage. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 259--272.

Digital Library

[7]

&ast;Anderson, E., Hobbs, M., Keeton, K., Spence, S., Uysal, M., and Veitch, A. 2002. Hippodrome: Running circles around storage administration. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 175--188.

Digital Library

[8]

&ast;Anderson, E., Kallahalla, M., Uysal, M., and Swaminathan, R. 2004. Buttress: A toolkit for flexible and high fidelity I/O benchmarking. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 45--58.

Digital Library

[9]

Andrerson, D. 2002. Fstress: A flexible network file service benchmark. Tech. Rep. TR-2001-2002, Duke University. May.

[10]

&ast;Aranya, A., Wright, C. P., and Zadok, E. 2004. Tracefs: A file system to trace them all. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 129--143.

Digital Library

[11]

&ast;Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., Burnett, N. C., Denehy, T. E., Engle, T. J., Gunawi, H. S., Nugent, J. A., and Popovici, F. I. 2003. Transforming policies into mechanisms with Infokernel. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (ACM SIGOPS), Bolton Landing, NY, 90--105.

Digital Library

[12]

Blaze, M. 1992. NFS tracing by passive network monitoring. In Proceedings of the USENIX Winter Conference, San Francisco, CA.

[13]

Bray, T. 1996. Bonnie home page. www.textuality.com/bonnie.

[14]

Bryant, R., Forester, R., and Hawkes, J. 2002. Filesystem performance and scalability in Linux 2.4.17. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, Monterey, CA, 259--274.

Digital Library

[15]

Bryant, R., Raddatz, D., and Sunshine, R. 2001. PenguinoMeter: A new file-I/O benchmark for Linux. In Proceedings of the 5th Annual Linux Showcase and Conference, Oakland, CA, 5--10.

Digital Library

[16]

Chen, P. M. and Patterson, D. A. 1993. A new approach to I/O performance evaluation—Self-Scaling I/O benchmarks, predicted I/O performance. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (ACM SIGOPS), Seattle, WA, 1--12.

Digital Library

[17]

&ast;Cipar, J., Corner, M. D., and Berger, E. D. 2007. TFS: A transparent file system for contributory storage. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 215--229.

Digital Library

[18]

Coker, R. 2001. Bonnie++ home page. www.coker.com.au/bonnie++.

[19]

&ast;Corbett, P., English, B., Goel, A., Grcanac, T., Kleiman, S., Leong, J., and Sankar, S. 2004. Row-Diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 1--14.

Digital Library

[20]

&ast;Dabek, F., Kaashoek, M. F., Karger, D., and Morris, R. 2001. Wide-Area cooperative storage with CFS. In Proceedings of the 18th ACM Symposium on Operating Systems Principles, Banff, Canada.

Digital Library

[21]

&ast;DeBergalis, M., Corbett, P., Kleiman, S., Lent, A., Noveck, D., Talpey, T., and Wittle, M. 2003. The direct access file system. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 175--188.

Digital Library

[22]

&ast;Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2002. Bridging the information gap in storage protocol stacks. In Proceedings of the Annual USENIX Technical Conference, Monterey, CA, 177--190.

Digital Library

[23]

&ast;Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005. Journal-guided resynchronization for software RAID. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 87--100.

Digital Library

[24]

&ast;Dimitrijevic, Z., Rangaswami, R., and Chang, E. 2003. Design and implementation of semi-preemptible IO. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 145--158.

Digital Library

[25]

&ast;Eisler, M., Corbett, P., Kazar, M., Nydick, D. S., and Wagner, J. C. 2007. Data ONTAP GX: A scalable storage cluster. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 139--152.

Digital Library

[26]

Ellard, D., Ledlie, J., Malkani, P., and Seltzer, M. 2003. Passive NFS tracing of email and research workloads. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA.

Digital Library

[27]

Ellard, D. and Seltzer, M. 2003a. New NFS tracing tools and techniques for system analysis. In Proceedings of the Annual USENIX Conference on Large Installation Systems Administration, San Diego, CA.

Digital Library

[28]

Ellard, D. and Seltzer, M. 2003b. NFS tricks and benchmarking traps. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, San Antonio, TX, 101--114.

Digital Library

[29]

&ast;Flinn, J., Sinnamohideen, S., Tolia, N., and Satyanaryanan, M. 2003. Data staging on untrusted surrogates. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 15--28.

Digital Library

[30]

&ast;Fraser, K. and Chang, F. 2003. Operating system I/O speculation: How two invocations are faster than one. In Proceedings of the Annual USENIX Technical Conference. San Antonio, TX, 325--338.

[31]

&ast;Fu, K., Kaashoek, M. F., and Mazières, D. 2000. Fast and secure distributed read-only file system. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 181--196.

Digital Library

[32]

&ast;Ghemawat, S., Gobioff, H., and Leung, S. T. 2003. The Google file system. In Proceedings of the 19th ACM Symposium on Operating Systems Principles. (ACM SIGOPS), Bolton Landing, NY, 29--43.

Digital Library

[33]

&ast;Gniady, C., Butt, A. R., and Hu, Y. C. 2004. Program-Counter-Based pattern classification in buffer caching. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (ACM SIGOPS), San Francisco, CA, 395--408.

Digital Library

[34]

&ast;Gopal, B. and Manber, U. 1999. Integrating content-based access mechanisms with hierarchical file systems. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation. (ACM SIGOPS), New Orleans, LA, 265--278.

Digital Library

[35]

&ast;Grönvall, B., Westerlund, A., and Pink, S. 1999. The design of a multicast-based distributed file system. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation (ACM SIGOPS), New Orleans, LA, 251--264.

Digital Library

[36]

&ast;Gulati, A., Naik, M., and Tewari, R. 2007. Nache: Design and implementation of a caching proxy for nfsv4. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 199--214.

Digital Library

[37]

&ast;Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanan, M., Sidebotham, R. N., and West, M. J. 1988. Scale and performance in a distributed file system. ACM Trans. Comput. Syst. 6, 1 (Feb.), 51--81.

Digital Library

[38]

&ast;Huang, H., Hung, W., and Shin, K. 2005. FS2: Dynamic data replication in free disk space for improving disk performance and energy consumption. In Proceedings of the 20th ACM Symposium on Operating Systems Principles, Brighton, UK, 263--276.

Digital Library

[39]

Huang, L. and Chiueh, T. 2001. Charm: An I/O-driven execution strategy for high-performance transaction processing. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 275--288.

Digital Library

[40]

&ast;Joglekar, A., Kounavis, M. E., and Berry, F. L. 2005. A scalable and high performance software iSCSI implementation. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 267--280.

Digital Library

[41]

Joukov, N., Traeger, A., Iyer, R., Wright, C. P., and Zadok, E. 2006. Operating system profiling via latency analysis. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (ACM SIGOPS), Seattle, WA, 89--102.

Digital Library

[42]

&ast;Joukov, N., Wong, T., and Zadok, E. 2005. Accurate and efficient replaying of file system traces. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 337--350.

Digital Library

[43]

&ast;Kallahalla, M., Riedel, E., Swaminathan, R., Wang, Q., and Fu, K. 2003. Plutus: Scalable secure file sharing on untrusted storage. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 29--42.

Digital Library

[44]

Kaminsky, M., Savvides, G., Mazieres, D., and Kaashoek, M. F. 2003. Decentralized user authentication in a global file system. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (ACM SIGOPS), Bolton Landing, NY.

Digital Library

[45]

Katcher, J. 1997. PostMark: A new filesystem benchmark. Tech. Rep. TR3022, Network Appliance. www.netapp.com/tech_library/3022.html.

[46]

&ast;Kim, J. M., Choi, J., Kim, J., Noh, S. H., Min, S. L., Cho, Y., and Kim, C. S. 2000. A low-overhead, high-performance unified buffer management scheme that exploits sequential and looping references. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 119--134.

Digital Library

[47]

&ast;Kim, M., Cox, L., and Noble, B. 2002. Safety, visibility, and performance in a wide-area file system. In Proceedings of the 1st USENIX Conference on File and Storage Technologies. Monterey, CA.

Digital Library

[48]

&ast;Kroeger, T. M. and Long, D. D. E. 2001. Design and implementation of a predictive file prefetching algorithm. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 105--118.

Digital Library

[49]

LASS. 2006. UMass trace repository. http://traces.cs.umass.edu.

[50]

&ast;Lee, Y., Leung, K., and Satyanarayanan, M. 1999. Operation-Based update propagation in a mobile file system. In Proceedings of the Annual USENIX Technical Conference, Monterey, CA, 43--56.

Digital Library

[51]

&ast;Li, J., Krohn, M., Mazières, D., and Shasha, D. 2004. Secure untrusted data repository (SUNDR). In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, San Francisco, CA, 121--136.

Digital Library

[52]

&ast;Lu, C., Alvarez, G. A., and Wilkes, J. 2002. Aqueduct: Online data migration with performance guarantees. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA.

Digital Library

[53]

&ast;Lumb, C. R., Merchant, A., and Alvarez, G. A. 2003. Façade: Virtual storage devices with performance guarantees. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 131--144.

Digital Library

[54]

&ast;Lumb, C. R., Schindler, J., and Ganger, G. R. 2002. Freeblock scheduling outside of disk firmware. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 275--288.

Digital Library

[55]

&ast;MacCormick, J., Murphy, N., Najork, M., Thekkath, C., and Zhou, L. 2004. Boxwood: Abstractions as the foundation for storage infrastructure. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, San Francisco, CA, 105--120.

Digital Library

[56]

&ast;Magoutis, K., Addetia, S., Fedorova, A., and Seltzer, M. I. 2003. Making the most out of direct-access network attached storage. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 189--202.

Digital Library

[57]

&ast;Magoutis, K., Addetia, S., Fedorova, A., Seltzer, M. I., Chase, J. S., Gallatin, A. J., Kisley, R., Wickremesinghe, R. G., and Gabber, E. 2002. Structure and performance of the direct access file system. In Proceedings of the Annual USENIX Technical Conference, Monterey, CA.

Digital Library

[58]

&ast;Maziéres, D. 2001. A toolkit for user-level file systems. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 261--274.

Digital Library

[59]

&ast;Mazières, D., Kaminsky, M., Kaashoek, M. F., and Witchel, E. 1999. Separating key management from file system security. In Proceedings of the 17th ACM Symposium on Operating Systems Principles, Charleston, SC, 124--139.

Digital Library

[60]

McDougall, R. and Mauro, J. 2005. FileBench. www.solarisinternals.com/si/tools/filebench/.

Digital Library

[61]

&ast;Memik, G., Kandemir, M., and Choudhary, A. 2002. Exploiting inter-file access patterns using multi-collective I/O. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA.

Digital Library

[62]

Mesnier, M. P., Wachs, M., Sambasivan, R. R., Lopez, J., Hendricks, J., Ganger, G. R., and O'Hallaron, D. 2007. //TRACE: Parallel trace replay with approximate causal events. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 153--167.

Digital Library

[63]

&ast;Miller, E., Freeman, W., Long, D., and Reed, B. 2002. Strong security for network-attached storage. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 1--13.

Digital Library

[64]

Mogul, J. 1999. Brittle metrics in operating systems research. In Proceedings of the IEEE Workshop on Hot Topics in Operating Systems (HOTOS), Rio Rica, AZ, 90--95.

Digital Library

[65]

Mummert, L. and Satyanarayanan, M. 1994. Long term distributed file reference tracing: Implementation and experience. Tech. Rep. CMU-CS-94-213, Carnegie Mellon University, Pittsburgh, Pennsylvania.

Digital Library

[66]

&ast;Muniswamy-Reddy, K., Holland, D. A., Braun, U., and Seltzer, M. 2006. Provenance-Aware storage systems. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 43--56.

Digital Library

[67]

&ast;Muniswamy-Reddy, K., Wright, C. P., Himmer, A., and Zadok, E. 2004. A versatile and user-oriented versioning file system. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 115--128.

Digital Library

[68]

&ast;Muthitacharoen, A., Chen, B., and Mazieres, D. 2001. A low-bandwidth network file system. In Proceedings of the 18th ACM Symposium on Operating Systems Principles, Banff, Canada.

Digital Library

[69]

&ast;Muthitacharoen, A., Morris, R., Gil, T. M., and Che, B. 2002. Ivy: A read/write peer-to-peer file system. In Proceedings of the 5th Symposium on Operating System Design and Implementation, Boston, MA, 31--44.

Digital Library

[70]

&ast;Ng, W. T., Sun, H., Hillyer, B., Shriver, E., Gabber, E., and Ozden, B. 2002. Obtaining high performance for storage outsourcing. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 145--158.

Digital Library

[71]

&ast;Nightingale, E. B., Chen, P., and Flinn, J. 2005. Speculative execution in a distributed file system. In Proceedings of the 20th ACM Symposium on Operating Systems Principles, Brighton, UK, 191--205.

Digital Library

[72]

&ast;Nightingale, E. B. and Flinn, J. 2004. Energy-Efficiency and storage flexibility in the Blue file system. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, San Francisco, CA, 363--378.

Digital Library

[73]

&ast;Nightingale, E. B., Veeraraghavan, K., Chen, P. M., and Flinn, J. 2006. Rethink the sync. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Seattle, WA, 1--14.

Digital Library

[74]

&ast;Nugent, J., Arpaci-Dusseau, A., and Arpaci-Dusseau, R. 2003. Controlling your PLACE in the file system with gray-box techniques. In Proceedings of the Annual USENIX Technical Conference, San Antonio, TX, 311--323.

[75]

OSDL. 2004. Iometer project. www.iometer.org/.

[76]

OSDL. 2007. Database test suite. www.osdl.org/lab_activities/kernel_testing/osdl_database_test_suite/.

[77]

Ousterhout, J. 1990. Why aren't operating systems getting faster as fast as hardware&quest; In Proceedings of the Summer USENIX Technical Conference, Anaheim, CA, 247--256.

[78]

Ousterhout, J., Costa, H., Harrison, D., Kunze, J., Kupfer, M., and Thompson, J. 1985. A trace-driven analysis of the UNIX 4.2 BSD file system. In Proceedings of the 10th ACM Symposium on Operating System Principles, Orcas Island, WA, 15--24.

Digital Library

[79]

&ast;Padioleau, Y. and Ridoux, O. 2003. A logic file system. In Proceedings of the Annual USENIX Technical Conference, San Antonio, TX, 99--112.

[80]

&ast;Papathanasiou, A. E. and Scott, M. L. 2004. Energy efficient prefetching and caching. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 255--268.

Digital Library

[81]

&ast;Patterson, H., Manley, S., Federwisch, M., Hitz, D., Kleinman, S., and Owara, S. 2002. SnapMirror: File system based asynchronous mirroring for disaster recovery. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 117--129.

Digital Library

[82]

&ast;Peek, D. and Flinn, J. 2006. EnsemBlue: Integrating distributed storage and consumer electronics. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Seattle, WA, 219--232.

Digital Library

[83]

PEL. 2001. BYU trace distribution center. http://tds.cs.byu.edu/tds.

[84]

&ast;Peterson, Z. N. J., Burns, R., Ateniese, G., and Bono, S. 2007. Design and implementation of verifiable audit trails for a versioning file system. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 93--106.

Digital Library

[85]

&ast;Peterson, Z. N. J., Burns, R., J. Herring, A. S., and Rubin, A. D. 2005. Secure deletion for a versioning file system. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 143--154.

Digital Library

[86]

&ast;Prabhakaran, V., Agrawal, N., Bairavasundaram, L. N., Gunawi, H. S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005a. IRON file systems. In Proceedings of the 20th ACM Symposium on Operating Systems Principles, Brighton, UK, 206--220.

Digital Library

[87]

&ast;Prabhakaran, V., Arpaci-Dusseau, A. C., and Arpaci-Dussea, R. H. 2005b. Analysis and evolution of journaling file systems. In Proceedings of the Annual USENIX Technical Conference, Anaheim, CA, 105--120.

Digital Library

[88]

&ast;Quinlan, S. and Dorward, S. 2002. Venti: A new approach to archival storage. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 89--101.

Digital Library

[89]

&ast;Radkov, P., Yin, L., Goyal, P., Sarkar, P., and Shenoy, P. 2004. A performance comparison of NFS and iSCSI for IP-networked storage. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 101--114.

Digital Library

[90]

&ast;Rhea, S., Eaton, P., Geels, D., Weatherspoon, H., Zhao, B., and Kubiatowicz, J. 2003. Pond: The OceanStore prototype. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 1--14.

Digital Library

[91]

Robinson, D. 1999. The advancement of NFS benchmarking: SFS 2.0. In Proceedings of the 13th USENIX Systems Administration Conference, Seattle, WA, 175--185.

Digital Library

[92]

Roselli, D., Lorch, J. R., and Anderson, T. E. 2000. A comparison of file system workloads. In Proceedings of the Annual USENIX Technical Conference, San Diego, CA, 41--54.

Digital Library

[93]

Rosenblum, M. 1992. The design and implementation of a log-structured file system. Ph.D. thesis, Electrical Engineering and Computer Sciences, Computer Science Division, University of California.

Digital Library

[94]

&ast;Rowstron, A. and Druschel, P. 2001. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the 18th ACM Symposium on Operating Systems Principles, Banff, Canada.

Digital Library

[95]

Ruemmler, C. and Wilkes, J. 1993. UNIX disk access patterns. In Proceedings of the Winter USENIX Technical Conference, San Diego, CA, 405--420.

[96]

Ruwart, T. M. 2001. File system performance benchmarks, then, now, and tomorrow. In Proceedings of the 14th IEEE Symposium on Mass Storage Systems, San Diego, CA.

Digital Library

[97]

&ast;Saito, Y., Karamanolis, C., Karlsson, M., and Mahalingam, M. 2002. Taming aggressive replication in the Pangaea wide-area file system. In Proceedings of the 5th Symposium on Operating System Design and Implementation, Boston, MA, 15--30.

Digital Library

[98]

Sandberg, R., Goldberg, D., Kleiman, S., Walsh, D., and Lyon, B. 1985. Design and implementation of the Sun network filesystem. In Proceedings of the Summer USENIX Technical Conference, Portland, Oregon, 119--130.

[99]

&ast;Santry, D. S., Feeley, M. J., Hutchinson, N. C., Veitch, A. C., Carton, R. W., and Ofir, J. 1999. Deciding when to forget in the Elephant file system. In Proceedings of the 17th ACM Symposium on Operating Systems Principles, Charleston, SC, 110--123.

Digital Library

[100]

&ast;Sarkar, P., Uttamchandani, S., and Voruganti, K. 2003. Storage over IP: When does hardware support help&quest; In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 231--244.

Digital Library

[101]

&ast;Schindler, J., Griffin, J. L., Lumb, C. R., and Ganger, G. R. 2002. Track-Aligned extents: Matching access patterns to disk drive characteristics. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 259--274.

Digital Library

[102]

&ast;Schindler, J., Schlosser, S. W., Shao, M., and Ailamaki, A. 2004. Atropos: A disk array volume manager for orchestrated use of disks. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 159--172.

Digital Library

[103]

&ast;Schlosser, S. W., Schindler, J., Papadomanolakis, S., Shao, M., Ailamaki, A., Faloutsos, C., and Ganger, G. R. 2005. On multidimensional data and modern disks. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 225--238.

Digital Library

[104]

Schmidt, A., Waas, F., Kersten, M., Florescu, D., Carey, M. J., Manolescu, I., and Busse, R. 2001. Why and how to benchmark XML databases. ACM SIGMOD Rec. 30, 3 (Sept.), 27--32.

Digital Library

[105]

&ast;Schmuck, F. and Haskin, R. 2002. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 231--244.

Digital Library

[106]

&ast;Seltzer, M. I., Ganger, G. R., McKusick, M. K., Smith, K. A., Soules, C. A. N., and Stein, C. A. 2000. Journaling versus soft updates: Asynchronous meta-data protection in file systems. In Proceedings of the Annual USENIX Technical Conference, San Diego, CA, 71--84.

Digital Library

[107]

Seltzer, M. I., Krinsky, D., Smith, K. A., and Zhang, X. 1999. The case for application-specific benchmarking. In Proceedings of the IEEE Workshop on Hot Topics in Operating Systems (HOTOS), Rio Rica, AZ, 102--107.

Digital Library

[108]

Shein, B., Callahan, M., and Woodbury, P. 1989. NFSSTONE: A network file server performance benchmark. In Proceedings of the Summer USENIX Technical Conference, Baltimore, MD, 269--275.

[109]

Shepler, S. 2005. NFS version 4. In Proceedings of the Annual USENIX Technical Conference, Anaheim, CA. http://mediacast.sun.com/share/shepler/20050414_usenix_ext.pdf.

[110]

&ast;Shrira, L. and Xu, H. 2006. Thresher: An efficient storage manager for copy-on-write snapshots. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 57--70.

Digital Library

[111]

&ast;Sivathanu, G., Sundararaman, S., and Zadok, E. 2006. Type-Safe disks. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Seattle, WA, 15--28.

Digital Library

[112]

&ast;Sivathanu, M., Bairavasundaram, L. N., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2004a. Life or death at block-level. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, San Francisco, CA, 379--394.

Digital Library

[113]

&ast;Sivathanu, M., Bairavasundaram, L. N., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005. Database-Aware semantically-smart storage. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 239--252.

Digital Library

[114]

&ast;Sivathanu, M., Prabhakaran, V., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2004b. Improving storage system availability with D-GRAID. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 15--30.

Digital Library

[115]

&ast;Sivathanu, M., Prabhakaran, V., Popovici, F. I., Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2003. Semantically-Smart disk systems. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 73--88.

Digital Library

[116]

Small, C., Ghosh, N., Saleeb, H., Seltzer, M., and Smith, K. 1997. Does systems research measure up&quest; Tech. Rep. TR-16-97, Harvard University. November.

[117]

Smith, K. A. and Seltzer, M. I. 1997. File system aging—Increasing the relevance of file system benchmarks. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, Seattle, WA, 203--213.

Digital Library

[118]

SNIA. 2007. SNIA—Storage network industry association: IOTTA repository. http://iotta.snia.org.

[119]

&ast;Sobti, S., Garg, N., Zhang, C., Yu, X., Krishnamurthy, A., and Wang, R. 2002. PersonalRAID: Mobile storage for distributed and disconnected computers. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 159--174.

Digital Library

[120]

&ast;Soules, C. A. N., Goodson, G. R., Strunk, J. D., and Ganger, G. R. 2003. Metadata efficiency in versioning file systems. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 43--58.

Digital Library

[121]

Spadavecchia, J. and Zadok, E. 2002. Enhancing NFS cross-administrative domain access. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, Monterey, CA, 181--194.

Digital Library

[122]

SPC. 2007. Storage performance council. www.storageperformance.org.

[123]

SPEC. 2001. SPEC SFS97_R1 V3.0. www.spec.org/sfs97r1.

[124]

SPEC. 2003. SPEC SMT97. www.spec.org/osg/smt97/.

[125]

SPEC. 2004. SPEC SDM Suite. www.spec.org/osg/sdm91/.

[126]

SPEC. 2005a. The SPEC organization. www.spec.org/.

[127]

SPEC. 2005b. SPECweb99. www.spec.org/web99.

[128]

SPEC. 2007. SPECviewperf 9. www.spec.org/gpc/opc.static/vp9info.html.

[129]

&ast;Stein, C. A., Howard, J. H., and Seltzer, M. I. 2001. Unifying file system protection. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 79--90.

Digital Library

[130]

&ast;Strunk, J. D., Goodson, G. R., Scheinholtz, M. L., Soules, C. A. N., and Ganger, G. R. 2000. Self-Securing storage: Protecting data in compromised systems. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 165--180.

Digital Library

[131]

Swartz, K. L. 1996. The brave little toaster meets Usenet. In Proceedings of the 10th USENIX System Administration Conference (LISA), Chicago, IL, 161--170.

Digital Library

[132]

&ast;Tan, Y., Wong, T., Strunk, J. D., and Ganger, G. R. 2005. Comparison-Based file server verification. In Proceedings of the Annual USENIX Technical Conference, Anaheim, CA, 121--133.

Digital Library

[133]

Tang, D. 1995. Benchmarking filesystems. Tech. Rep. TR-19-95, Harvard University.

[134]

Tang, D. and Seltzer, M. 1994. Lies, damned lies, and file system benchmarks. Tech. Rep. TR-34-94, Harvard University. December. In VINO: The 1994 Fall Harvest.

[135]

&ast;Thereska, E., Schindler, J., Bucy, J., Salmon, B., Lumb, C. R., and Ganger, G. R. 2004. A framework for building unobtrusive disk maintenance applications. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 213--226.

Digital Library

[136]

&ast;Tian, L., Feng, D., Jiang, H., Zhou, K., Zeng, L., Chen, J., Wang, Z., and Song, Z. 2007. PRO: A popularity-based multi-threaded reconstruction optimization for RAID-structured storage systems. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 277--290.

Digital Library

[137]

&ast;Tolia, N., Harkes, J., Kozuch, M., and Satyanarayanan, M. 2004. Integrating portable and distributed storage. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, San Francisco, CA, 227--238.

Digital Library

[138]

&ast;Tolia, N., Kozuch, M., Satyanarayanan, M., Karp, B., Bressoud, T., and Perrig, A. 2003. Opportunistic use of content addressable storage for distributed file systems. In Proceedings of the Annual USENIX Technical Conference, San Antonio, TX, 127--140.

[139]

TPC. 2005. Transaction processing performance council. www.tpc.org.

[140]

Tridgell, A. 1999. Dbench-3.03 README. http://samba.org/ftp/tridge/dbench/README.

[141]

Van Meter, R. 1997. Observing the effects of multi-zone disks. In Proceedings of the Annual USENIX Technical Conference, Anaheim, CA, 19--30.

Digital Library

[142]

&ast;Van Meter, R. and Gao, M. 2000. Latency management in storage systems. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 103--118.

Digital Library

[143]

&ast;Veeraraghavan, K., Myrick, A., and Flinn, J. 2007. Cobalt: Separating content distribution from authorization in distributed file systems. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 231--244.

Digital Library

[144]

VERITAS Software. 1999. VERITAS file server edition performance brief: A PostMark 1.11 benchmark comparison. Tech. Rep., Veritas Software Corporation. June. http://eval.veritas.com/webfiles/docs/fsedition-postmark.pdf.

[145]

VeriTest. 2002. NetBench. www.veritest.com/benchmarks/netbench/.

[146]

&ast;Vilayannur, M., Nath, P., and Sivasubramaniam, A. 2005. Providing tunable consistency for a parallel file store. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 17--30.

Digital Library

[147]

Vogels, W. 1999. File system usage in Windows NT 4.0. In Proceedings of the 17th ACM Symposium on Operating Systems Principles, Charleston, SC, 93--109.

Digital Library

[148]

&ast;Wachs, M., Abd-El-Malek, M., Thereska, E., and Ganger, G. R. 2007. Argon: Performance insulation for shared storage servers. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 61--76.

Digital Library

[149]

&ast;Wang, A. A., Reiher, P., Popek, G. J., and Kuenning, G. H. 2002. Conquest: Better performance through a disk/persistent-RAM hybrid file system. In Proceedings of the Annual USENIX Technical Conference, Monterey, CA, 15--28.

Digital Library

[150]

&ast;Wang, R. Y., Anderson, T. E., and Patterson, D. A. 1999. Virtual log based file systems for a programmable disk. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation, New Orleans, LA, 29--44.

Digital Library

[151]

&ast;Wang, Y. and Merchant, A. 2007. Proportional-Share scheduling for distributed storage systems. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 47--60.

Digital Library

[152]

Watson, A. and Nelson, B. 1992. LADDIS: A multi-vendor and vendor-neutral SPEC NFS benchmark. In Proceedings of the 6th USENIX Systems Administration Conference (LISA VI), Long Beach, CA, 17--32.

[153]

&ast;Weddle, C., Oldham, M., Qian, J., Wang, A. A., Reiher, P., and Kuenning, G. 2007. PARAID: A gear-shifting power-aware RAID. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 245--260.

Digital Library

[154]

&ast;Weil, S., Brandt, S., Miller, E., Long, D., and Maltzahn, C. 2006. Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Seattle, WA, 307--320.

Digital Library

[155]

Wittle, M. and Keith, B. E. 1993. LADDIS: The next generation in NFS file server benchmarking. In Proceedings of the Summer USENIX Technical Conference, Cincinnati, OH, 111--128.

Digital Library

[156]

Wright, C. P., Dave, J., and Zadok, E. 2003a. Cryptographic file systems performance: What you don't know can hurt you. In Proceedings of the 2nd IEEE International Security In Storage Workshop. IEEE Computer Society, Washington, DC, 47--61.

Digital Library

[157]

Wright, C. P., Joukov, N., Kulkarni, D., Miretskiy, Y., and Zadok, E. 2005. Auto-Pilot: A platform for system software benchmarking. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, Anaheim, CA, 175--187.

Digital Library

[158]

&ast;Wright, C. P., Martino, M., and Zadok, E. 2003b. NCryptfs: A secure and convenient cryptographic file system. In Proceedings of the Annual USENIX Technical Conference, San Antonio, TX, 197--210.

[159]

&ast;Yu, X., Gum, B., Chen, Y., Wang, R. Y., Li, K., Krishnamurthy, A., and Anderson, T. E. 2000. Trading capacity for performance. In Proceedings of the 4th USENIX Symposium on Operating System Design and Implementation, San Diego, CA, 243--258.

Digital Library

[160]

&ast;Yumerefendi, A. R. and Chase, J. S. 2007. Strong accountability for network storage. In Proceedings of the 5th USENIX Conference on File and Storage Technologies, San Jose, CA, 77--92.

Digital Library

[161]

&ast;Zadok, E. 2002. Overhauling Amd for the '00s: A case study of GNU autotools. In Proceedings of the Annual USENIX Technical Conference, FREENIX Track, Monterey, CA, 287--297.

Digital Library

[162]

&ast;Zadok, E., Anderson, J. M., Bădulescu, I., and Nieh, J. 2001. Fast indexing: Support for size-changing algorithms in stackable file systems. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 289--304.

Digital Library

[163]

&ast;Zadok, E., Bădulescu, I., and Shender, A. 1999. Extending file systems using stackable templates. In Proceedings of the Annual USENIX Technical Conference, Monterey, CA, 57--70.

Digital Library

[164]

&ast;Zadok, E. and Nieh, J. 2000. FiST: A language for stackable file systems. In Proceedings of the Annual USENIX Technical Conference, San Diego, CA, 55--70.

Digital Library

[165]

&ast;Zhang, C., Yu, X., Krishnamurthy, A., and Wang, R. Y. 2002. Configuring and scheduling an eager-writing disk array for a transaction processing workload. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 289--304.

Digital Library

[166]

&ast;Zhang, Z. and Ghose, K. 2003. yFS: A journaling file system design for handling large data sets with reduced seeking. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, San Francisco, CA, 59--72.

Digital Library

[167]

&ast;Zhou, Y., Philbin, J., and Li, K. 2001. The multi-queue replacement algorithm for second level buffer caches. In Proceedings of the Annual USENIX Technical Conference, Boston, MA, 91--104.

Digital Library

[168]

&ast;Zhu, N., Chen, J., and Chiueh, T. 2005a. TBBT: Scalable and accurate trace replay for file server evaluation. In Proceedings of the 4th USENIX Conference on File and Storage Technologies, San Francisco, CA, 323--336.

Digital Library

[169]

&ast;Zhu, Q., Chen, Z., Tan, L., Zhou, Y., Keeton, K., and Wilkes, J. 2005b. Hibernator: Helping disk arrays sleep through the winter. In Proceedings of the 20th ACM Symposium on Operating Systems Principles, Brighton, UK, 177--190.

Digital Library

Cited By

Chiang KLi YWang WShih WHo C(2024)Enhancing Data Integrity with Efficient Retention-Refilling Programming SchemesACM SIGAPP Applied Computing Review10.1145/3699839.369984224:3(37-50)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3699839.3699842
Tehrany NDoekemeijer KTrivedi AIbrahim SByna SZhou A(2024)zns-tools: An eBPF-powered, Cross-Layer Storage Profiling Tool for NVMe ZNS SSDsProceedings of the 4th Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems10.1145/3642963.3652205(23-32)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642963.3652205
Volpert SWinkelhofer SWesner SDomaschka JBalsamo SKnottenbelt WAbad CShang W(2024)An Empirical Analysis of Common OCI Runtimes' Performance Isolation CapabilitiesProceedings of the 15th ACM/SPEC International Conference on Performance Engineering10.1145/3629526.3645044(60-70)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3629526.3645044
Show More Cited By

Index Terms

A nine year study of file system and storage benchmarking
1. General and reference
  1. Cross-computing tools and techniques

Recommendations

A five-year study of file-system metadata

For five years, we collected annual snapshots of file-system metadata from over 60,000 Windows PC file systems in a large corporation. In this article, we use these snapshots to study temporal changes in file size, file age, file-type frequency, ...
A Novel Reordering Write Buffer to Improve Write Performance of Log-Structured File Systems

Abstract--This paper presents a novel reordering write buffer which improves the performance of Log-structured File Systems (LFS). While LFS has a good write performance, high garbage-collection overhead degrades its performance under high disk space ...
A multiple-file write scheme for improving write performance of small files in Fast File System

Fast File System (FFS) stores files to disk in separate disk writes, each of which incurs a disk positioning (seek + rotation) limiting the write performance for small files. We propose a new scheme called co-writing to accelerate small file writes in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 4, Issue 2

May 2008

80 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/1367829

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2008

Accepted: 01 November 2007

Revised: 01 November 2007

Received: 01 May 2007

Published in TOS Volume 4, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

178
Total Citations
View Citations
4,946
Total Downloads

Downloads (Last 12 months)133
Downloads (Last 6 weeks)17

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chiang KLi YWang WShih WHo C(2024)Enhancing Data Integrity with Efficient Retention-Refilling Programming SchemesACM SIGAPP Applied Computing Review10.1145/3699839.369984224:3(37-50)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3699839.3699842
Tehrany NDoekemeijer KTrivedi AIbrahim SByna SZhou A(2024)zns-tools: An eBPF-powered, Cross-Layer Storage Profiling Tool for NVMe ZNS SSDsProceedings of the 4th Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems10.1145/3642963.3652205(23-32)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642963.3652205
Volpert SWinkelhofer SWesner SDomaschka JBalsamo SKnottenbelt WAbad CShang W(2024)An Empirical Analysis of Common OCI Runtimes' Performance Isolation CapabilitiesProceedings of the 15th ACM/SPEC International Conference on Performance Engineering10.1145/3629526.3645044(60-70)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3629526.3645044
Chiang KLi YWang WShih WHong JPark J(2024)On Enhancing Data Integrity with Low-cost Retention-Refillable Programming SchemeProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635905(420-427)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635905
Bhat W(2024)Performance-Baseline Estimation of File System Operations for Linux-Based Edge DevicesIEEE Transactions on Industrial Informatics10.1109/TII.2024.336309020:5(7537-7544)Online publication date: May-2024
https://doi.org/10.1109/TII.2024.3363090
Zhang JJiang WTang BMa HCao LJiang ZNie YWang FZhang LLiang Y(2023)CDSBen: Benchmarking the Performance of Storage Services in Cloud-Native Database System at ByteDanceProceedings of the VLDB Endowment10.14778/3611540.361154916:12(3584-3596)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611549
Volpert SErb BEisenhart GSeybold DWesner SDomaschka JVieira MCardellini VDi Marco ATuma P(2023)A Methodology and Framework to Determine the Isolation Capabilities of Virtualisation TechnologiesProceedings of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578244.3583728(149-160)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3578244.3583728
Wang WHo CLi YChen LChang Y(2023)Reaping Both Latency and Reliability Benefits With Elaborate Sanitization Design for 3D TLC NAND FlashIEEE Transactions on Computers10.1109/TC.2023.327228072:11(3029-3041)Online publication date: Nov-2023
https://doi.org/10.1109/TC.2023.3272280
Chen FLi ZJiang CXiang TYang Y(2022)Cloud Object Storage Synchronization: Design, Analysis, and ImplementationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.318506733:12(4295-4310)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1109/TPDS.2022.3185067
Bhimani JMaruf AMi NPandurangan RBalakrishnan V(2022)Auto-Tuning Parameters for Emerging Multi-Stream Flash-Based Storage Drives Through New I/O Pattern GenerationsIEEE Transactions on Computers10.1109/TC.2020.304830371:2(309-322)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TC.2020.3048303
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents