Abstract
Grids enable sharing resources of heterogeneous nature and administration. In such distributed systems, the network is usually taken for granted which is potentially problematic due to the complexity and unpredictability of public networks that typically underlie grids. This article introduces GridMAP, a mechanism for considering the network state for enhancing grid scheduling. Network measurements are collected in a passive manner from a user-centric vantage point. This mechanism has been evaluated on a production e-science grid infrastructure, with results showing the ability of GridMAP to improve grid scheduling with minimal network, computational and deployment overheads.
Similar content being viewed by others
References
Akella, A., Seshan, S., Shaikh, A.: An empirical valuation of wide-area internet bottlenecks. In: Proceedings of ACM SIGCOMM IMC, pp. 101–114 (2003)
Cisco Visual Networking Index: Global mobile data traffic forecast update, 20122017. Tech. rep., Cisco Press (2013)
Lee, C., Lee, D.K., Yi, Y., Moon, S.: Operating a Network Link at 100 %. In: Proc. PAM, pp. 1–10 (2011)
Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. The International Journal of Supercomputer Applications and High Performance Computing 11(2), 115–128 (1997)
Berman, F.: The grid: blueprint for a new computing infrastructure, chap. High-Performance Schedulers, pp. 279–309. Morgan Kaufmann (1998)
Batista, D.M., da Fonseca, N.L.S.: A survey of self-adaptive grids. IEEE Commun. Mag. 48(7), 94–100 (2010)
Qureshi, M.B., Dehnavi, M.M., Min-Allah, N., Qureshi, M.S., Hussain, H., Rentifis, I., Tziritas, N., Loukopoulos, T., Khan, S.U., Xu, C.Z., et al.: Survey on grid resource allocation mechanisms. J. Grid. Comput., 1–43 (2014)
Agarwal, A., Kumar, P.: Multidimensional QoS Oriented Task Scheduling in Grid Environments. International Journal of Grid Computing & Applications (IJGCA) 2(1), 28–37 (2011)
Tomás, L., Caminero, A.C., Carrión, C., Caminero, B.: Network-aware meta-scheduling in advance with autonomous self-tuning system. Future Gener. Comp. Sys. 27(5), 486–497 (2011)
Chinnaiah, V., Somasundaram, T.S.: A grid resource brokering strategy based on resource and network performance in grid. Future Gener. Comp. Sys. 28(3), 491–499 (2012)
McClatchey, R., Anjum, A., Stockinger, H., Ali, A., Willers, I., Thomas, M.: Data intensive and network aware (DIANA) grid scheduling. J Grid Comput 5(1), 43–64 (2007)
Pezaros, D., Sifalakis, M., Schmid, S., Hutchison, D.: Dynamic link measurements using active components. In: Proceedings of IFIP conference on active networks (2004)
He, Q., Dovrolis, C., Ammar, M.: On the predictability of large transfer TCP throughput. Comput. Netw. 51(14), 3959–3977 (2007)
Lowekamp, B.B.: Combining active and passive network measurements to build scalable monitoring systems on the grid. ACM SIGMETRICS Performance Evaluation Review 30(4), 19–26 (2003)
Wolski, R.: Dynamically forecasting network performance using the network weather service. Clust. Comput. 1(1), 119–132 (1998)
Vazhkudai, S., Schopf, J.M., Foster, I.T.: Predicting the performance of wide area data transfers. In: Proceedings of IPDPS, p. 270 (2002)
Yousaf, M.M., Welzl, M., Junaid, M.M.: Fog in the network weather service: a case for novel approaches. In: Proceedings of conference on networks for grid applied, pp. 1–6 (2007)
Goga, O., Teixeira, R.: Speed measurements of residential internet access. In: Proceedings PAM, pp. 168–178 (2012)
Huang, A.C., Steenkiste, P.: Network-sensitive service discovery. J. Grid. Comput. 1(3), 309–326 (2003)
Madhyastha, H.V., Isdal, T., Piatek, M., Dixon, C., Anderson, T., Krishnamurthy, A., Venkataramani, A.: iPlane: an information plane for distributed services. In: Proceedings of USENIX OSDI, pp. 367–380 (2006)
Hullár, B., Laki, S., Stéger, J., Csabai, I., Vattay, G.: SONoMA: a service oriented network measurement architecture. Tech. Rep. TR-ELTE-CNL-2010/1. Eötvös Loránd University, Hungary (2010)
Wenwei, L., Dafang, Z., Jinmin, Y., Gaogang, X.: On evaluating the differences of TCP and ICMP in network measurement. Comput. Commun. 30(2), 428–439 (2007)
Paxson, V.E.: Measurements and analysis of end-to-end internet dynamics. Ph.D. thesis, University of California at Berkeley (1998)
Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.J.: WOW: self-organizing wide area overlay networks of virtual workstations. In: Proceedings of IEEE HPDC, pp. 30–42 (2006)
Raicu, I., Dumitrescu, C., Ripeanu, M., Foster, I.: The design, performance, and use of diperf: An automated distributed performance evaluation framework. J. Grid. Comput. 4(3), 287–309 (2006)
Zhang, X., Freschl, J.L., Schopf, J.M.: Scalability analysis of three monitoring and information systems: MDS2, R-GMA, and Hawkeye. J Parallel Distrib. Comput. 67(8), 883–902 (2007)
Fitzgerald, S., Foster, I., Kesselman, C., von Laszewski, G., Smith, W., Tuecke, S.: A directory service for configuring high-performance distributed computations. In: Proceedings of IEEE HPDC, pp. 365–375 (1997)
Cooke, A.W., Gray, A.J., Nutt, W., Magowan, J., Oevers, M., Taylor, P., Cordenonsi, R., Byrom, R., Cornwall, L., Djaoui, A., et al.: The relational grid monitoring architecture: mediating information about the grid. J. Grid. Comput. 2(4), 323–339 (2004)
Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A grid monitoring architecture. Tech. rep., Global Grid Forum (2002)
Zanikolas, S., Sakellariou, R.: A taxonomy of grid monitoring systems. Future Gener. Comp. Sys. 21(1), 163–188 (2005). doi:10.1016/j.future.2004.07.002
Lee, S.: eTOP: End-to-end Performance Measurement Infrastructure on KOREN and APAN Links., http://master.apan.net/meetings/busan03/measurement/koren.ppt (2003)
Sacerdoti, F.D., Katz, M.J., Massie, M.L., Culler, D.E.: Wide area cluster monitoring with ganglia. In: Proceedings of IEEE CLUSTER, pp. 289–298 (2003)
Al-Ali, R.J., Amin, K., Von Laszewski, G., Rana, O.F., Walker, D.W., Hategan, M., Zaluzec, N.: Analysis and provision of qos for distributed grid applications. J. Grid. Comput. 2(2), 163–182 (2004)
Elkhatib, Y.: Monitoring, analysing and predicting network performance in grids. Ph.D. thesis, Lancaster University (2011)
Elkhatib, Y., Edwards, C.: A survey-based study of grid traffic. In: Proceedings of conference on networks for grid applications (2007)
Savage, S.: Sting: A TCP-based network measurement tool. In: Proceedings of USENIX symposium on internet technologies and systems, vol. 2 (1999)
Horneffer, M.: Assessing Internet Performance Metrics Using Large-Scale TCP-SYN based Measurements. In: Proceedings of PAM (2000)
Jiang, H., Dovrolis, C.: Passive estimation of TCP round-trip times. ACM SIGCOMM Comput. Commun. Rev. 32(3), 75–88 (2002)
Elkhatib, Y., Edwards, C., Mackay, M., Tyson, G.: Providing grid schedulers with passive network measurements. In: Proceedings of ICCCN (2009)
Network Measurement Working Group. http://nmwg.internet2.edu
Akima, H.: A new method of interpolation and smooth curve fitting based on local procedures. J. ACM 17(4), 589–602 (1970)
White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., Joglekar, A.: An integrated experimental environment for distributed systems and networks. In: Proceedings of USENIX OSDI, pp. 255–270 (2002)
Marco, C., Fabio, C., Alvise, D., Antonia, G., Francesco, G., Alessandro, M., Moreno, M., Salvatore, M., Fabrizio, P., Luca, P., et al.: The gLite workload management system. In: Advances in grid and pervasive computing, pp. 256–268. Springer (2009)
AutoDock, the scripps research institute. http://autodock.scripps.edu/
Case, D.A., Cheatham, T.E., Darden, T., Gohlke, H., Luo, R., Merz, K.M., Onufriev, A., Simmerling, C., Wang, B., Woods, R.J.: The amber biomolecular simulation programs. J. Comput. Chem. 26(16), 1668–1688 (2005)
Jacq, N., Salzemann, J., Jacq, F., Legré, Y., Medernach, E., Montagnat, J., Maaß, A., Reichstadt, M., Schwichtenberg, H., Sridhar, M., Kasam, V., Zimmermann, M., Hofmann, M., Breton, V.: Grid-enabled virtual screening against malaria. J Grid Comp 6(1), 29–43 (2008)
Kasam, V., Salzemann, J., Jacq, N., Maaß, A., Breton, V.: Large scale deployment of molecular docking application on computational grid infrastructures for combating malaria. In: Proceedings CCGrid, pp. 691–700 (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Elkhatib, Y., Edwards, C. Passive Network Awareness as a Means for Improved Grid Scheduling. J Grid Computing 13, 275–291 (2015). https://doi.org/10.1007/s10723-015-9332-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-015-9332-0