Abstract
In multicluster systems, and more generally in grids, jobs may require co-allocation, i.e., the simultaneous allocation of resources such as processors in multiple clusters to improve their performance. In previous work, we have studied processor co-allocation through simulations. Here, we extend this work with the design and implementation of a dynamic processor co-allocation service in multicluster systems. While an implementation of basic co-allocation mechanisms has existed for some years in the form of the DUROC component of the Globus Toolkit, DUROC does not provide resource-brokering functionality or fault tolerance in the face of job submission or completion failures. Our design adds these two elements in the form of a software layer on top of DUROC. We have performed experiments that show that our co-allocation service works reliably.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Czajkowski, K., Foster, I., Kesselman, C.: Resource Co-Allocation in Computational Grids. In: Proc. of the 8th IEEE Int’l Symp. on High Performance Distributed Computing (HPDC-8), pp. 219–228 (1999)
Banen, S., Bucur, A., Epema, D.: A Measurement-Based Simulation Study of Processor Co-Allocation in Multicluster Systems. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 105–128. Springer, Heidelberg (2003)
The Globus Toolkit, http://www.globus.org
Bucur, A., Epema, D.: The Performance of Processor Co-Allocation in Multicluster Systems. In: Proc. of the 3rd IEEE/ACM Int’l Symp. on Cluster Computing and the GRID (CCGrid 2003), pp. 302–309. IEEE Computer Society Press, Los Alamitos (2003)
Bucur, A., Epema, D.: Trace-Based Simulations of Processor Co-Allocation Policies in Multiclusters. In: Proc. of the 12th IEEE Int’l Symp. on High Performance Distributed Computing (HPDC-12), pp. 70–79. IEEE Computer Society Press, Los Alamitos (2003)
The Portable Batch System, http://www.openpbs.org
Lifka, D.: The ANL/IBM SP Scheduling Systems. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)
The Distributed ASCI Supercomputer (DAS), http://www.cs.vu.nl/das2
Kumar, V., Grama, A., Gupta, A., Karypis, G.: Introduction to Parallel Computing. Benjamin/Cummings (1994)
Frey, J., Tannenbaum, T., Foster, I., Livny, M., Tuecke, S.: Condor-G: A Computation Management Agent for Multi-Institutional Grids. In: Proc. of the 10th IEEE Symp. on High Performance Distributed Computing (HPDC-10), pp. 7–9. IEEE Computer Society Press, Los Alamitos (2001)
Raman, R., Livny, M., Solomon, M.: Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching. In: Proc. of the 12th IEEE Int’l Symp. on High Performance Distributed Computing (HPDC-12), pp. 80–89. IEEE Computer Society Press, Los Alamitos (2003)
Deelman, E., et al.: Mapping Abstract Complex Workflows onto Grid Environments. J. of Grid Computing 1, 25–39 (2003)
Deelman, E., et al.: Pegasus: Mapping Scientific Workflows onto the Grid. In: Proc. of the 2nd European Across Grids Conference (2004)
Foster, I., Vockler, J., Wilde, M., Zhao, Y.: Chimera: A Virtual Data System for Representing, Querying, and Automating Data Derivation. In: 14th Int’l Conf. on Scientific and Statistical Database Management, SSDBM 2002 (2002)
Ernemann, C., Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On Advantages of Grid Computing for Parallel Job Scheduling. In: Proc. of the 2nd IEEE/ACM Int’l Symp. on Cluster Computing and the GRID (CCGrid 2002), pp. 39–46 (2002)
Ernemann, C., Hamscher, V., Streit, A., Yahyapour, R.: Enhanced Algorithms for Multi-Site Scheduling. In: 3rd Int’l Workshop on Grid Computing, pp. 219–231 (2002)
Bucur, A., Epema, D.: The Maximal Utilization of Processor Co-Allocation in Multicluster Systems. In: Proc. of the Int’l Parallel and Distributed Processing Symp (IPDPS), pp. 60–69. IEEE Computer Society Press, Los Alamitos (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sinaga, J.M.P., Mohamed, H.H., Epema, D.H.J. (2005). A Dynamic Co-allocation Service in Multicluster Systems. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2004. Lecture Notes in Computer Science, vol 3277. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11407522_11
Download citation
DOI: https://doi.org/10.1007/11407522_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25330-3
Online ISBN: 978-3-540-31795-1
eBook Packages: Computer ScienceComputer Science (R0)