Abstract
We are developing a distributed computing environment based on virtual machines featuring application monitoring, network monitoring, and an adaptive virtual network. In this paper, we describe our initial results in monitoring the communication traffic of parallel applications, and inferring its spatial communication properties. The ultimate goal is to be able to exploit such knowledge to maximize the parallel efficiency of the running parallel application by using VM migration, virtual overlay network configuration and network reservation techniques, which are a part of the distributed computing environment. Specifically, we demonstrate that: (1) we can monitor the parallel application network traffic in our layer 2 virtual network system with very low overhead, (2) we can aggregate the monitoring information captured on each host machine to form a global picture of the parallel application’s traffic load matrix, and (3) we can infer from the traffic load matrix the application topology. In earlier work, we have demonstrated that we can capture the time dynamics of the applications. We begin here by considering offline traffic monitoring and inference as a proof of concept, testing it with a variety of synthetic and actual workloads. Next, we describe the design and implementation of our online system, the Virtual Topology and Traffic Inference Framework (VTTIF), and evaluate it using a NAS benchmark.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bailey, D.H., Barszcz, E., Barton, J.T., Browning, D.S., Carter, R.L., Dagum, D., Fatoohi, R.A., Frederickson, P.O., Lasinski, T.A., Schreiber, R.S., Simon, H.D., Venkatakrishnan, V., Weeratunga, S.K.: The NAS Parallel Benchmarks. The International Journal of Supercomputer Applications 5(3), 63–73 (Fall 1991)
Dilger, A., Flierl, J., Begg, L., Grove, M., Dispot, F.: The PVM patch for POV-Ray, Available at http://pvmpov.sourceforge.net
Dinda, P.A., Garcia, B., Leung, K.S.: The measured network traffic of compilerparallelized programs. In: Proceedings of the 30th International Conference on Parallel Processing (ICPP 2001), September 2001, pp. 175–184 (2001)
Figueiredo, R., Dinda, P.A., Fortes, J.: A case for grid computing on virtual machines. In: Proceedings of the 23rd International Conference on Distributed Computing Systems (ICDCS 2003) (May 2003)
Geist, A., Beguelin, A., Dongarra, J., Weicheng, J., Mancheck, R., Sunderam, V.: PVM: Parallel Virtual Machine. MIT Press, Cambridge (1994)
Gerbessiotis, A.V., Valiant, L.G.: Direct bulk-synchronous parallel algorithms. Journal of Parallel and Distributed Computing 22(2), 251–267 (1994)
Rolia, J., Jim Pruyne, X.Z., Arlitt, M.: Grids for Enterprise Applications. In: Proceedings of the 9th Workshops on Job Scheduling Strategies for Parallel Processing (JSSPS 2003) (June 2003)
Leighton, F.T.: Introductio to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes. Morgan Kaufmann, San Francisco (1992)
Lowekamp, B., Beguelin, A.E.: Efficient collective operations for communication on heterogeneous networks. In: Proceedings of the International Parallel Processing Symposium (IPPS 1996), pp. 399–405 (1996)
Savage, S., Collins, A., Hoffman, E., Snell, J., Anderson, T.E.: The endto-end effects of internet path selection. In: SIGCOMM, pp. 289–299 (1999)
Stasko, J.: Samba Algorithm Animation System, Available at http://www.cc.gatech.edu/gvu/softviz/algoanim/samba.html
Sundararaj, A., Dinda, P.: Towards virtual networks for virtual machine grid computing. In: Proceedings of the 3rd USENIX Virtual Machine Research And Technology Symposium (VM 2004) (May 2004) (to appear); Earlier version available as Technical Report NWU-CS-03-27, Department of Computer Science, Northwestern University
White, S., Alund, A., Sunderam, V.S.: Performance of the NAS parallel benchmarks on PVM-Based networks. Journal of Parallel and Distributed Computing 26(1), 61–71 (1995)
Zhang, Y., Du, N., Paxson, E., Shenker, S.: The constancy of internet path properties (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gupta, A., Dinda, P.A. (2005). Inferring the Topology and Traffic Load of Parallel Programs Running in a Virtual Machine Environment. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2004. Lecture Notes in Computer Science, vol 3277. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11407522_7
Download citation
DOI: https://doi.org/10.1007/11407522_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25330-3
Online ISBN: 978-3-540-31795-1
eBook Packages: Computer ScienceComputer Science (R0)