[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Design and implementation of a workflow-based resource broker with information system on computational grids

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The grid is a promising infrastructure that can allow scientists and engineers to access resources among geographically distributed environments. Grid computing is a new technology which focuses on aggregating resources (e.g., processor cycles, disk storage, and contents) from a large-scale computing platform. Making grid computing a reality requires a resource broker to manage and monitor available resources. This paper presents a workflow-based resource broker whose main functions are matching available resources with user requests and considering network information statuses during matchmaking in computational grids. The resource broker provides a graphic user interface for accessing available and the appropriate resources via user credentials. This broker uses the Ganglia and NWS tools to monitor resource status and network-related information, respectively. Then we propose a history-based execution time estimation model to predict the execution time of parallel applications, according to previous execution results. The experimental results show that our model can accurately predict the execution time of embarrassingly parallel applications. We also report on using the Globus Toolkit to construct a grid platform called the TIGER project that integrates resources distributed across five universities in Taichung city, Taiwan, where the resource broker was developed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allcock B, Bester J, Bresnahan J, Chervenak AL, Foster I, Kesselman C, Meder S, Nefedova V, Quesnal D, Tuecke S (2002) Data management and transfer in high performance computational grid environments. Parallel Comput 28(5):749–771

    Article  Google Scholar 

  2. Allcock B, Bester J, Bresnahan J, Chervenak AL, Liming L, Meder S, Tuecke S (2002) GridFTP protocol specification. GGF GridFTP working group document

  3. Allcock B, Tuecke S, Foster I, Chervenak A, Kesselman C (2000) Protocols and services for distributed data-intensive science. In: ACAT2000 proceedings, 2000, pp 161–163

  4. Allcock W, Bresnahan J, Foster I, Liming L, Link J, Plaszczac P (2002) GridFTP update January 2002, Technical Report, 2002. Available at http://www-fp.globus.org/datagrid/deliverables/GridFTP-Overview-200201.pdf

  5. Buyya R (1999) Deploying a high throughput computing cluster. In: High performance cluster computing, vol 1. Prentice Hall PTR, Englewood Cliffs

    Google Scholar 

  6. Chervenak A, Foster I, Kesselman C, Salisbury C, Tuecke S (2001) The data grid: towards an architecture for the distributed management and analysis of large scientific datasets. J Netw Comput Appl 23:187–200

    Article  Google Scholar 

  7. Czajkowski K, Fitzgerald S, Foster I, Kesselman C (2001) Grid information services for distributed resource sharing. In: Proceedings of the 10th IEEE international symposium on high-performance distributed computing (HPDC-10’01), August 2001, pp 181–194

  8. Foster I, Kesselman C (2003) The grid 2: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann, San Mateo, ISBN: 1558609334

    Google Scholar 

  9. Foster I (2002) The grid: a new infrastructure for 21st century science. Phys Today 55(2):42–47

    Article  Google Scholar 

  10. Foster I, Karonis NT (1998) A Grid-Enabled MPI: message passing in heterogeneous distributed computing systems. In: Proceedings of 1998 supercomputing conference, 1998

  11. Foster I, Kesselman C (1997) Globus: a metacomputing infrastructure toolkit. Int J Supercomput Appl 11(2):115–128

    Google Scholar 

  12. Ferreira L, Berstis V, Armstrong J, Kendzierski M, Neukoetter A, Takagi M, Bing-Wo R, Amir A, Murakawa R, Hernandez O, Magowan J, Bieberstein N (2003) Introduction to grid computing with globus. IBM Redbooks Press, Raleigh. Available at http://www.ibm.com/redbooks

    Google Scholar 

  13. Java CoG. http://www-unix.globus.org/cog/

  14. Yang CT, Lai CL, Li KC, Hsu CH, Chu WC (2005) On utilization of the grid computing technology for video conversion and 3D rendering. In: Parallel and distributed processing and applications: third international symposium, ISPA 2005. Lecture notes in computer science, vol 3758. Springer, Berlin, pp 442–453

    Google Scholar 

  15. Laszewski V, Foster I, Gawor J, Lane P (2001) A Java commodity grid kit. Concurr Comput Pract Exp 13:645–662

    Article  MATH  Google Scholar 

  16. Le H, Coddington P, Wendelborn AL (2004) A data-aware resource broker for data grids. In: IFIP international conference on network and parallel computing (NPC’2004). LNCS, vol 3222. Springer, Berlin

    Google Scholar 

  17. Yang CT, Shih PC, Li KC (2005) A high-performance computational resource broker for grid computing environments. In: Proceedings of the international conference on AINA’05, Taipei, Taiwan, March 2005, vol 2, pp 333–336

  18. Yang CT, Li KC, Chiang WC, Shih PC (2005) Design and implementation of TIGER grid: an integrated metropolitan-scale grid environment. In: Proceedings of the 6th IEEE international conference on PDCAT’05, Dec 2005, pp 518–520

  19. Nabrzyski J, Schopf JM, Weglarz J (2005) Grid resource management. Kluwer Academic, Dordrecht

    Google Scholar 

  20. Park SM, Kim JH (2003) Chameleon: a resource scheduler in a data grid environment. In: Proceedings of the 3rd IEEE/ACM international symposium on cluster computing and the grid, May 2003, pp 258–265

  21. Yang CT, Lai CL, Shih PC, Li KC (2004) A resource broker for computing nodes selection in grid environments. In: Grid and cooperative computing—GCC 2004: 3rd international conference. Lecture notes in computer science, vol 3251. Springer, Berlin, pp 931–934

    Google Scholar 

  22. Yang CT, Shih PC, Chen SY, Shih WC (2005) An efficient network information modeling using NWS for grid computing environments. In: Grid and cooperative computing—GCC 2005: 4th international conference. Lecture notes in computer science, vol 3795. Springer, Berlin, pp 287–299

    Google Scholar 

  23. Network Weather Service. http://nws.cs.ucsb.edu/ewiki/

  24. Ganglia. http//ganglia.sourceforge.net/

  25. TIGER. http://gamma2.hpc.csie.thu.edu.tw/ganglia/

  26. Aloisio G, Cafaro M (2002) Web-based access to the grid using the grid resource broker portal. Concurr Comput Pract Exp 14:1145–1160

    Article  MATH  Google Scholar 

  27. Krauter K, Buyya R, Maheswaran M (2002) A taxonomy and survey of grid resource management systems for distributed computing. Softw Pract Exp 32:135–164

    Article  MATH  Google Scholar 

  28. Rodero I, Corbalán J, Badia RM, Labarta J (2005) In: eNANOS grid resource broker. LNCS, vol 3470. Springer, Berlin, pp 111–121

    Google Scholar 

  29. Venugopal S, Buyya R, Winton L (2006) A grid service broker for scheduling e-science applications on global data grids. Concurr Comput Pract Exp 18:685–699

    Article  Google Scholar 

  30. Aloisio G, Cafaro M, Carteni G, Epicoco I, Fiore S, Lezzi D, Mirto M, Mocavero S (2007) The grid resource broker portal. Concurr Comput Pract Exp 19(12):1663–1670

    Article  Google Scholar 

  31. Cafaro M, Epicoco I, Mirto M, Lezzi D, Aloisio G (2007) The grid resource broker workflow engine. In: Proceedings of the sixth international conference on grid and cooperative computing (GCC 2007), 2007

  32. Deelman E, Singh G, Su M, Blythe J, Gil Y, Kesselman C, Mehta G, Vahi K, Berriman GB, Good J, Laity A, Jacob JC, Katz DS (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program J 13(3):219–237

    Google Scholar 

  33. Shah SP, He DYM, Sawkins JN, Druce JC, Quon G, Lett D, Zheng GXY, Xu T, Quellette BFF (2004) Pegasys: software for executing and integrating analyses of biological sequences. BMC Bioinform 5:40

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. High-Performance Computing Laboratory, Department of Computer Science and Information Engineering, Tunghai University, Taichung, 40704, Taiwan, Republic of China

    Chao-Tung Yang & Po-Chi Shih

  2. Department of Computer and Information Science, National TaiChung University, No. 140, Ming-Sheng Rd., Taichung, Taiwan, Republic of China

    Kuan-Chou Lai

Authors
  1. Chao-Tung Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kuan-Chou Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Po-Chi Shih
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Tung Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, CT., Lai, KC. & Shih, PC. Design and implementation of a workflow-based resource broker with information system on computational grids. J Supercomput 47, 76–109 (2009). https://doi.org/10.1007/s11227-008-0201-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-008-0201-5

Keywords

Search

Navigation