Abstract
There is emerging interest in many scientific disciplines to deal with “dynamic” data, arising from sensors and scientific instruments, which require workflow graphs that can be dynamically adapted – as new data becomes available. Additionally, the elastic nature of many Cloud environments subsequently enable such dynamic workflow graphs to be enacted more efficiently. One of the challenges of scientific workflows is that they must be designed with the needed level of dynamism to take account of the availability of data and the variability of the execution environment, which can be dynamically scaled out based on demand (and budget). In this paper, we present a novel approach for specifying scientific workflows with the two main requirements of: (i) dynamic / adaptive workflow structure well suited for and responsive to change, and (ii) support for large-scale and variable parallelism. We utilise the superscalar pipeline as a model of computation and the well-known Montage workflow for illustrating our approach.
This work has been supported by the research project TIN2010-17905, granted by the Spanish Ministry of Education and Science.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Zinn, D., Hart, Q., McPhillips, T., Ludaescher, B., Simmhan, Y., Giakkoupis, M., Prasanna, V.K.: Towards reliable, performant workflows for streaming-applications on cloud platforms. In: 11st International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2011), Newport Beach, USA (2011)
Pautasso, C., Alonso, G.: Parallel computing patterns for Grid workflows. In: Proceedings of the HPDC 2006 Workshop on Workflows in Support of Large-Scale Science (WORKS 2006), Paris, France (June 19-23, 2006)
Kim, H., el-Khamra, Y., Jha, S., Parashar, M.: An autonomic approach to integrated hpc grid and cloud usage. In: Proceedings of 5 th IEEE International Conference on e-Science (e-Science 2009), Oxford, UK (2009)
Abramson, D., Enticott, C., Altintas, I.: Nimrod/k: towards massively parallel dynamic grid workflows. In: Proceedings of the ACM/IEEE Conference on High Performance Computing, SC 2008. IEEE/ACM, Austin, Texas, USA (2008)
Kummer, O., Wienberg, F., Duvigneau, M., Schumacher, J., Köhler, M., Moldt, D., Rölke, H., Valk, R.: An Extensible Editor and Simulation Engine for Petri Nets: Renew. In: Cortadella, J., Reisig, W. (eds.) ICATPN 2004. LNCS, vol. 3099, pp. 484–493. Springer, Heidelberg (2004)
Tolosana-Calasanz, R., Bañares, J.A., Rana, O.F., Álvarez, P., Ezpeleta, J., Hoheisel, A.: Adaptive exception handling for scientific workflows. Concurr. Comput.: Pract. Exper. 22, 617–642 (2010)
Tolosana-Calasanz, R., Bañares, J.A., Álvarez, P., Ezpeleta, J.: Vega: A service-oriented grid workflow management system. In: Meersman, R., Tari, Z. (eds.) OTM 2007, Part II. LNCS, vol. 4804, pp. 1516–1523. Springer, Heidelberg (2007)
Deelman, E., Mehta, G., Singh, G., Su, M., Vahi, K.: Pegasus: Mapping Large-Scale Workflows to Distributed Resources. In: Workflows for eScience, pp. 376–394. Springer, Heidelberg (2007)
van der Aalst, W., Moldt, D., Rolke, H.: Pattern based workflow design using reference nets. In: van der Aalst, W.M.P., ter Hofstede, A.H.M., Weske, M. (eds.) BPM 2003. LNCS, vol. 2678, pp. 246–260. Springer, Heidelberg (2003)
Pautasso, C., Heinis, T., Alonso, G.: Autonomic execution of Web service compositions. In: 2005 IEEE International Conference on Web Services (ICWS 2005), July 11-15, pp. 435–442. IEEE Computer Society, Orlando (2005)
Tolosana-Calasanz, R., Bañares, J.A., Rana, O.F.: Autonomic streaming pipeline for scientific workflows. Concurr. Comput.: Pract. Exper. 1–31 (to appear, 2011)
Taylor, I., Shields, M., Wang, I., Harrison, A.: The Triana Workflow Environment: Architecture and Applications. In: Workflows for eScience, pp. 320–339. Springer, Heidelberg (2007)
Pautasso, C., Alonso, G.: The jopera visual composition language. J. Vis. Lang. Comput. 16, 119–152 (2005)
Ngu, A., Bowers, S., Haasch, N., McPhillips, T., Critchlow, T.: Flexible scientific workflow modeling using frames, templates, and dynamic embedding. Scientific and Statistical Database Management, 566–572 (2008)
Oinn, T., Greenwood, M., Addis, M., Alpdemir, M.N., Ferris, J., Glover, K., Goble, C., Goderis, A., Hull, D., Marvin, D., Li, P., Lord, P., Pocock, M.R., Senger, M., Stevens, R., Wipat, A., Wroe, C.: Taverna: lessons in creating a workflow environment for the life sciences: Research articles. Concurr. Comput.: Pract. Exper. 18, 1067–1100 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tolosana-Calasanz, R., Bañares, J.A., Rana, O.F. (2011). Dynamic Workflow Adaptation over Adaptive Infrastructures. In: O’Shea, J., Nguyen, N.T., Crockett, K., Howlett, R.J., Jain, L.C. (eds) Agent and Multi-Agent Systems: Technologies and Applications. KES-AMSTA 2011. Lecture Notes in Computer Science(), vol 6682. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22000-5_68
Download citation
DOI: https://doi.org/10.1007/978-3-642-22000-5_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21999-3
Online ISBN: 978-3-642-22000-5
eBook Packages: Computer ScienceComputer Science (R0)