More Web Proxy on the site http://driver.im/

Article

Free access

Runtime Dynamic Structural Changes of Scientific Workflows in Clouds

Authors:

Igor Dos Santos,

Daniel De Oliveira,

Eduardo Ogasawara,

Marta MattosoAuthors Info & Claims

UCC '13: Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing

Pages 417 - 422

https://doi.org/10.1109/UCC.2013.83

Published: 09 December 2013 Publication History

Abstract

Existing Scientific Workflow Management Systems (i.e. SWfMS) effectively support workflows that do not need dynamic changes at runtime. SWfMS execute workflows by providing process management, provenance data and distributed execution on clusters and clouds. However, the support for dynamic changes in workflows is still an open, yet important, problem. For example, when the program associated to an activity of the workflow is taking more time than expected to produce results or if the results do not comply with some quality criteria, the scientist may want to try an alternative algorithm implementation. However, scientists may not want to re-execute the entire workflow for each change they make in the workflow structure. Alternatively, changing the structure of the workflow dynamically (i.e. change the programs associated with workflow activities) can improve the workflow execution without restarting the execution. In this paper, we propose DynAdapt, a dynamic approach that allows scientist to perform structural changes of the workflow specification at runtime. We evaluated DynAdapt using SciPhy, a large-scale bioinformatics workflow, and results show up to 40% of execution improvement.

References

[1]

E. Deelman, D. Gannon, M. Shields, and I. Taylor, 2009, Workflows and e-Science: An overview of workflow system features and capabilities, Future Generation Computer Systems, v. 25, n. 5, p. 528-540.

Digital Library

[2]

I. Foster and C. Kesselman, 2004, The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann.

Digital Library

[3]

L.M. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, 2009, A break in the clouds: towards a cloud definition, SIGCOMM Comput. Commun. Rev., v. 39, n. 1, p. 50-55.

Digital Library

[4]

Y.L. Simmhan, B. Plale, and D. Gannon, 2005, A survey of data provenance in e-science, ACM SIGMOD Record, v. 34, n. 3, p. 31-36.

Digital Library

[5]

M. Wilde, M. Hategan, J.M. Wozniak, B. Clifford, D.S. Katz, and I. Foster, 2011, Swift: A language for distributed parallel scripting, Parallel Computing, n. 37(9), p. 633-652.

Digital Library

[6]

E. Deelman, G. Mehta, G. Singh, M.-H. Su, and K. Vahi, 2007, "Pegasus: Mapping Large-Scale Workflows to Distributed Resources", Workflows for e-Science, Springer, p. 376-394.

[7]

D. Oliveira, E. Ogasawara, F. Baiao, and M. Mattoso, 2010, SciCumulus: A Lightweight Cloud Middleware to Explore Many Task Computing Paradigm in Scientific Workflows, In: 3rd International Conference on Cloud Computing, p. 378-385.

Digital Library

[8]

Y. Gil, E. Deelman, M. Ellisman, T. Fahringer, G. Fox, D. Gannon, C. Goble, M. Livny, L. Moreau, et al., 2007, Examining the Challenges of Scientific Workflows, Computer, v. 40, n. 12, p. 24-32.

Digital Library

[9]

P. Missier, S. Soiland-Reyes, S. Owen, W. Tan, A. Nenadic, I. Dunlop, A. Williams, T. Oinn, and C. Goble, 2010, Taverna, reloaded, In: Proceedings of the 22nd international conference on Scientific and statistical database management, p. 471-481.

Digital Library

[10]

S. Bowers, B. Ludascher, A.H.H. Ngu, and T. Critchlow, 2006, Enabling Scientific Workflow Reuse through Structured Composition of Dataflow and Control-Flow, In: Proceedings of the 22nd International Conference on Data Engineering Workshops, p. 70.

Digital Library

[11]

T. Samak, D. Gunter, M. Goode, E. Deelman, G. Mehta, F. Silva, and K. Vahi, 2011, Failure prediction and localization in large scientific workflows, In: Proceedings of the 6th workshop on Workflows in support of large-scale science, p. 107-116.

Digital Library

[12]

J. Dias, E. Ogasawara, D. Oliveira, F. Porto, A. Coutinho, and M. Mattoso, 2011, Supporting Dynamic Parameter Sweep in Adaptive and User-Steered Workflow, In: 6th Workshop on Workflows in Support of Large-Scale Science, p. 31-36.

Digital Library

[13]

K.A.C.S. Ocana, D. Oliveira, E. Ogasawara, A.M.R. Davila, A.A.B. Lima, and M. Mattoso, 2011, "SciPhy: A Cloud-Based Workflow for Phylogenetic Analysis of Drug Targets in Protozoan Genomes", In: O. Norberto de Souza, G.P. Telles, and M. Palakal, eds., Advances in Bioinformatics and Computational Biology, chapter 6832, Berlin, Heidelberg: Springer, p. 66-70.

Digital Library

[14]

A. Alqaoud, I. Taylor, and A. Jones, 2010, Scientific workflow interoperability framework, International Journal of Business Process Integration and Management, v. 5, n. 1 (Jan.), p. 93-105.

[15]

P.J. Kammer, G.A. Bolcer, R.N. Taylor, A.S. Hitomi, and M. Bergman, 2000, Techniques for Supporting Dynamic and Adaptive Workflow, Computer Supported Cooperative Work (CSCW), v. 9, n. 3-4, p. 269-292.

Digital Library

[16]

C. Ellis, K. Keddara, and G. Rozenberg, 1995, Dynamic change within workflow systems, p. 10-21.

Digital Library

[17]

M. Dabrowski, M. Drabik, M. Trzaska, and K. Subieta, Dynamic Changes of Workflow Processes.

[18]

F. Costa, V. Silva, D. Oliveira, K. Ocana, J. Dias, E. Ogasawara, and M. Mattoso, 2013, Capturing and Querying Workflow Runtime Provenance with PROV: a Practical Approach, In: Proc. of the International Workshop on Managing and Querying Provenance Data at Scale.

Digital Library

[19]

C. Boeres, I. Sardina, and L. Drummond, 2011, An efficient weighted biobjective scheduling algorithm for heterogeneous systems, Parallel Computing, v. 37, n. 8 (Agosto.), p. 349-364.

Digital Library

[20]

D. Oliveira, K. Ocana, F. Baiao, and M. Mattoso, 2012, A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds, Journal of Grid Computing, v. 10, n. 3, p. 521-552.

Digital Library

[21]

E. Ogasawara, J. Dias, D. Oliveira, F. Porto, P. Valduriez, and M. Mattoso, 2011, An Algebraic Approach for Data-Centric Scientific Workflows, Proc. of VLDB Endowment, v. 4, n. 12, p. 1328-1339.

[22]

L.M. Northrop, 2002, SEI's software product line tenets, IEEE Software, v. 19, n. 4, p. 32-40.

Digital Library

[23]

E. Ogasawara, C. Paulino, L. Murta, C. Werner, and M. Mattoso, 2009, Experiment Line: Software Reuse in Scientific Workflows, In: Scientific and Statistical Database Management, p. 264-272.

Digital Library

Cited By

Heng NYang ZZhu SZeng J(2019)A Novel Method of Dynamic Cloud Workflow Processing Based on 3D_DWFNProceedings of the 8th International Conference on Software and Information Engineering10.1145/3328833.3328876(109-113)Online publication date: 9-Apr-2019
https://dl.acm.org/doi/10.1145/3328833.3328876

Index Terms

Recommendations

Dynamic steering of HPC scientific workflows

The field of scientific workflow management systems has grown significantly as applications start using them successfully. In 2007, several active researchers in scientific workflow developments presented the challenges for the state of the art in ...
Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Scientists have many computing infrastructures available to conduct their research, including grids and public or private clouds. This article explores the use of these cyberinfrastructures to execute scientific workflows, an important class of ...
Protecting scientific workflows in clouds with an intrusion tolerant system

With the development of cloud computing technology, more and more scientific workflows are delivered to cloud platforms to complete. However, there are many threats in clouds due to the multi‐tenant coexistence. In order to protect scientific workflows in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

UCC '13: Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing

December 2013

530 pages

ISBN:9780769551524

Publisher

IEEE Computer Society

United States

Publication History

Published: 09 December 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
54
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)4

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Heng NYang ZZhu SZeng J(2019)A Novel Method of Dynamic Cloud Workflow Processing Based on 3D_DWFNProceedings of the 8th International Conference on Software and Information Engineering10.1145/3328833.3328876(109-113)Online publication date: 9-Apr-2019
https://dl.acm.org/doi/10.1145/3328833.3328876

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten