[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2499896.2499898acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

A continuous workflow scheduling framework

Published: 23 June 2013 Publication History

Abstract

Traditional workflow management or enactment systems (WfMS) and workflow design processes view the workflow as a one-time interaction with the various data sources, i.e., when a workflow is invoked, its steps are executed once and in-order. The fundamental underlying assumption has been that data sources are passive and all interactions are structured along the request/reply (query) model. Hence, traditional WfMS cannot effectively support business or scientific monitoring applications that require the processing of data streams such as those generated nowadays by sensing devices as well as mobile and web applications.
Our hypothesis is that WfMS, both in the scientific and business domains, can be extended to support data stream semantics to enable monitoring applications. This includes the ability to apply flexible bounds on unbounded data streams and the ability to facilitate on-the-fly processing of bounded bundles of data (window semantics). In our previous work we have developed and implemented a Continuous Workflow Model that supports our hypothesis. This implementation of a CONtinuous workFLow ExeCution Engine (CONFLuEnCE) led to the realization that different applications have different performance requirements and hence an integrated workflow scheduling framework is essential. Such a framework is the main contribution of this paper. In particular, we designed and implemented STAFiLOS, a STreAm FLOw Scheduling for Continuous Workflows framework within CONFLuEnCE and evaluated STAFiLOS based on the Linear Road Benchmark.

References

[1]
R. Adaikkalavan and S. Chakravarthy. Seamless event and data stream processing: Reconciling windows and consumption modes. In Database Systems for Advanced Applications, LNCS 6587, pp. 341--356. 2011.
[2]
ADMT Laboratory. CMPI/Data Echange Server. http://www.dataxs.org, 2010.
[3]
A. Arasu et al. Linear Road: a stream data management benchmark. In VLDB, pp. 480--491, 2004.
[4]
S. Babu and J. Widom. Continuous queries over data streams. ACM SIGMOD Record, 30(3), 2001.
[5]
M. Beck. Linux kernel internals. Addison-Wesley, 1998.
[6]
A. Berfield, P. K. Chrysanthis, I. Tsamardinos, M. E. Pollack, and S. Banerjee. A scheme for integrating e-services in establishing virtual enterprises. In IEEE RIDE, pp. 134--142, 2002.
[7]
D. Carney et al. Monitoring streams: A new class of data management applications. In VLDB, pp. 215--226, 2002.
[8]
S. Chandrasekaran et al. TelegraphCQ: continuous dataflow processing. In ACM SIGMOD, pp. 668--668, 2003.
[9]
P. K. Chrysanthis. AQSIOS - next generation data stream management system. CONET Newsletter, 2010.
[10]
E. Deelman et al. PEGASUS: A framework for mapping complex scientific workflows onto distributed systems. Scientific Programming, 13(3):219--237, 2005.
[11]
J. Eker, J. W. Janneck, E. A. Lee, J. Liu, X. Liu, J. Ludvig, S. Neuendorffer, S. Sachs, and Y. Xiong. Taming heterogeneity-the ptolemy approach. Proceedings of the IEEE, 91(1):127--144, 2003.
[12]
J. Hidders, N. Kwasnikowska, J. Sroka, J. Tyszkiewicz, and J. V. den Bussche. Dfl: A dataflow language based on petri nets and nested relational calculus. Information Systems, 33(3):261--284, 2008.
[13]
IBM System S. http://www-01.ibm.com/software/sw-library/en_US/detail/R924335M43279V91.html, 2008.
[14]
M. C. Jaeger, G. Rojec-Goldmann, and G. Muhl. QoS aggregation for web service composition using workflow patterns. In IEEE EDOC, pp. 149--159, 2004.
[15]
B. Ludäscher et al. Scientific workflow management and the kepler system. Concurrency and Computation: Practice and Experience, 18(10):1039--1065, 2006.
[16]
Microsoft streaminsight, http://www.microsoft.com/sqlserver/2008/en/us/r2-complex-event.aspx, 2008.
[17]
R. Motwani et al. Query processing, resource management, and approximation ina data stream management system. Technical Report 2002-41, Stanford InfoLab, 2002.
[18]
P. Neophytou. Continuous Workflows: From Model to Enactment System. PhD thesis, University of Pittsburgh, 2013.
[19]
P. Neophytou, P. K. Chrysanthis, and A. Labrinidis. Towards continuous workflow enactment systems. In CollaborateCom, pp. 162--178, 2008.
[20]
P. Neophytou, P. K. Chrysanthis, and A. Labrinidis. CONFLUENCE: Continuous workflow execution engine. In ACM SIGMOD, pp. 1311--1314, 2011.
[21]
P. Neophytou, P. K. Chrysanthis, and A. Labrinidis. CONFLUENCE: Implementation and application design. In CollaborateCom, pp. 181--190, 2011.
[22]
P. Neophytou, R. Gheorghiu, R. Hachey, T. Luciani, D. Bao, A. Labrinidis, E. G. Marai, and P. K. Chrysanthis. Astroshelf: understanding the universe through scalable navigation of a galaxy of annotations. In ACM SIGMOD, pp. 713--716, 2012.
[23]
T. M. Oinn et al. Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics, 20(17):3045--3054, 2004.
[24]
C. Paszko and C. Pugsley. Considerations in selecting a laboratory information management system (LIMS). American Laboratory, 32(18):38--43, 2000.
[25]
O. Perrin and C. Godart. A model to support collaborative work in virtual enterprises. DKE, 50(1):63--86, 2004.
[26]
T. N Pham, P. K. Chrysanthis, and A. Labrinidis. Self-managing load shedding for data stream management systems. In IEEE SMDB, 2013.
[27]
T. N. Pham, L. A. Moakar, P. K. Chrysanthis, and A. Labrinidis. Dilos: A dynamic integrated load manager and scheduler for continuous queries. In IEEE SMDB, pp. 10--15, 2011.
[28]
M. A. Sharaf, P. K. Chrysanthis, A. Labrinidis, and K. Pruhs. Algorithms and metrics for processing multiple heterogeneous continuous queries. ACM Trans. Database Syst., 33(1), 2008.
[29]
I. StreamBase. Streambase: Real-time, low latency data processing with a stream processing engine. http://www.streambase.com, 2006.
[30]
P. Tucker, D. Maier, T. Sheard, and L. Fegaras. Exploiting punctuation semantics in continuous data streams. IEEE TKDE, pp. 555--568, 2003.
[31]
J. Wang, D. Crawl, and I. Altintas. Kepler + hadoop: a general architecture facilitating data-intensive applications in scientific workflow systems. In ACM WORKS, pp. 1--8, 2009.
[32]
J. Yu, R. Buyya, and C.-K. Tham. Cost-based scheduling of scientific workflow application on utility grids. In e-Science, pp. 140--147, 2005.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
SWEET '13: Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
June 2013
48 pages
ISBN:9781450323499
DOI:10.1145/2499896
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. continuous queries
  2. data streams

Qualifiers

  • Research-article

Funding Sources

Conference

SIGMOD/PODS'13
Sponsor:

Acceptance Rates

SWEET '13 Paper Acceptance Rate 4 of 6 submissions, 67%;
Overall Acceptance Rate 4 of 6 submissions, 67%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 194
    Total Downloads
  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)1
Reflects downloads up to 26 Dec 2024

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media