More Web Proxy on the site http://driver.im/

survey

Multiple Workflows Scheduling in Multi-tenant Distributed Systems: A Taxonomy and Future Directions

Authors:

Muhammad H. Hilman,

Maria A. Rodriguez,

Rajkumar BuyyaAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 53, Issue 1

Article No.: 10, Pages 1 - 39

https://doi.org/10.1145/3368036

Published: 06 February 2020 Publication History

Abstract

Workflows are an application model that enables the automated execution of multiple interdependent and interconnected tasks. They are widely used by the scientific community to manage the distributed execution and dataflow of complex simulations and experiments. As the popularity of scientific workflows continue to rise, and their computational requirements continue to increase, the emergence and adoption of multi-tenant computing platforms that offer the execution of these workflows as a service becomes widespread. This article discusses the scheduling and resource provisioning problems particular to this type of platform. It presents a detailed taxonomy and a comprehensive survey of the current literature and identifies future directions to foster research in the field of multiple workflow scheduling in multi-tenant distributed computing systems.

Supplementary Material

a10-hilman-apndx.pdf (hilman.zip)

Supplemental movie, appendix, image and software files for, MultipleWorkflows Scheduling in Multi-tenant Distributed Systems: A Taxonomy and Future Directions

Download
115.77 KB

References

[1]

Roger Barga and Dennis Gannon. 2007. Scientific versus Business Workflows. Springer, London, 9--16.

[2]

Ian J. Taylor, Ewa Deelman, Dennis B. Gannon, and Matthew Shields. 2014. Workflows for e-Science: Scientific Workflows for Grids. Springer.

Digital Library

[3]

Carlos Goncalves, Luis Assuncao, and Jose C. Cunha. 2012. Data analytics in the cloud with flexible MapReduce workflows. In Proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science. 427--434.

Digital Library

[4]

Jia Yu and Rajkumar Buyya. 2005. A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3, 3 (2005), 171--200.

[5]

Marek Wieczorek, Andreas Hoheisel, and Radu Prodan. 2008. Taxonomies of the Multi-criteria Grid Workflow Scheduling Problem. Springer US, 237--264.

[6]

Fuhui Wu, Qingbo Wu, and Yusong Tan. 2015. Workflow scheduling in cloud: A survey. J. Supercomput. 71, 9 (2015), 3373--3418.

Digital Library

[7]

Sukhpal Singh and Inderveer Chana. 2016. A survey on resource scheduling in cloud computing: Issues and challenges. J. Grid Comput. 14, 2 (2016), 217--264.

Digital Library

[8]

Ehab N. Alkhanak, Sai P. Lee, and Saif U. R. Khan. 2015. Cost-aware challenges for workflow scheduling approaches in cloud computing environments: Taxonomy and opportunities. Fut. Gener. Comput. Syst. 50, Suppl. C (2015), 3--21.

Digital Library

[9]

Sucha Smanchat and Kanchana Viriyapant. 2015. Taxonomies of workflow scheduling problem and techniques in the cloud. Fut. Gener. Comput. Syst. 52, Suppl. C (2015), 1--12.

[10]

Maria A. Rodriguez and Rajkumar Buyya. 2017. A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurr. Comput.: Pract. Exp. 29, 8 (2017), e4041--n/a.

[11]

Philipp Leitner and Jürgen Cito. 2016. Patterns in the Chaos: A study of performance variation and predictability in public IaaS clouds. ACM Trans. Internet Technol. 16, 3 (2016), 1--23.

Digital Library

[12]

Saima G. Ahmad, Chee S. Liew, Muhammad M. Rafique, Ehsan U. Munir, and Samee U. Khan. 2014. Data-intensive workflow optimization based on application task graph partitioning in heterogeneous computing systems. In Proceedings of the 4th IEEE International Conference on Big Data and Cloud Computing. 129--136.

[13]

Kathy Svitil. 2016. Gravitational waves detected 100 years after Einstein’s prediction. (2016). Retrieved from http://www.caltech.edu/news/gravitational-waves-detected-100-years-after-einstein-s-prediction-49777.

[14]

Jun Qin and Thomas Fahringer. 2012. Scientific Workflows: Programming, Optimization, and Synthesis with ASKALON and AWDL. Springer Science 8 Business Media.

[15]

Maria A. Rodriguez and Rajkumar Buyya. 2017. Scientific workflow management system for clouds. In Software Architecture for Big Data and the Cloud, Ivan Mistrik, Rami Bahsoon, Nour Ali, Maritta Heisel, and Bruce Maxim (Eds.). Morgan Kaufmann, Boston, 367--387.

[16]

Bartosz Balis. 2016. HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows. Fut. Gener. Comput. Syst. 55 (2016), 147--162.

Digital Library

[17]

Prakashan Korambath, Jianwu Wang, Ankur Kumar, Lorin Hochstein, Brian Schott, Robert Graybill, Michael Baldea, and Jim Davis. 2014. Deploying kepler workflows as services on a cloud infrastructure for smart manufacturing. Proc. Comput. Sci. 29 (2014), 2254--2259.

[18]

E. Deelman, K. Vahi, M. Rynge, R. Mayani, R. F. da Silva, G. Papadimitriou, and M. Livny. 2019. The evolution of the pegasus workflow management software. Comput. Sci. Eng. 21, 4 (2019), 22--36.

[19]

Katherine Wolstencroft, Robert Haines, Donal Fellows, Alan Williams, David Withers, Stuart Owen, Stian Soiland-Reyes, Ian Dunlop, Aleksandra Nenadic, Paul Fisher, Jiten Bhagat, Khalid Belhajjame, Finn Bacall, Alex Hardisty, Abraham Nieva de la Hidalga, Maria P. Balcazar Vargas, Shoaib Sufi, and Carole Goble. 2013. The Taverna workflow suite: Designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res. 41, 1 (2013), 557--561.

[20]

Bill Howe, Garret Cole, Emad Souroush, Paraschos Koutris, Alicia Key, Nodira Khoussainova, and Leilani Battle. 2011. Database-as-a-service for long-tail science. In Scientific and Statistical Database Management. Springer, Berlin, 480--489.

[21]

Jianwu Wang, Prakashan Korambath, Ilkay Altintas, Jim Davis, and Daniel Crawl. 2014. Workflow as a service in the cloud: Architecture and scheduling algorithms. Proced. Comput. Sci. 29, Suppl. C (2014), 546--556.

[22]

Sérgio Esteves and Luís Veiga. 2016. WaaS: Workflow-as-a-service for the cloud with scheduling of continuous and data-intensive workflows. Comput. J. 59, 3 (2016), 371--383.

[23]

Bhaskar P. Rimal and Martin Maier. 2017. Workflow scheduling in multi-tenant cloud computing environments. IEEE Trans. Parallel Distrib. Syst. 28, 1 (2017), 290--304.

Digital Library

[24]

Gideon Juve, Ann Chervenak, Ewa Deelman, Shishir Bharathi, Gaurang Mehta, and Karan Vahi. 2013. Characterizing and profiling scientific workflows. Fut. Gener. Comput. Syst. 29, 3 (2013), 682--692.

Digital Library

[25]

Ewa Deelman, Gurmeet Singh, Miron Livny, Bruce Berriman, and John Good. 2008. The cost of doing science on the cloud: The montage example. In Proceedings of the ACM/IEEE Conference on Supercomputing. 1--12.

[26]

Philip Maechling, Ewa Deelman, Li Zhao, Robert Graves, Gaurang Mehta, Nitin Gupta, John Mehringer, Carl Kesselman, Scott Callaghan, David Okaya, Hunter Francoeur, Vipin Gupta, Yifeng Cui, Karan Vahi, Thomas Jordan, and Edward Field. 2007. SCEC CyberShake Workflows—Automating Probabilistic Seismic Hazard Analysis Calculations. Springer, London, 143--163.

[27]

Phuong Nguyen and Klara Nahrstedt. 2017. MONAD: Self-adaptive micro-service infrastructure for heterogeneous scientific workflows. In Proceedings of the 2017 IEEE International Conference on Autonomic Computing. 187--196.

[28]

Lincoln Bryant, Jeremy Van, Benedikt Riedel, Robert W. Gardner, Jose C. Bejar, John Hover, Ben Tovar, Kenyi Hurtado, and Douglas Thain. 2018. VC3: A virtual cluster service for community computation. In Proceedings of the Practice and Experience on Advanced Research Computing. 30:1--30:8.

[29]

Maxim Belkin, Roland Haas, Galen W. Arnold, Hon W. Leong, Eliu A. Huerta, David Lesny, and Mark Neubauer. 2018. Container solutions for HPC systems: A case study of using shifters on blue waters. In Proceedings of Practice and Experience in Advanced Research Computing. 1--8.

Digital Library

[30]

Carl Witt, Marc Bux, Wladislaw Gusew, and Ulf Leser. 2019. Predictive performance modeling for distributed batch processing using black box monitoring and machine learning. Inf. Syst. 82 (2019), 33--52.

Digital Library

[31]

Farrukh Nadeem and Thomas Fahringer. 2009. using templates to predict execution time of scientific workflow applications in the grid. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid. 316--323.

Digital Library

[32]

Rafael F. da Silva, Gideon Juve, Mats Rynge, Ewa Deelman, and Miron Livny. 2015. Online task resource consumption prediction for scientific workflows. Parallel Process. Lett. 25, 3 (2015), 1541003.

[33]

Thanh P. Pham, Juan J. Durillo, and Thomas Fahringer. 2017. Predicting workflow task execution time in the cloud using a two-stage machine learning approach. IEEE Trans. Cloud Comput. 99 (2017), 1--1.

[34]

Keith R. Jackson, Lavanya Ramakrishnan, Krishna Muriki, Shane Canon, Shreyas Cholia, John Shalf, Harvey J. Wasserman, and Nicholas J. Wright. 2010. Performance analysis of high performance computing applications on the amazon web services cloud. In Proceedings of the 2nd IEEE International Conference on Cloud Computing Technology and Science. 159--168.

[35]

Ming Mao and Marty Humphrey. 2012. A performance study on the VM startup time in the cloud. In Proceedings of the 5th IEEE International Conference on Cloud Computing. 423--430.

Digital Library

[36]

Mike Jones, Bill Arcand, Bill Bergeron, David Bestor, Chansup Byun, Lauren Milechin, Vijay Gadepally, Matt Hubbell, Jeremy Kepner, Pete Michaleas, Julie Mullen, Andy Prout, Tony Rosa, Siddharth Samsi, Charles Yee, and Albert Reuther. 2016. Scalability of VM provisioning systems. In Proceedings of the IEEE High Performance Extreme Computing Conference. 1--5.

[37]

Michael A. Murphy, Brandon Kagey, Michael Fenn, and Sebastien Goasguen. 2009. Dynamic provisioning of virtual organization clusters. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid. 364--371.

Digital Library

[38]

Wei Chen, Young C. Lee, Alan Fekete, and Albert Y. Zomaya. 2015. Adaptive multiple-workflow scheduling with task rearrangement. J. Supercomput. 71, 4 (2015), 1297--1317.

Digital Library

[39]

Yirong Wang, Kuochan Huang, and Fengjian Wang. 2016. Scheduling online mixed-parallel workflows of rigid tasks in heterogeneous multi-cluster environments. Fut. Gener. Comput. Syst. 60, Suppl. C (2016), 35--47.

Digital Library

[40]

Hamid Arabnejad, Jorge G. Barbosa, and Frédéric Suter. 2014. Fair resource sharing for dynamic scheduling of workflows on heterogeneous systems. In High-Performance Computing on Complex Environments.

[41]

Amelia C. Zhou, Bingsheng He, and Cheng Liu. 2016. Monetary cost optimizations for hosting workflow-as-a-service in IaaS clouds. IEEE Trans. Cloud Comput. 4, 1 (2016), 34--48.

Digital Library

[42]

Zhifeng Yu and Weisong Shi. 2008. A planner-guided scheduling strategy for multiple workflow applications. In Proceedings of the International Conference on Parallel Processing. 1--8.

Digital Library

[43]

Haluk Topcuoglu, Salim Hariri, and Min You Wu. 2002. Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13, 3 (2002), 260--274.

Digital Library

[44]

Meng Xu, Lizhen Cui, Haiyang Wang, and Yanbing Bi. 2009. A multiple QoS constrained scheduling strategy of multiple workflows for cloud computing. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing with Applications. 629--634.

[45]

Cui Lizhen, Xu Meng, and Yanbing Bi. 2009. A scheduling strategy for multiple QoS constrained grid workflows. In Proceedings of the Joint Conferences on Pervasive Computing. 561--566.

[46]

Jorge G. Barbosa and Belmiro Moreira. 2011. Dynamic scheduling of a batch of parallel task jobs on heterogeneous clusters. Parallel Comput. 37, 8 (2011), 428--438.

Digital Library

[47]

Jorge G. Barbosa, Celeste Morais, Ruben Nobrega, and Antònio Monteiro. 2005. Static scheduling of dependent parallel tasks on heterogeneous clusters. In Proceedings of the IEEE International Conference on Cluster Computing. 1--8.

[48]

Chihchiang Hsu, Kuochan Huang, and Fengjian Wang. 2011. Online scheduling of workflow applications in grid environments. Fut. Gener. Comput. Syst. 27, 6 (2011), 860--870.

Digital Library

[49]

Hamid Arabnejad and Jorge G. Barbosa. 2017. Maximizing the completion rate of concurrent scientific applications under time and budget constraints. J. Comput. Sci. 23, Suppl. C (2017), 120--129.

[50]

Hamid Arabnejad and Jorge G. Barbosa. 2017. Multi-QoS constrained and profit-aware scheduling approach for concurrent workflows on heterogeneous systems. Fut. Gener. Comput. Syst. 68, Suppl. C (2017), 211--221.

[51]

Georgios L. Stavrinides and Helen D. Karatza. 2011. Scheduling multiple task graphs in heterogeneous distributed real-time systems by exploiting schedule holes with bin packing techniques. Simul. Model. Pract. Theory 19, 1 (2011), 540--552.

[52]

Georgios L. Stavrinides and Helen D. Karatza. 2015. A cost-effective and QoS-aware approach to scheduling real-time workflow applications in PaaS and SaaS clouds. In Proceedings of the 3rd International Conference on Future Internet of Things and Cloud. 231--239.

[53]

Yuxin Wang, Shijie Cao, Guan Wang, Zhen Feng, Chi Zhang, and He Guo. 2017. Fairness scheduling with dynamic priority for multi workflow on heterogeneous systems. In Proceedings of the 2nd IEEE International Conference on Cloud Computing and Big Data Analysis. 404--409.

[54]

Guoqi Xie, Liangjiao Liu, Liu Yang, and Renfa Li. 2017. Scheduling trade-off of dynamic multiple parallel workflows on heterogeneous distributed computing systems. Concurr. Comput.: Pract. Exp. 29, 2 (2017), e3782--n/a.

[55]

Yinglin Tsai, Hsiaoching Liu, and Kuochan Huang. 2015. Adaptive dual-criteria task group allocation for clustering-based multi-workflow scheduling on parallel computing platform. J. Supercomput. 71, 10 (2015), 3811--3831.

Digital Library

[56]

Bing Lin, Wenzhong Guo, and Xiuyan Lin. 2016. Online optimization scheduling for scientific workflows with deadline constraint on hybrid clouds. Concurr. Comput.: Pract. Exp. 28, 11 (2016), 3079--3095.

Digital Library

[57]

Shaghayegh Sharif, Javid Taheri, Albert Y. Zomaya, and Surya Nepal. 2014. Online multiple workflow scheduling under privacy and deadline in hybrid cloud environment. In Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science. 455--462.

Digital Library

[58]

Maria A. Rodriguez and Rajkumar Buyya. 2018. Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Fut. Gener. Comput. Syst. 79, 2 (2018), 739--750.

Digital Library

[59]

Xiaomin Zhu, Ji Wang, Hui Guo, Dakai Zhu, Laurence T. Yang, and Ling Liu. 2016. Fault-tolerant scheduling for real-time scientific workflows with elastic resource provisioning in virtualized clouds. IEEE Trans. Parallel Distrib. Syst. 27, 12 (2016), 3501--3517.

Digital Library

[60]

Xiaolong Xu, Wanchun Dou, Xuyun Zhang, and Jinjun Chen. 2016. EnReal: An energy-aware resource allocation method for scientific workflow executions in cloud environment. IEEE Trans. Cloud Comput. 4, 2 (2016), 166--179.

[61]

Huangke Chen, Xiaomin Zhu, Dishan Qiu, Hui Guo, Laurence T. Yang, and Peizhong Lu. 2016. EONS: Minimizing energy consumption for executing real-time workflows in virtualized cloud data centers. In Proceedings of the 45th International Conference on Parallel Processing Workshops. 385--392.

[62]

Guoqi Xie, Gang Zeng, Junqiang Jiang, Chunnian Fan, Renfa Li, and Keqin Li. 2017. Energy management for multiple real-time workflows on cyber--physical cloud systems. Fut. Gener. Comput. Syst. (2017).

[63]

Huangke Chen, Xiaomin Zhu, Dishan Qiu, and Ling Liu. 2016. Uncertainty-aware real-time workflow scheduling in the cloud. In Proceedings of the 9th IEEE International Conference on Cloud Computing. 577--584.

[64]

Huangke Chen, Jianghan Zhu, Zhenshi Zhang, Manhao Ma, and Xin Shen. 2017. Real-time workflows oriented online scheduling in uncertain cloud environment. J. Supercomput. 73, 11 (2017), 4906--4922.

Digital Library

[65]

Huangke Chen, Xiaomin Zhu, Guipeng Liu, and Witold Pedrycz. 2018. Uncertainty-aware online scheduling for real-time workflows in cloud service environment. IEEE Trans. Serv. Comput. (2018), 1--1.

[66]

Jiagang Liu, Ju Ren, Wei Dai, Deyu Zhang, Pude Zhou, Yaoxue Zhang, Geyong Min, and Noushin Najjari. 2019. Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans. Cloud Comput. (2019), 1--1.

[67]

Georgios L. Stavrinides, Francisco R. Duro, Helen D. Karatza, Javier G. Blas, and Jesus Carretero. 2017. Different aspects of workflow scheduling in large-scale distributed systems. Simul. Model. Pract. Theory 70, Suppl. C (2017), 120--134.

[68]

Naqin Zhou, FuFang Li, Kefu Xu, and Deyu Qi. 2018. Concurrent workflow budget- and deadline-constrained scheduling in heterogeneous distributed environments. Soft Computing 22, 23 (2018), 7705--7718.

Digital Library

[69]

Huangke Chen, Jianghan Zhu, Guohua Wu, and Lisu Huo. 2018. Cost-efficient reactive scheduling for real-time workflows in clouds. J. Supercomput. 74, 11 (2018), 6291--6309.

Digital Library

[70]

Georgios L. Stavrinides and Helen D. Karatza. 2010. Scheduling multiple task graphs with end-to-end deadlines in distributed real-time systems utilizing imprecise computations. J. Syst. Softw. 83, 6 (2010), 1004--1014.

Digital Library

[71]

Francisco R. Duro, Javier G. Blas, and Jesus Carretero. 2013. A hierarchical parallel storage system based on distributed memory for large scale systems. In Proceedings of the 20th European MPI Users’ Group Meeting. 139--140.

Digital Library

[72]

Xiaoyong Tang, Kenli Li, Guiping Liao, Kui Fang, and Fan Wu. 2011. A stochastic scheduling algorithm for precedence constrained tasks on grid. Fut. Gener. Comput. Syst. 27, 8 (2011), 1083--1091.

Digital Library

[73]

Deepak Poola, Saurab Garg, Rajkumar Buyya, Yun Yang, and Kotagiri Ramamohanarao. 2014. Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In Proceedings of the 28th IEEE International Conference on Advanced Information Networking and Applications. 858--865.

Digital Library

[74]

Vahid Ebrahimirad, Maziar Goudarzi, and Aboozar Rajabi. 2015. Energy-aware scheduling for precedence-constrained parallel virtual machines in virtualized data centers. J. Grid Comput. 13, 2 (2015), 233--253.

Digital Library

[75]

Ilia Pietri and Rizos Sakellariou. 2014. Energy-aware workflow scheduling using frequency scaling. In Proceedings of the 43rd International Conference on Parallel Processing Workshops. 104--113.

Digital Library

[76]

Xiao Qin and Hong Jiang. 2006. A novel fault-tolerant scheduling algorithm for precedence constrained tasks in real-time heterogeneous systems. Parallel Comput. 32, 5 (2006), 331--356.

Digital Library

[77]

Shuo Zhang, Baosheng Wang, Baokang Zhao, and Jing Tao. 2013. An energy-aware task scheduling algorithm for a heterogeneous data center. In Proceedings of the 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications. 1471--1477.

Digital Library

[78]

Juan J. Durillo, Hamid M. Fard, and Radu Prodan. 2012. MOHEFT: A multi-objective list-based method for workflow scheduling. In Proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science. 185--192.

Digital Library

[79]

Guo Zhong Tian, Chuang Bai Xiao, Zhu Sheng Xu, and Xia Xiao. 2012. Hybrid scheduling strategy for multiple DAGs workflow in heterogeneous system. J. Softw. 23, 10 (2012), 2720--2734.

[80]

Guoqi Xie, Renfa Li, Xiongren Xiao, and Yuekun Chen. 2014. A high-performance DAG task scheduling algorithm for heterogeneous networked embedded systems. In Proceedings of the 28th IEEE International Conference on Advanced Information Networking and Applications. 1011--1016.

Digital Library

[81]

Zhuo Tang, Ling Qi, Zhenzhen Cheng, Kenli Li, Samee U. Khan, and Keqin Li. 2016. An energy-efficient task scheduling algorithm in DVFS-enabled cloud environment. J. Grid Comput. 14, 1 (2016), 55--74.

Digital Library

[82]

Ewa Deelman, Tom Peterka, Ilkay Altintas, Christopher D. Carothers, Kerstin K. van Dam, Kenneth Moreland, Manish Parashar, Lavanya Ramakrishnan, Michela Taufer, and Jeffrey Vetter. 2018. The future of scientific workflows. Int. J. High Perf. Comput. Appl. 32, 1 (2018), 159--175.

Digital Library

[83]

Maria Fazio, Antonio Celesti, Rajiv Ranjan, Chang Liu, Lydia Chen, and Massimo Villari. 2016. Open issues in scheduling microservices in the cloud. IEEE Cloud Comput. 3, 5 (2016), 81--88.

[84]

Zhanibek Kozhirbayev and Richard O. Sinnott. 2017. A performance comparison of container-based technologies for the cloud. Fut. Gener. Comput. Syst. 68, Suppl. C (2017), 175--182.

[85]

Wolfgang Gerlach, Wei Tang, Andreas Wilke, Dan Olson, and Folker Meyer. 2015. Container orchestration for scientific workflows. In Proceedings of the IEEE International Conference on Cloud Engineering. 377--378.

Digital Library

[86]

Rawaa Qasha, Jacek Cala, and Paul Watson. 2016. Dynamic deployment of scientific workflows in the cloud using container virtualization. In Proceedings of the IEEE International Conference on Cloud Computing Technology and Science. 269--276.

[87]

Kai Liu, Kento Aida, Shigetoshi Yokoyama, and Yoshinobu Masatani. 2016. Flexible container-based computing platform on cloud for scientific workflows. In Proceedings of the International Conference on Cloud Computing Research and Innovations. 56--63.

[88]

Eidah J. Alzahrani, Zahir Tari, Young C. Lee, Deafallah Alsadie, and Albert Y. Zomaya. 2017. adCFS: Adaptive completely fair scheduling policy for containerised workflows systems. In Proceedings of the 16th IEEE International Symposium on Network Computing and Applications. 1--8.

[89]

Theo Combe, Antony Martin, and Roberto D. Pietro. 2016. To Docker or not to Docker: A security perspective. IEEE Cloud Comput. 3, 5 (2016), 54--62.

[90]

Gregory M. Kurtzer, Vanessa Sochat, and Michael W. Bauer. 2017. Singularity: Scientific containers for mobility of compute. PLoS ONE 12, 5 (2017), 1--20.

[91]

Emily Le and David Paz. 2017. Performance analysis of applications using singularity container on SDSC comet. In Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact. 66:1--66:4.

[92]

Heru Suhartanto, Agung P. Pasaribu, Muhammad F. Siddiq, Muhammad I. Fadhila, Muhammad H. Hilman, and Arry Yanuar. 2017. A preliminary study on shifting from virtual machine to Docker container for insilico drug discovery in the cloud. Int. J. Technol. 8, 4 (2017).

[93]

Sareh Fotuhi Piraghaj, Amir Vahid Dastjerdi, Rodrigo N. Calheiros, and Rajkumar Buyya. 2017. ContainerCloudSim: An environment for modeling and simulation of containers in cloud data centers. Softw.: Pract. Exp. 47, 4 (2017), 505--521.

[94]

Maciej Malawski. 2016. Towards serverless execution of scientific workflows - HyperFlow case study. In Proceedings of the Workshop of Workflows in Support of Large-Scale Sciences. 25--33.

[95]

Qingye Jiang, Young C. Lee, and Albert Y. Zomaya. 2017. Serverless execution of scientific workflows. In Proceedings of the 15th International Conference Service-Oriented Computing. 706--721.

[96]

Maciej Malawski, Adam Gajek, Adam Zima, Bartosz Balis, and Kamil Figiela. 2017. Serverless execution of scientific workflows: Experiments with HyperFlow, AWS lambda and Google cloud functions. Fut. Gener. Comput. Syst. (2017).

[97]

Josef Spillner, Cristian Mateos, and David A. Monge. 2018. FaaSter, better, cheaper: The prospect of serverless scientific computing and HPC. In High Performance Computing, Esteban Mocskos and Sergio Nesmachnow (Eds.). Springer International Publishing, Cham, 154--168.

[98]

Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library operating systems for the cloud. In Proceedings of the 18th International Conference on Architectural Support for Programming Languages and Operating Systems. 461--472.

Digital Library

[99]

Dan Williams, Ricardo Koller, Martin Lucina, and Nikhil Prakash. 2018. Unikernels as processes. In Proceedings of the ACM Symposium on Cloud Computing. 199--211.

Digital Library

[100]

Foued Jrad, Jie Tao, and Achim Streit. 2013. A broker-based framework for multi-cloud workflows. In Proceedings of the International Workshop on Multi-cloud Applications and Federated Clouds. 61--68.

Digital Library

[101]

Javier D. Montes, Mengsong Zou, Rahul Singh, Shu Tao, and Manish Parashar. 2014. Data-driven workflows in multi-cloud marketplaces. In Proceedings of the 7th IEEE International Conference on Cloud Computing. 168--175.

Digital Library

[102]

Yushi Omote, Takahiro Shinagawa, and Kazuhiko Kato. 2015. Improving agility and elasticity in bare-metal clouds. In Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems. 145--159.

Digital Library

[103]

Ryan Shea, Feng Wang, Haiyang Wang, and Jiangchuan Liu. 2014. A deep investigation into network performance in virtual machine based cloud environments. In Proceeding of the IEEE Conference on Computer Communications. 1285--1293.

[104]

Farrukh Nadeem, Daniyal Alghazzawi, Abdulfattah Mashat, Khalid Fakeeh, Abdullah Almalaise, and Hani Hagras. 2017. Modeling and predicting execution time of scientific workflows in the grid using radial basis function neural network. Cluster Comput. 20, 3 (2017), 2805--2819.

Digital Library

[105]

Doyen Sahoo, Steven C. H. Hoi, and Bin Li. 2019. Large scale online multiple kernel regression with application to time-series prediction. ACM Trans. Knowl. Discov. Data 13, 1 (2019), 9:1--9:33.

Digital Library

[106]

Muhammad H. Hilman, Maria A. Rodríguez, and Rajkumar Buyya. 2018. Task runtime prediction in scientific workflows using an online incremental learning approach. In Proceedings of the 11th IEEE/ACM International Conference on Utility and Cloud Computing. 93--102.

[107]

Jan Zenisek, Florian Holzinger, and Michael Affenzeller. 2019. Machine learning based concept drift detection for predictive maintenance. Comput. Industr. Eng. 137 (2019), 106031.

[108]

Taghrid Samak, Dan Gunter, Monte Goode, Ewa Deelman, Gideon Juve, Gaurang Mehta, Fabio Silva, and Karan Vahi. 2011. Online fault and anomaly detection for large-scale scientific workflows. In Proceedings of the IEEE International Conference on High Performance Computing and Communications. 373--381.

Digital Library

[109]

Prathamesh Gaikwad, Anirban Mandal, Paul Ruth, Gideon Juve, Dariusz Król, and Ewa Deelman. 2016. Anomaly detection for scientific workflow applications on networked clouds. In Proceedings of the International Conference on High Performance Computing Simulation. 645--652.

[110]

Maria A. Rodriguez, Ramamohanarao Kotagiri, and Rajkumar Buyya. 2018. Detecting performance anomalies in scientific workflows using hierarchical temporal memory. Fut. Gener. Comput. Syst. 88 (2018), 624--635.

[111]

Hamid Arabnejad and Jorge G. Barbosa. 2015. Multi-workflow QoS-constrained scheduling for utility computing. In Proceedings of the 18th IEEE International Conference on Computational Science and Engineering. 137--144.

[112]

Mozhgan Ghasemzadeh, Hamid Arabnejad, and Jorge G. Barbosa. 2017. Deadline-budget constrained scheduling algorithm for scientific workflows in a cloud environment. In Proceedings of the 20th International Conference on Principles of Distributed Systems, Vol. 70. 19:1--19:16.

[113]

Hamid M. Fard, Radu Prodan, Juan J. Durillo, and Thomas Fahringer. 2012. A multi-objective approach for workflow scheduling in heterogeneous environments. In Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 300--309.

Digital Library

[114]

Maciej Malawski, Gideon Juve, Ewa Deelman, and Jarek Nabrzyski. 2015. Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Fut. Gener. Comput. Syst. 48, Suppl. C (2015), 1--18.

[115]

Anton Beloglazov, Rajkumar Buyya, Young C. Lee, and Albert Y. Zomaya. 2011. Chapter 3 - A taxonomy and survey of energy-efficient data centers and cloud computing systems. Advances in Computers, Vol. 82. Elsevier, 47--111.

[116]

Adel Nadjaran Toosi, Chenhao Qu, Marcos Dias de Assunção, and Rajkumar Buyya. 2017. Renewable-aware geographical load balancing of web applications for sustainable data centers. J. Netw. Comput. Appl. 83, C (2017), 155--168.

[117]

Lingfang Zeng, Bharadwaj Veeravalli, and Xiaorong Li. 2015. SABA: A security-aware and budget-aware workflow scheduling strategy in clouds. J. Parallel Distrib. Comput. 75 (2015), 141--151.

Digital Library

[118]

Feng Zhao, Chao Li, and Chunfeng Liu. 2014. A cloud computing security solution based on fully homomorphic encryption. In Proceedings of the 16th International Conference on Advanced Communication Technology. 485--488.

[119]

Jayavardhana Gubbi, Rajkumar Buyya, Slaven Marusic, and Marimuthu Palaniswami. 2013. Internet of Things (IoT): A vision, architectural elements, and future directions. Fut. Gener. Comput. Syst. 29, 7 (2013), 1645--1660.

Digital Library

[120]

Charalampos Doukas and Fabio Antonelli. 2014. A full end-to-end platform as a service for smart city applications. In Proceedings of the 10th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications. 181--186.

[121]

Matteo Nardelli, Stefan Nastic, Schahram Dustdar, Massimo Villari, and Rajiv Ranjan. 2017. Osmotic flow: Osmotic computing + IoT workflow. IEEE Cloud Comput. 4, 2 (2017), 68--75.

[122]

Georgios L. Stavrinides and Helen D. Karatza. 2019. A hybrid approach to scheduling real-time IoT workflows in fog and cloud environments. Multimedia Tools Appl. 78, 17 (2019), 24639--24655.

Cited By

Tao XQian XHan LFan WShi YZhu XLi ZWei SXu R(2024)Key Flow First Prioritized Flow Scheduling Strategy in Multi-Tenant Data CentersIEEE Transactions on Network and Service Management10.1109/TNSM.2024.336414921:3(3264-3277)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3364149
Knorst TJordan MKorol GRutzig MBeck A(2024)An Automatic Framework for Collaborative CPU Thread Throttling and FPGA HLS-Versioning2024 XIV Brazilian Symposium on Computing Systems Engineering (SBESC)10.1109/SBESC65055.2024.10771920(1-6)Online publication date: 26-Nov-2024
https://doi.org/10.1109/SBESC65055.2024.10771920
Nguyen T(2024)Holistic cold-start management in serverless computing cloud with deep learning for time seriesFuture Generation Computer Systems10.1016/j.future.2023.12.011153:C(312-325)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.future.2023.12.011
Show More Cited By

Index Terms

Multiple Workflows Scheduling in Multi-tenant Distributed Systems: A Taxonomy and Future Directions

Recommendations

Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Scientists have many computing infrastructures available to conduct their research, including grids and public or private clouds. This article explores the use of these cyberinfrastructures to execute scientific workflows, an important class of ...
Deadline-constrained algorithms for scheduling of bag-of-tasks and workflows in cloud computing environments
HP3C: Proceedings of the 2nd International Conference on High Performance Compilation, Computing and Communications

Cloud computing is an emerging distributed computing paradigm that solves immense scientific applications through distributing computing resources over the Internet. These applications may have a huge number of tasks that may increase their execution ...
Bringing Scientific Workflows to Amazon SWF
SEAA '13: Proceedings of the 2013 39th Euromicro Conference on Software Engineering and Advanced Applications

In response to the ever-increasing needs of scientific applications for resources, Cloud computing emerged as an alternative on-demand and cost-effective resource provisioning approach. In this context, Cloud providers have recognised the importance of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 53, Issue 1

January 2021

781 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3382040

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 February 2020

Accepted: 01 October 2019

Revised: 01 September 2019

Received: 01 July 2019

Published in CSUR Volume 53, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Survey
Refereed

Funding Sources

Indonesia Endowment Fund For Education (LPDP), Republic of Indonesia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

48
Total Citations
View Citations
1,381
Total Downloads

Downloads (Last 12 months)176
Downloads (Last 6 weeks)14

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tao XQian XHan LFan WShi YZhu XLi ZWei SXu R(2024)Key Flow First Prioritized Flow Scheduling Strategy in Multi-Tenant Data CentersIEEE Transactions on Network and Service Management10.1109/TNSM.2024.336414921:3(3264-3277)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3364149
Knorst TJordan MKorol GRutzig MBeck A(2024)An Automatic Framework for Collaborative CPU Thread Throttling and FPGA HLS-Versioning2024 XIV Brazilian Symposium on Computing Systems Engineering (SBESC)10.1109/SBESC65055.2024.10771920(1-6)Online publication date: 26-Nov-2024
https://doi.org/10.1109/SBESC65055.2024.10771920
Nguyen T(2024)Holistic cold-start management in serverless computing cloud with deep learning for time seriesFuture Generation Computer Systems10.1016/j.future.2023.12.011153:C(312-325)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.future.2023.12.011
Karami SAzizi SAhmadizar F(2024)A bi-objective workflow scheduling in virtualized fog-cloud computing using NSGA-II with semi-greedy initializationApplied Soft Computing10.1016/j.asoc.2023.111142151(111142)Online publication date: Jan-2024
https://doi.org/10.1016/j.asoc.2023.111142
Vaish PAnand NSingh VSharma G(2024)Applications hosting over cloud-assisted IOT: a productivity model and method defining accessibility of data securityThe Journal of Supercomputing10.1007/s11227-023-05668-480:4(5540-5564)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s11227-023-05668-4
Saeedizade EAshtiani M(2024)Scientific workflow scheduling algorithms in cloud environments: a comprehensive taxonomy, survey, and future directionsJournal of Scheduling10.1007/s10951-024-00820-1Online publication date: 28-Oct-2024
https://doi.org/10.1007/s10951-024-00820-1
Khaledian NVoelp MAzizi SShirvani M(2024)AI-based & heuristic workflow scheduling in cloud and fog computing: a systematic reviewCluster Computing10.1007/s10586-024-04442-227:8(10265-10298)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10586-024-04442-2
Chen LLiu GZhang JZhang X(2024)Scheduling ensemble workflows on hybrid resources in IaaS cloudsComputing10.1007/s00607-024-01386-8107:1Online publication date: 8-Dec-2024
https://doi.org/10.1007/s00607-024-01386-8
Chen YLai K(2024)A Fairness-Aware Load Balancing Strategy in Multi-tenant CloudsFrontier Computing on Industrial Applications Volume 410.1007/978-981-99-9342-0_24(222-233)Online publication date: 21-Jan-2024
https://doi.org/10.1007/978-981-99-9342-0_24
Roy SGharote MRamamurthy APawar ALodha S(2024)Security-Aware Scheduling of Multiple Scientific Workflows in CloudCloud Computing and Services Science10.1007/978-3-031-68165-3_1(1-24)Online publication date: 15-Aug-2024
https://doi.org/10.1007/978-3-031-68165-3_1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents