[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Optimal distributed task scheduling in volunteer clouds

Published: 01 May 2017 Publication History

Abstract

A framework for task scheduling policies in a large-scale distributed cloud.A mathematical formulation driven by real system requirements.Model with: FIFO queue, tasks with deadlines, the actual load on the machines.Application of the distributed Alternating Direction Method of Multipliers (ADMM). The ever increasing request of computational resources has shifted the computing paradigm towards solutions where less computation is performed locally. The most widely adopted approach nowadays is represented by cloud computing. With the cloud, users can transparently access to virtually infinite resources with the same aptitude of using any other utility. Next to the cloud, the volunteer computing paradigm has gained attention in the last decade, where the spared resources on each personal machine are shared thanks to the users' willingness to cooperate. Cloud and volunteer paradigms have been recently seen as companion technologies to better exploit the use of local resources. Conversely, this scenario places complex challenges in managing such a large-scale environment, as the resources available on each node and the presence of the nodes online are not known a-priori. The complexity further increases in presence of tasks that have an associated Service Level Agreement specified, e.g., through a deadline. Distributed management solutions have then be advocated as the only approaches that are realistically applicable.In this paper, we propose a framework to allocate tasks according to different policies, defined by suitable optimization problems. Then, we provide a distributed optimization approach relying on the Alternating Direction Method of Multipliers (ADMM) for one of these policies, and we compare it with a centralized approach. Results show that, when a centralized approach can not be adopted in a real environment, it could be possible to rely on the good suboptimal solutions found by the ADMM.

References

[1]
V.D. Cunsolo, S. Distefano, A. Puliafito, M. Scarpa, Volunteer computing and desktop cloud: the cloud@home paradigm, 2009.
[2]
M. Satyanarayanan, P. Bahl, R. Caceres, N. Davies, The case for VM-based cloudlets in mobile computing, IEEE Pervasive Comput, 8 (2009) 14-23.
[3]
S. Sebastio, M. Amoretti, A. Lluch Lafuente, AVOCLOUDY: a simulator of volunteer clouds, Softw Pract Exp, 46 (2016) 3-30.
[4]
F. Costa, L. Silva, M. Dahlin, Volunteer cloud computing: mapreduce over the internet, 2011.
[5]
D.P. Anderson, BOINC: a system for public-resource computing and storage, IEEE Computer Society, Washington, DC, USA, 2004.
[6]
D. Thain, T. Tannenbaum, M. Livny, Distributed computing in practice: the condor experience., Concurr Pract Exp, 17 (2005) 323-356.
[7]
F. Brasileiro, E. Araujo, W. Voorsluys, M. Oliveira, F. Figueiredo, Bridging the high performance computing gap: the ourgrid experience, IEEE Computer Society, Washington, DC, USA, 2007.
[8]
J. Cappos, I. Beschastnikh, A. Krishnamurthy, T. Anderson, Seattle: a platform for educational cloud Computing, SIGCSE Bull, 41 (2009) 111-115.
[9]
D.P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, D. Werthimer, SETI@home: an experiment in public-resource computing, Commun ACM, 45 (2002) 56-61.
[10]
S. Sebastio, A. Scala, A workload-based approach to partition the volunteer cloud, 2015.
[11]
O. Babaoglu, M. Marzolla, M. Tamburini, Design and implementation of a P2P cloud system, ACM, New York, NY, USA, 2012.
[12]
E. Di Nitto, D.J. Dubois, R. Mirandola, On exploiting decentralized bio-inspired self-organization algorithms to develop real systems, IEEE Computer Society, Washington, DC, USA, 2009.
[13]
M. Amoretti, A.L. Lafuente, S. Sebastio, A cooperative approach for distributed task execution in autonomic clouds, 2013.
[14]
D. Talia, Cloud computing and software agents: towards cloud intelligent services, in: CEUR Workshop Proceedings, vol. 741, 2011, pp. 2-6. http://ceur-ws.org/Vol-741/INV02_Talia.pdf
[15]
M. Dorigo, L.M. Gambardella, Ant colony system: a cooperative learning approach to the traveling salesman problem, IEEE Trans Evol Comput, 1 (1997) 53-66.
[16]
S. Sebastio, M. Amoretti, A. Lluch Lafuente, A computational field framework for collaborative task execution in volunteer clouds, ACM, New York, NY, USA, 2014.
[17]
J.-T. Tsai, J.-C. Fang, J.-H. Chou, Optimized task scheduling and resource allocation on cloud computing environment using improved differential evolution algorithm, Comput Oper Res, 40 (2013) 3045-3055.
[18]
Zambonelli F., Mamei M. Spatial computing: an emerging paradigm for autonomic computing and communication. In: Smirnov M., editor. Autonomic communication; vol. 3457of Lecture Notes in Computer Science. Springer, Berlin, Heidelberg. ISBN 978-3-540-27417-9;, p. 44-57. 10.1007/11520184_4.
[19]
A. Celestini, A. Lluch Lafuente, P. Mayer, S. Sebastio, F. Tiezzi, Reputation-based cooperation in the clouds, in: IFIP Advances in Information and Communication Technology, vol. 430, Springer, Berlin, Heidelberg, 2014, pp. 213-220.
[20]
S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found Trends Mach Learn, 3 (2011) 1-122.
[21]
Grant M., Boyd S. CVX: matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx; 2014.
[22]
Gurobi Optimization I. Gurobi Optimizer Reference Manual. 2015. http://www.gurobi.com.
[23]
H. Haridas, S. Kailasam, J. Dharanipragada, Cloudy knapsack problems: an optimization model for distributed cloud-assisted systems, 2014.
[24]
M. Malawski, K. Figiela, J. Nabrzyski, Cost minimization for computational applications on hybrid cloud infrastructures, Futur Gener Comput Syst, 29 (2013) 1786-1794.
[25]
R. Fourer, D.M. Gay, B. Kernighan, AMPL: a modeling language for mathematical programming, Cengage Learning, 2002.
[26]
M. Malawski, K. Figiela, M. Bubak, E. Deelman, J. Nabrzyski, Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization, Sci Program, 2015 (2015) 1-13.
[27]
P. Wendell, J.W. Jiang, M.J. Freedman, J. Rexford, DONAR: decentralized server selection for cloud services, SIGCOMM Comput Commun Rev, 41 (2010).
[28]
H. Xu, B. Li, Joint request mapping and response routing for geo-distributed cloud services, 2013.
[29]
B. Li, S.L. Song, I. Bezakova, K.W. Cameron, EDR: an energy-aware runtime load distribution system for data-intensive applications in the cloud, 2013.
[30]
A. Nedic, A. Ozdaglar, P.A. Parrilo, Constrained consensus and optimization in multi-agent networks, IEEE Trans Autom Control, 55 (2010) 922-938.
[31]
Q. Zhu, H. Zeng, W. Zheng, M.D. Natale, A. Sangiovanni-Vincentelli, Optimization of task allocation and priority assignment in hard real-time distributed systems, ACM Trans Embed Comput Syst, 11 (2013) 1-30.
[32]
IBM. ILOG CPLEX optimizer. https://www.ibm.com/software/commerce/optimization/cplex-optimizer, March 2014.
[33]
S. Boyd, L. Vandenberghe, Convex optimization, Cambridge University Press, New York, NY, USA, 2004.
[34]
E. Ghadimi, A. Teixeira, I. Shames, M. Johansson, Optimal parameter selection for the alternating direction method of multipliers (ADMM): quadratic problems, IEEE Trans Autom Control, 60 (2015) 644-658.
[35]
M.J. Feizollahi, M. Costley, S. Ahmed, S. Grijalva, Large-scale decentralized unit commitment, Int J Electr Power Energy Syst, 73 (2015) 97-106.
[36]
O. Miksik, V. Vineet, P. Pérez, P. Torr, Distributed non-convex ADMM-based inference in large-scale random fields, BMVA Press, 2014.
[37]
The service level agreement. 2015. http://www.sla-zone.co.uk.
[38]
M. Grant, S. Boyd, Graph implementations for nonsmooth convex programs, in: Lecture Notes in Control and Information Sciences, Springer-Verlag Limited, 2008, pp. 95-110.
[39]
R.M. Karp, Reducibility among combinatorial problems, in: The IBM Research Symposia Series, Springer, US, 1972, pp. 85-103.
[40]
J. Nocedal, S.J. Wright, Numerical optimization, Springer, New York, 2006.
[41]
T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to algorithms, The MIT Press, 2009.
[42]
A.K. Mishra, J.L. Hellerstein, W. Cirne, C.R. Das, Towards characterizing cloud backend workloads: insights from google compute clusters, ACM SIGMETRICS Perform Eval Rev, 37 (2010) 34-41.
[43]
Hellerstein J.L. Google cluster data, Google research blog, 2010. Posted at http://googleresearch.blogspot.com/2010/01/google-cluster-data.html.
[44]
K.S. Trivedi, Probability and statistics with reliability, queuing and computer science applications, John Wiley and Sons Ltd., Chichester, UK, 2002.
[45]
Sebastio S., Gnecco G. A green policy to schedule tasks in a distributed cloud. Submitted for publication 2016.
[46]
M. Ehrgott, Multicriteria optimization, Springer, 2005.

Cited By

View all
  • (2020)Towards Multi-criteria Volunteer Cloud Service SelectionCloud Computing – CLOUD 202010.1007/978-3-030-59635-4_21(278-286)Online publication date: 18-Sep-2020
  • (2018)A Holistic Approach for Collaborative Workload Execution in Volunteer CloudsACM Transactions on Modeling and Computer Simulation10.1145/315533628:2(1-27)Online publication date: 9-Mar-2018
  • (2017)A novel hybrid of Shortest job first and round Robin with dynamic variable quantum time task scheduling techniqueJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-017-0085-06:1(1-12)Online publication date: 1-Dec-2017
  1. Optimal distributed task scheduling in volunteer clouds

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Computers and Operations Research
    Computers and Operations Research  Volume 81, Issue C
    May 2017
    333 pages

    Publisher

    Elsevier Science Ltd.

    United Kingdom

    Publication History

    Published: 01 May 2017

    Author Tags

    1. ADMM
    2. Cloud computing
    3. Combinatorial optimization
    4. Distributed optimization
    5. Integer programming

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 13 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2020)Towards Multi-criteria Volunteer Cloud Service SelectionCloud Computing – CLOUD 202010.1007/978-3-030-59635-4_21(278-286)Online publication date: 18-Sep-2020
    • (2018)A Holistic Approach for Collaborative Workload Execution in Volunteer CloudsACM Transactions on Modeling and Computer Simulation10.1145/315533628:2(1-27)Online publication date: 9-Mar-2018
    • (2017)A novel hybrid of Shortest job first and round Robin with dynamic variable quantum time task scheduling techniqueJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-017-0085-06:1(1-12)Online publication date: 1-Dec-2017

    View Options

    View options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media