More Web Proxy on the site http://driver.im/

research-article

SpotADAPT: Spot-Aware (re-)Deployment of Analytical Processing Tasks on Amazon EC2

Authors:

Dalia Kaulakienė,

Christian Thomsen,

Torben Bach Pedersen,

Ugur Çetintemel,

Tim KraskaAuthors Info & Claims

DOLAP '15: Proceedings of the ACM Eighteenth International Workshop on Data Warehousing and OLAP

Pages 59 - 68

https://doi.org/10.1145/2811222.2811227

Published: 22 October 2015 Publication History

Abstract

Having constantly increasing amounts of data, the analysis of it is often entrusted for a MapReduce framework. The execution of an analytical workload can be cheapened by adopting cloud computing resources, and in particular by using spot instances (cheap, fluctuating price instances) offered by Amazon Web Services (AWS). The users aiming for the spot market are presented with many instance types placed in multiple datacenters in the world, and thus it is difficult to choose the optimal deployment. In this paper, we propose the framework SpotADAPT (Spot-Aware (re-)Deployment of Analytical Processing Tasks) which is designed to help users by first, estimating the workload execution time on different AWS instance types, and, second, proposing the deployment (i.e., specific availability zone, instance type, pricing model) aligned with user-provided optimization goals (fastest or cheapest execution within boundaries). Moreover, during the execution of the workload, SpotADAPT suggests a redeployment if the current spot instance gets terminated by Amazon or a better deployment becomes possible due to fluctuations of the spot prices. The approach is evaluated using the actual execution times of typical analytical workloads and real spot price traces. SpotADAPT's suggested deployments are comparable to the theoretically optimal ones, and in particular, it shows good cost benefits for the budget optimization - on average SpotADAPT is at most 0.3% more expensive than the theoretically optimal deployments.

References

[1]

O. Agmon Ben-Yehuda, M. Ben-Yehuda, A. Schuster, and D. Tsafrir. Deconstructing Amazon EC2 spot instance pricing. ACM Trans. Econ. Comput., 1(3):16:1--16:20, 2013.

Digital Library

[2]

F. Ahmad. PUMA: MapReduce Benchmarks. https://engineering.purdue.edu/puma/, 2015.

[3]

Amazon Web Services. Amazon EC2 Spot Instances, 2015. http://aws.amazon.com/ec2/purchasing-options/spot-instances/.

[4]

A. Andrzejak, D. Kondo, and S. Yi. Decision model for cloud computing under SLA constraints. In MASCOTS, pages 257--266, 2010.

Digital Library

[5]

C. Binnig, A. Salama, E. Zamanian, M. El-Hindi, S. Feil, and T. Ziegler. Spotgres - parallel data analytics on spot instances. In Proc. of ICDE Workshops, CloudDM'2015, 2015.

[6]

N. Chohan, C. Castillo, M. Spreitzer, M. Steinder, A. Tantawi, and C. Krintz. See spot run: Using spot instances for MapReduce workflows. In Proc. of HotCloud, pages 7--7, 2010.

Digital Library

[7]

B. Javadi, R. K. Thulasiram, and R. Buyya. Characterizing spot price dynamics in public cloud environments. Future Gener. Comput. Syst., 29(4):988--999, 2013.

Digital Library

[8]

A. Labrinidis, H. Qu, and J. Xu. Quality contracts for real-time enterprises. In Proc. of BIRTE, pages 143--156, 2007.

Digital Library

[9]

A. Marathe, R. Harris, D. Lowenthal, B. R. de Supinski, B. Rountree, and M. Schulz. Exploiting redundancy for cost-effective, time-constrained execution of HPC applications on Amazon EC2. In Proc. of HPCD, pages 279--290, 2014.

Digital Library

[10]

V. Raghavan and E. A. Rundensteiner. CAQE: A contract driven approach to processing concurrent decision support queries. In Proc. of EDBT, pages 121--132, 2014.

[11]

S. Tang, J. Yuan, and X.-Y. Li. Towards optimal bidding strategy for Amazon EC2 cloud spot instance. In IEEE CLOUD, pages 91--98, 2012.

Digital Library

[12]

S. Tang, J. Yuan, C. Wang, and X.-Y. Li. A framework for Amazon EC2 bidding strategy under SLA constraints. IEEE TPDC, 25(1):2--11, 2014.

Digital Library

[13]

W. Voorsluys and R. Buyya. Reliable provisioning of spot instances for compute-intensive applications. In Proc. of IEEE AINA, pages 542--549, 2012.

Digital Library

[14]

W. Voorsluys, S. K. Garg, and R. Buyya. Provisioning spot market cloud resources to create cost-effective virtual clusters. In Proc. of ICA3PP, Vol. Part I, pages 395--408, 2011.

Digital Library

[15]

S. Yi, A. Andrzejak, and D. Kondo. Monetary cost-aware checkpointing and migration on Amazon cloud spot instances. IEEE Trans. Serv. Comput., 5(4):512--524, 2012.

Digital Library

[16]

S. Yi, D. Kondo, and A. Andrzejak. Reducing costs of spot instances via checkpointing in the Amazon elastic compute cloud. In IEEE CLOUD, pages 236--243, 2010.

Digital Library

[17]

Q. Zhang, E. Gürses, R. Boutaba, and J. Xiao. Dynamic resource allocation for spot markets in clouds. In Proc. of Hot-ICE, pages 1--1, 2011.

Digital Library

Cited By

Leis VKuschewski M(2021)Towards cost-optimal query processing in the cloudProceedings of the VLDB Endowment10.14778/3461535.346154914:9(1606-1612)Online publication date: 22-Oct-2021
https://dl.acm.org/doi/10.14778/3461535.3461549
Kumar DBaranwal GRaza ZVidyarthi D(2018)A Survey on Spot Pricing in Cloud ComputingJournal of Network and Systems Management10.1007/s10922-017-9444-x26:4(809-856)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10922-017-9444-x
Rosati PFox GKenny DLynn T(2017)Quantifying the Financial Value of Cloud Investments: A Systematic Literature Review2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2017.28(194-201)Online publication date: Dec-2017
https://doi.org/10.1109/CloudCom.2017.28
Show More Cited By

Index Terms

SpotADAPT: Spot-Aware (re-)Deployment of Analytical Processing Tasks on Amazon EC2
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

How Small and Medium Enterprises SMEs Should Bid for Spot Instances of Amazon's EC2 Cloud

In cloud service provisioning, spot instances are spare slots for which it has no pre-booking, unlike reserved or on-demand instances for which a cloud service provider CSP has a priori booking. CSPs like Amazon prefer spot instance approach to sell ...
WhoWas: A Platform for Measuring Web Deployments on IaaS Clouds
IMC '14: Proceedings of the 2014 Conference on Internet Measurement Conference

Public infrastructure-as-a-service (IaaS) clouds such as Amazon EC2 and Microsoft Azure host an increasing number of web services. The dynamic, pay-as-you-go nature of modern IaaS systems enable web services to scale up or down with demand, and only pay ...
Methods for improving the availability of spot instances: A survey
Abstract
The burgeoning development of the cloud market has promoted the expansion of resources held by cloud providers, but the resulting underutilization caused by the over-provisioned resources has become a challenge. To improve the ...
Highlights
- Describe the development history of spot instances and analyze what benefits the spot instance provides to cloud providers and users, respectively.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DOLAP '15: Proceedings of the ACM Eighteenth International Workshop on Data Warehousing and OLAP

October 2015

108 pages

ISBN:9781450337854

DOI:10.1145/2811222

General Chair:
Il-Yeol Song
Drexel University, USA
,
Program Chairs:
Carlos Garcia-Alvarado
Pivotal Software Inc., USA
,
Carlos Ordonez
University of Houston & ATT, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM'15

Sponsor:

CIKM'15: 24th ACM International Conference on Information and Knowledge Management

October 23, 2015

Melbourne, Australia

Acceptance Rates

DOLAP '15 Paper Acceptance Rate 8 of 31 submissions, 26%;

Overall Acceptance Rate 29 of 79 submissions, 37%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
180
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Leis VKuschewski M(2021)Towards cost-optimal query processing in the cloudProceedings of the VLDB Endowment10.14778/3461535.346154914:9(1606-1612)Online publication date: 22-Oct-2021
https://dl.acm.org/doi/10.14778/3461535.3461549
Kumar DBaranwal GRaza ZVidyarthi D(2018)A Survey on Spot Pricing in Cloud ComputingJournal of Network and Systems Management10.1007/s10922-017-9444-x26:4(809-856)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10922-017-9444-x
Rosati PFox GKenny DLynn T(2017)Quantifying the Financial Value of Cloud Investments: A Systematic Literature Review2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2017.28(194-201)Online publication date: Dec-2017
https://doi.org/10.1109/CloudCom.2017.28
Rajan KKakadia DCurino CKrishnan S(2016)PerfOratorProceedings of the Seventh ACM Symposium on Cloud Computing10.1145/2987550.2987566(415-427)Online publication date: 5-Oct-2016
https://dl.acm.org/doi/10.1145/2987550.2987566
Garcia-Alvarado COrdonez CSong IBailey JMoffat AAggarwal Cde Rijke MKumar RMurdock VSellis TYu J(2015)DOLAP 2015 Workshop SummaryProceedings of the 24th ACM International on Conference on Information and Knowledge Management10.1145/2806416.2806876(1939-1940)Online publication date: 17-Oct-2015
https://dl.acm.org/doi/10.1145/2806416.2806876

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents