Article

A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds

Authors:

Norman Lim,

Shikharesh Majumdar,

Peter Ashwood-SmithAuthors Info & Claims

BRACIS '14: Proceedings of the 2014 Brazilian Conference on Intelligent Systems

Pages 411 - 421

https://doi.org/10.1109/ICPP.2014.50

Published: 09 September 2014 Publication History

Abstract

Clouds that are rapidly gaining in popularity require an effective resource manager that can harness the power of the underlying resource pool, and provide resources on demand to its users. This paper focuses on resource management on clouds for workflow requests characterized by Service Level Agreements (SLAs). Specifically, we devise a novel MapReduce constraint programming based resource manager (MRCP-RM) that can effectively perform matchmaking and scheduling of MapReduce jobs, each characterized by an SLA comprising an earliest start time, execution time, and an end-to-end deadline. Using discrete event simulation a performance evaluation of MRCP-RM is conducted for an open system subjected to a stream of job arrivals. The simulation results demonstrate the effectiveness of the resource manager and provide insights into system behaviour and performance.

Cited By

View all

Zeng XGarg SBarika MZomaya AWang LVillari MChen DRanjan R(2020)SLA Management for Big Data Analytical Applications in CloudsACM Computing Surveys10.1145/338346453:3(1-40)Online publication date: 12-Jun-2020
https://dl.acm.org/doi/10.1145/3383464
Carbonnel CCohen DCooper MŽivný S(2019)On Singleton Arc Consistency for CSPs Defined by Monotone PatternsAlgorithmica10.1007/s00453-018-0498-281:4(1699-1727)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1007/s00453-018-0498-2
Vinueza Naranjo PBaccarelli EScarpiniti M(2018)Design and energy-efficient resource management of virtualized networked Fog architectures for the real-time support of IoT applicationsThe Journal of Supercomputing10.1007/s11227-018-2274-074:6(2470-2507)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1007/s11227-018-2274-0
Show More Cited By

A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning

Recommendations

A Constraint Programming Based Energy Aware Resource Management Middleware for Clouds Processing MapReduce Jobs with Deadlines
ICPE '16 Companion: Companion Publication for ACM/SPEC on International Conference on Performance Engineering

This paper concerns guarantees on system performance through Service Level Agreement (SLA) compliance and focuses on devising energy aware resource management techniques based on Dynamic Voltage and Frequency Scaling (DVFS) used by resource management ...
A Constraint Programming Based Hadoop Scheduler for Handling MapReduce Jobs with Deadlines on Clouds
ICPE '15: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

A novel MapReduce constraint programming based matchmaking and scheduling algorithm (MRCP) that can handle MapReduce jobs with deadlines and achieve high system performance is devised. The MRCP algorithm is incorporated into Hadoop, which is a widely ...
Techniques for Handling Error in User-estimated Execution Times During Resource Management on Systems Processing MapReduce Jobs
CCGrid '17: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

In our previous work, we described a resource allocation and scheduling technique for processing an open stream of MapReduce jobs with SLAs (characterized by an earliest start time, an execution time, and a deadline) called the Hadoop Constraint ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

BRACIS '14: Proceedings of the 2014 Brazilian Conference on Intelligent Systems

October 2014

914 pages

ISBN:9781479956180

Publisher

IEEE Computer Society

United States

Publication History

Published: 09 September 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zeng XGarg SBarika MZomaya AWang LVillari MChen DRanjan R(2020)SLA Management for Big Data Analytical Applications in CloudsACM Computing Surveys10.1145/338346453:3(1-40)Online publication date: 12-Jun-2020
https://dl.acm.org/doi/10.1145/3383464
Carbonnel CCohen DCooper MŽivný S(2019)On Singleton Arc Consistency for CSPs Defined by Monotone PatternsAlgorithmica10.1007/s00453-018-0498-281:4(1699-1727)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1007/s00453-018-0498-2
Vinueza Naranjo PBaccarelli EScarpiniti M(2018)Design and energy-efficient resource management of virtualized networked Fog architectures for the real-time support of IoT applicationsThe Journal of Supercomputing10.1007/s11227-018-2274-074:6(2470-2507)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1007/s11227-018-2274-0
Lim NMajumdar SAshwood-Smith P(2017)A resource management technique for processing deadline-constrained multi-stage workflowsJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-017-0091-26:1(1-24)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1186/s13677-017-0091-2
Lim NMajumdar SAshwood-Smith P(2017)MRCP-RMIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2016.261732428:5(1375-1389)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TPDS.2016.2617324
Morla RGonçalves PBarbosa J(2016)High-performance network traffic analysis for continuous batch intrusion detectionThe Journal of Supercomputing10.1007/s11227-016-1743-672:11(4107-4128)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1007/s11227-016-1743-6

Abstract

Cited By

Recommendations

A Constraint Programming Based Energy Aware Resource Management Middleware for Clouds Processing MapReduce Jobs with Deadlines

A Constraint Programming Based Hadoop Scheduler for Handling MapReduce Jobs with Deadlines on Clouds

Techniques for Handling Error in User-estimated Execution Times During Resource Management on Systems Processing MapReduce Jobs

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations