[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1109/ICDE.2011.5767930guideproceedingsArticle/Chapter ViewAbstractPublication PagesConference Proceedingsacm-pubtype
Article

SystemML: Declarative machine learning on MapReduce

Published: 11 April 2011 Publication History

Abstract

MapReduce is emerging as a generic parallel programming paradigm for large clusters of machines. This trend combined with the growing need to run machine learning (ML) algorithms on massive datasets has led to an increased interest in implementing ML algorithms on MapReduce. However, the cost of implementing a large class of ML algorithms as low-level MapReduce jobs on varying data and machine cluster sizes can be prohibitive. In this paper, we propose SystemML in which ML algorithms are expressed in a higher-level language and are compiled and executed in a MapReduce environment. This higher-level language exposes several constructs including linear algebra primitives that constitute key building blocks for a broad class of supervised and unsupervised ML algorithms. The algorithms expressed in SystemML are compiled and optimized into a set of MapReduce jobs that can run on a cluster of machines. We describe and empirically evaluate a number of optimization strategies for efficiently executing these algorithms on Hadoop, an open-source MapReduce implementation. We report an extensive performance evaluation on three ML algorithms on varying data and cluster sizes.

Cited By

View all

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings
ICDE '11: Proceedings of the 2011 IEEE 27th International Conference on Data Engineering
April 2011
1457 pages
ISBN:9781424489596

Publisher

IEEE Computer Society

United States

Publication History

Published: 11 April 2011

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2022)Scalable Graph Convolutional Network Training on Distributed-Memory SystemsProceedings of the VLDB Endowment10.14778/3574245.357425616:4(711-724)Online publication date: 1-Dec-2022
  • (2022)Hyper-tuneProceedings of the VLDB Endowment10.14778/3514061.351407115:6(1256-1265)Online publication date: 1-Feb-2022
  • (2022)Credit Risk Measurement, Decision Analysis, Transformation and Upgrading for Financial Big DataComplexity10.1155/2022/89427732022Online publication date: 1-Jan-2022
  • (2021)VolcanoMLProceedings of the VLDB Endowment10.14778/3476249.347627014:11(2167-2176)Online publication date: 27-Oct-2021
  • (2021)Tensor relational algebra for distributed machine learning system designProceedings of the VLDB Endowment10.14778/3457390.345739914:8(1338-1350)Online publication date: 1-Apr-2021
  • (2021)Distributed numerical and machine learning computations via two-phase execution of aggregated join treesProceedings of the VLDB Endowment10.14778/3450980.345099114:7(1228-1240)Online publication date: 12-Apr-2021
  • (2021)Handling Iterations in Distributed Dataflow SystemsACM Computing Surveys10.1145/347760254:9(1-38)Online publication date: 8-Oct-2021
  • (2021)The Power of Nested Parallelism in Big Data Processing – Hitting Three Flies with One Slap –Proceedings of the 2021 International Conference on Management of Data10.1145/3448016.3457287(605-618)Online publication date: 9-Jun-2021
  • (2021)Hybrid Evaluation for Distributed Iterative Matrix ComputationProceedings of the 2021 International Conference on Management of Data10.1145/3448016.3452843(300-312)Online publication date: 9-Jun-2021
  • (2020)Synthesis of Incremental Linear Algebra ProgramsACM Transactions on Database Systems10.1145/338539845:3(1-44)Online publication date: 26-Aug-2020
  • Show More Cited By

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media