research-article

Free access

Exascale computing and big data

Authors:

Daniel A. Reed,

Jack DongarraAuthors Info & Claims

Communications of the ACM, Volume 58, Issue 7

Pages 56 - 68

https://doi.org/10.1145/2699414

Published: 25 June 2015 Publication History

All formats PDF

Abstract

Scientific discovery and engineering innovation requires unifying traditionally separated high-performance computing and big data analytics.

References

[1]

Amarasinghe, S. et al. Exascale Software Study: Software Challenges in Extreme-Scale Systems. Defense Advanced Research Projects Agency, Arlington, VA, 2009; http://www.cs.rice.edu/~vs3/PDF/Sarkar-ACS-July-2011-v2.pdf

Google Scholar

[2]

American Association for the Advancement of Science. Guide to R&D Funding - Historical Data. AAAS, Washington, D.C., 2015; http://www.aaas.org/page/historical-trends-federal-rd

Google Scholar

[3]

Chang, F. et al. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems 26, 2 (June 2008), 4:1--4:26.

Digital Library

Google Scholar

[4]

Datta, K. et al. Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures. In Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (Austin, TX, Nov. 15--21). IEEE Press, Piscataway, NJ, 2008, 1--12.

Digital Library

Google Scholar

[5]

Dennard, R.H., Gaensslen, F.H., Yu, H.-n., Rideout, V.L., Bassous, E., and LeBlanc, A.R. Design of ion-implanted MOSFETs with very small physical dimensions. IEEE Journal of Solid State Circuits 9, 5 (Jan. 1974), 256--268.

Crossref

Google Scholar

[6]

Dongarra, J.J. The LINPACK benchmark: An explanation. In Proceedings of the First International Conference on Supercomputing (Athens, Greece, June 8--12). Springer-Verlag, New York, 1988, 456--474.

Digital Library

Google Scholar

[7]

Dongarra, J.J. et al. The international exascale software project roadmap. International Journal of High Performance Computing Applications 25, 1 (Feb. 2011), 3--60.

Digital Library

Google Scholar

[8]

Esmaeilzadeh, H., Blem, E., Amant, R.S., Sankaralingam, K., and Burger, D. Dark silicon and the end of multicore scaling. In Proceedings of the 38^th Annual International Symposium on Computer Architecture (San Jose, CA, June 4--8). ACM, New York, 2011, 365--376.

Digital Library

Google Scholar

[9]

Fuller, S.H. and Millett, L.I. Computing performance: Game over or next level? Computer 44, 1 (Jan. 2011), 31--38.

Digital Library

Google Scholar

[10]

Geist, A. and Lucas, R. Major computer science challenges at exascale. International Journal of High Performance Applications 23, 4 (Nov. 2009), 427--436.

Digital Library

Google Scholar

[11]

Gill, P., Jain, N., and Nagappan, N. Understanding network failures in data centers: Measurement, analysis, and implications. Proceedings of ACM SIGCOMM 41, 4 (Aug, 2011), 350--361.

Digital Library

Google Scholar

[12]

Hey, T., Tansley, S., and Tolle, K. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond, WA, 2009; http://research.microsoft.com/en-us/UM/redmond/about/collaboration/fourthparadigm/4th_PARADIGM_BOOK_complete_HR.pdf

Google Scholar

[13]

Kamil, S., Shalf, J., and Strohmaier, E. Power efficiency in high-performance computing. In Proceedings of the Fourth Workshop on High-Performance, Power-Aware Computing (Miami, FL, Apr.). IEEE Press, 2008.

Crossref

Google Scholar

[14]

Kogge, P., Bergman, K., Borkar, S. et al. Exascale Computing Study: Technology Challenges in Achieving Exascale Systems. U.S. Defense Advanced Research Projects Agency, Arlington, VA, 2008; http://www.cse.nd.edu/Reports/2008/TR-2008-13.pdf

Google Scholar

[15]

Lucas, R., Ang, J., Bergman, K., Borkar, S. et al. Top Ten Exascale Research Challenges. Office of Science, U.S. Department of Energy, Washington, D.C., Feb. 2014; http://science.energy.gov/~/media/ascr/ascac/pdf/meetings/20140210/Top10reportFEB14.pdf

Google Scholar

[16]

Meuer, H., Strohmaier, E., Dongarra, J. and Simon, H. Top 500 Supercomputer Sites, 2015; http://www.top500.org

Google Scholar

[17]

Nobelprize.org. Nobel Prize in Chemistry 2013; http://www.nobelprize.org/nobel_prizes/chemistry/laureates/2013/press.html

Google Scholar

[18]

Olston, C., Reed, B., Strivastava, U., Kumar, R., and Tomkins, A. Pig Latin: A not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (Vancouver, BC, Canada, June 9-12). ACM Press, New York, 2008, 1099--1110.

Digital Library

Google Scholar

[19]

Partnership for Advanced Computing in Europe (PRACE). 2014; http://www.prace-ri.eu/

Google Scholar

[20]

Pinheiro, E., Weber, W.-D., and Barroso, L.A. Failure trends in a large disk drive population. In Proceedings of the Fifth USENIX Conference on File and Storage Technologies (San Jose, CA, Feb. 13--16). USENIX Association, Berkeley, CA, 2007.

Digital Library

Google Scholar

[21]

Schroeder, B. and Gibson, G.A. Understanding disk failure rates: What does an MTTF of 1,000,000 hours mean to you? ACM Transactions on Storage 3, 3 (Oct. 2007), 8.

Digital Library

Google Scholar

[22]

Schroeder, B., Pinheiro, E., and Weber, W.-D. DRAM errors in the wild: A large-scale field study. In Proceedings of the 11^th International Joint Conference on Measurement and Modeling of Computer Systems (Seattle, WA, June). ACM Press, New York, 2009, 193--204.

Digital Library

Google Scholar

[23]

U.S. Department of Energy. Synergistic Challenges in Data-Intensive Science and Exascale Computing. Report of the Advanced Scientific Computing Advisory Committee Subcommittee, Mar. 30, 2013; http://science.energy.gov/~/media/ascr/ascac/pdf/reports/2013/ASCAC_Data_Intensive_Computing_report_final.pdf

Google Scholar

[24]

U.S. Department of Energy. The Opportunities and Challenges of Exascale Computing. Office of Science, Washington, D.C., 2010; http://science.energy.gov/~/media/ascr/ascac/pdf/reports/Exascale_subcommittee_report.pdf

Google Scholar

[25]

White, T. Hadoop: The Definitive Guide. O'Reilly Media, May 2012.

Digital Library

Google Scholar

Cited By

View all

Robert-Hayek SZertal SCouvée P(2025)EVADyR: A new dynamic resampling algorithm for auto-tuning noisy High Performance Computing systemsJournal of Computational Science10.1016/j.jocs.2024.10246884(102468)Online publication date: Jan-2025
https://doi.org/10.1016/j.jocs.2024.102468
Carracciuolo LD’Amora U(2024)Mathematical Tools for Simulation of 3D Bioprinting Processes on High-Performance Computing Resources: The State of the ArtApplied Sciences10.3390/app1414611014:14(6110)Online publication date: 13-Jul-2024
https://doi.org/10.3390/app14146110
Montesano FMarotta RQuaglia F(2024)Spatial/Temporal Locality-Based Load-sharing in Speculative Discrete Event Simulation on Multi-core MachinesACM Transactions on Modeling and Computer Simulation10.1145/363970335:1(1-31)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3639703
Show More Cited By

Index Terms

Exascale computing and big data

Recommendations

The anatomy of big data computing

Advances in information technology and its widespread growth in several areas of business, engineering, medical, and scientific studies are resulting in information/data explosion. Knowledge discovery and decision-making from such rapidly growing ...
Big Data Analytics
Next-Generation Big Data: A Practical Guide to Apache Kudu, Impala, and Spark

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Communications of the ACM Volume 58, Issue 7

July 2015

102 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/2797100

Editor:
Moshe Y. Vardi
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2015

Published in CACM Volume 58, Issue 7

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

319
Total Citations
View Citations
25,148
Total Downloads

Downloads (Last 12 months)505
Downloads (Last 6 weeks)84

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Robert-Hayek SZertal SCouvée P(2025)EVADyR: A new dynamic resampling algorithm for auto-tuning noisy High Performance Computing systemsJournal of Computational Science10.1016/j.jocs.2024.10246884(102468)Online publication date: Jan-2025
https://doi.org/10.1016/j.jocs.2024.102468
Carracciuolo LD’Amora U(2024)Mathematical Tools for Simulation of 3D Bioprinting Processes on High-Performance Computing Resources: The State of the ArtApplied Sciences10.3390/app1414611014:14(6110)Online publication date: 13-Jul-2024
https://doi.org/10.3390/app14146110
Montesano FMarotta RQuaglia F(2024)Spatial/Temporal Locality-Based Load-sharing in Speculative Discrete Event Simulation on Multi-core MachinesACM Transactions on Modeling and Computer Simulation10.1145/363970335:1(1-31)Online publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1145/3639703
Taffoni GMignone ATornatore LSciacca EGuarrasi MLapenta GRiha LVavrik RVysocky OKadlubiak KStrakos PJaros MDolag KCommercon BRezzolla LPierre KDoulis GShen SMarazakis MGregori DBoella EPerna GZanotti MRaffin EPolsterer KTrujillo Gomez SMarin G(2024)Redesign of astrophysical codes for exascale computing: the SPACE experienceSoftware and Cyberinfrastructure for Astronomy VIII10.1117/12.3020361(32)Online publication date: 25-Jul-2024
https://doi.org/10.1117/12.3020361
Rajnoha RHadač J(2024)Strategic Key Elements in Big Data Analytics as Driving Forces of IoT Manufacturing Value Creation: A Challenge for Research FrameworkIEEE Transactions on Engineering Management10.1109/TEM.2021.311350271(90-105)Online publication date: 2024
https://doi.org/10.1109/TEM.2021.3113502
Devarajan HMoody ADai DStanavige CGonsiorowski EMcFadden MFaaland OKosinovsky GMohror K(2024)The Impact of Asynchronous I/O in Checkpoint-Restart Workloads2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW63119.2024.00089(397-405)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPSW63119.2024.00089
Vivas ATchernykh ACastro H(2024)Trends, Approaches, and Gaps in Scientific Workflow Scheduling: A Systematic ReviewIEEE Access10.1109/ACCESS.2024.350921812(182203-182231)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3509218
Mehrabinezhad ATeshnehlab MSharifi A(2024)A comparative study to examine principal component analysis and kernel principal component analysis-based weighting layer for convolutional neural networksComputer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization10.1080/21681163.2024.237952612:1Online publication date: 22-Jul-2024
https://doi.org/10.1080/21681163.2024.2379526
Abdali KSharifi MMousavi Khaneghah E(2024)A reinforcement learning-based mechanism for managing dynamic and interactive events affecting the functionality of load balancing in HPC systemsExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124818255:PCOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124818
Dele-Afolabi TMohamed Ariff AOjo-Kupoluyi OAtoyebi E(2024)Application of neural networks and artificial intelligence tools for modelling, characterization, and forecasting in materials engineeringComprehensive Materials Processing10.1016/B978-0-323-96020-5.00004-2(44-55)Online publication date: 2024
https://doi.org/10.1016/B978-0-323-96020-5.00004-2
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF Chinese translation

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

The anatomy of big data computing

Big Data Analytics

Next-Generation Big Data: A Practical Guide to Apache Kudu, Impala, and Spark

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations