[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

Klasky et al., 2011 - Google Patents

In situ data processing for extreme-scale computing

Klasky et al., 2011

View PDF
Document ID
9506500184258912188
Author
Klasky S
Abbasi H
Logan J
Parashar M
Schwan K
Shoshani A
Wolf M
Ahern S
Altintas I
Bethel W
Chacon L
Chang10 C
Chen J
Childs H
Cummings13 J
Ethier10 S
Grout R
Lin10 Z
Liu Q
Ma15 X
Moreland K
Pascucci12 V
Podhorszki N
Samatova15 N
Schroeder W
Tchoua R
Wu J
Yu14 W
Publication year
Publication venue
Proceedings of SciDAC

External Links

Snippet

The DOE leadership facilities were established in 2004 to provide scientist capability computing for high-profile science. Since it's inception, the systems went from 14 TF to 1.8 PF, an increase of 100 in 5 years, and will increase by another factor of 100 in 5 more years …
Continue reading at www.academia.edu (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30386—Retrieval requests
    • G06F17/30424—Query processing
    • G06F17/30442—Query optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30587—Details of specialised database models
    • G06F17/30592—Multi-dimensional databases and data warehouses, e.g. MOLAP, ROLAP
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30386—Retrieval requests
    • G06F17/30389—Query formulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30575—Replication, distribution or synchronisation of data between databases or within a distributed database; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30312—Storage and indexing structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30289—Database design, administration or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformations of program code
    • G06F8/41—Compilation
    • G06F8/44—Encoding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/76—Adapting program code to run in a different environment; Porting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/50—Computer-aided design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F1/00—Details of data-processing equipment not covered by groups G06F3/00 - G06F13/00, e.g. cooling, packaging or power supply specially adapted for computer application

Similar Documents

Publication Publication Date Title
Klasky et al. In situ data processing for extreme-scale computing
Zheng et al. PreDatA–preparatory data analytics on peta-scale machines
US10176236B2 (en) Systems and methods for a distributed query execution engine
US20160034547A1 (en) Systems and methods for an sql-driven distributed operating system
Pedreira et al. Velox: meta's unified execution engine
Boyuka et al. Transparent in situ data transformations in adios
Sinthong et al. Aframe: Extending dataframes for large-scale modern data analysis
Poeschel et al. Transitioning from file-based HPC workflows to streaming data pipelines with openPMD and ADIOS2
Peterka et al. ASCR workshop on in situ data management: Enabling scientific discovery from diverse data sources
Alkin et al. ALICE Run 3 analysis framework
Suter et al. Driving Next-Generation Workflows from the Data Plane
Klasky et al. A view from ORNL: Scientific data research opportunities in the big data age
AlBar et al. A journey towards building real-time big data analytics environment for drilling operations: Challenges and lessons learned
Kress In situ visualization techniques for high performance computing
Peterka et al. LowFive: In Situ Data Transport for High-Performance Workflows
Zheng et al. Flexio: Location-flexible execution of in situ data analytics for large scale scientific applications
Kantere A holistic framework for big scientific data management
Lee et al. A Case Study of Data Management Challenges Presented in Large-Scale Machine Learning Workflows
Ross et al. Storage systems and input/output: Organizing, storing, and accessing data for scientific discovery. report for the doe ascr workshop on storage systems and i/o.[full workshop report]
Kress In-line vs. in-transit in situ: Which technique to use at scale?
Bicer et al. Improving I/O throughput of scientific applications using transparent parallel compression
Pugmire et al. The Adaptable IO System (ADIOS)
Wang et al. Efficient clustered server-side data analysis workflows using SWAMP
Lustosa et al. SAVIME: A multidimensional system for the analysis and visualization of simulation data
Zhang et al. Assembling portable in-situ workflow from heterogeneous components using data reorganization