extended-abstract

clSPARSE: A Vendor-Optimized Open-Source Sparse BLAS Library

Authors:

Joseph L. Greathouse,

Kent Knox,

Jakub Poła,

Kiran Varaganti,

Mayank DagaAuthors Info & Claims

IWOCL '16: Proceedings of the 4th International Workshop on OpenCL

Article No.: 7, Pages 1 - 4

https://doi.org/10.1145/2909437.2909442

Published: 19 April 2016 Publication History

Get Access

Abstract

Sparse linear algebra is a cornerstone of modern computational science. These algorithms ignore the zero-valued entries found in many domains in order to work on much larger problems at much faster rates than dense algorithms. Nonetheless, optimizing these algorithms is not straightforward. Highly optimized algorithms for multiplying a sparse matrix by a dense vector, for instance, are the subject of a vast corpus of research and can be hundreds of times longer than naïve implementations. Optimized sparse linear algebra libraries are thus needed so that users can build applications without enormous effort.

Hardware vendors release proprietary libraries that are highly optimized for their devices, but they limit interoperability and promote vendor lock-in. Open libraries often work across multiple devices and can quickly take advantage of new innovations, but they may not reach peak performance. The goal of this work is to provide a sparse linear algebra library that offers both of these advantages.

We thus describe clSPARSE, a permissively licensed open-source sparse linear algebra library that offers state-of-the-art optimized algorithms implemented in OpenCL™. We test clSPARSE on GPUs from AMD and Nvidia and show performance benefits over both the proprietary cuSPARSE library and the open-source ViennaCL library.

References

[1]

H. M. Aktulga, A. Buluç, S. Williams, and C. Yang. Optimizing Sparse Matrix-Multiple Vectors Multiplication for Nuclear Configuration Interaction Calculations. In Proc. of the Int'l Parallel and Distributed Processing Symposium (IPDPS), 2014.

Digital Library

Google Scholar

[2]

M. Daga and J. L. Greathouse. Structural Agnostic SpMV: Adapting CSR-Adaptive for Irregular Matrices. In Proc. of the Int'l Conf. on High Performance Computing (HiPC), 2015.

Digital Library

Google Scholar

[3]

I. S. Duff, M. A. Heroux, and R. Pozo. An Overview of the Sparse Basic Linear Algebra Subprograms: The New Standard from the BLAS Technical Forum. Trans. on Mathematical Software, 28(2):239--267, 2002.

Digital Library

Google Scholar

[4]

J. L. Greathouse and M. Daga. Efficient Sparse Matrix-Vector Multiplication on GPUs Using the CSR Storage Format. In Proc. of the Int'l Conf. for High Performance Computing, Networking, Storage and Analysis (SC), 2014.

Digital Library

Google Scholar

[5]

W. D. Gropp, D. K. Kaushik, D. E. Keyes, and B. F. Smith. Towards Realistic Performance Bounds for Implicit CFD Codes. In Proc. of the Int'l Parallel Computational Fluid Dynamics Conf. (PARCFD), 1999.

Google Scholar

[6]

A. Kaiser, S. Williams, K. Madduri, K. Ibrahim, D. H. Bailey, J. W. Demmel, and E. Strohmaier. TORCH Computational Reference Kernels: A Testbed for Computer Science Research. Technical Report LBNL-4172E, Lawrence Berkeley National Laboratory, 2010.

Crossref

Google Scholar

[7]

M. Kreutzer, G. Hager, G. Wellein, H. Fehske, and A. R. Bishop. A Unified Sparse Matrix Data Format for Modern Processors with Wide SIMD Units. SIAM Journal on Scientific Computing, 36(5):C401--C423, 2014.

Crossref

Google Scholar

[8]

D. Langr and P. Tvrdík. Evaluation Criteria for Sparse Matrix Storage Formats. IEEE Trans. on Parallel and Distributed Systems, 27(2):428--440, Feb. 2016.

Digital Library

Google Scholar

[9]

W. Liu and B. Vinter. An Efficient GPU General Sparse Matrix-Matrix Multiplication for Irregular Data. In Proc. of the Int'l Parallel and Distributed Processing Symp. (IPDPS), 2014.

Digital Library

Google Scholar

[10]

K. Rupp, F. Rudolf, and J. Weinbub. ViennaCL - A High Level Linear Algebra Library for GPUs and Multi-Core CPUs. In Int'l Workshop on GPUs and Scientific Applications (GPUScA), 2010.

Google Scholar

[11]

K. Rupp, J. Weinbub, A. Jüngel, and T. Grasser. Pipelined Iterative Solvers with Kernel Fusion for Graphics Processing Units. CoRR, abs/1410.4054, 2014.

Google Scholar

[12]

B.-Y. Su and K. Keutzer. clSpMV: A Cross-Platform OpenCL SpMV Framework on GPUs. In Proc. of the Int'l Conf. on Supercomputing (ICS), 2012.

Digital Library

Google Scholar

Cited By

View all

Mittenbühler MZhang JBenigni A(2024)Automatically optimized component model computation for power system simulation on GPUElectric Power Systems Research10.1016/j.epsr.2024.110740235(110740)Online publication date: Oct-2024
https://doi.org/10.1016/j.epsr.2024.110740
Trias FÁlvarez-Farré XAlsalti-Baldellou ÀGorobets AOliva A(2023)An Efficient Eigenvalue Bounding Method: Cfl Condition RevisitedSSRN Electronic Journal10.2139/ssrn.4353590Online publication date: 2023
https://doi.org/10.2139/ssrn.4353590
Orachev EKarpenko MAlimov PGrigorev S(2022)SPbLA: The Library of GPGPU-powered Sparse Boolean Linear Algebra OperationsJournal of Open Source Software10.21105/joss.037437:76(3743)Online publication date: Aug-2022
https://doi.org/10.21105/joss.03743
Show More Cited By

Recommendations

On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing
SAAHPC '11: Proceedings of the 2011 Symposium on Application Accelerators in High-Performance Computing

The graphics processing unit (GPU) has made significant strides as an accelerator in parallel computing. However, because the GPU has resided out on PCIe as a discrete device, the performance of GPU applications can be bottlenecked by data transfers ...
Twin peaks: a software platform for heterogeneous computing on general-purpose and graphics processors
PACT '10: Proceedings of the 19th international conference on Parallel architectures and compilation techniques

Modern processors are evolving into hybrid, heterogeneous processors with both CPU and GPU cores used for general purpose computation. Several languages such as Brook, CUDA, and more recently OpenCL are being developed to fully harness the potential of ...
Caracal: dynamic translation of runtime environments for GPUs
GPGPU-4: Proceedings of the Fourth Workshop on General Purpose Processing on Graphics Processing Units

Graphics Processing Units (GPU) have become the platform of choice for accelerating a large range of data parallel and task parallel applications. Both AMD and NVIDIA have developed GPU implementations targeted at the high performance computing market. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IWOCL '16: Proceedings of the 4th International Workshop on OpenCL

April 2016

131 pages

ISBN:9781450343381

DOI:10.1145/2909437

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

In-Cooperation

The University of Bristol: The University of Bristol

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2016

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

IWOCL '16

IWOCL '16: The 4th International Workshop on OpenCL

April 19 - 21, 2016

Vienna, Austria

Acceptance Rates

Overall Acceptance Rate 84 of 152 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
153
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Mittenbühler MZhang JBenigni A(2024)Automatically optimized component model computation for power system simulation on GPUElectric Power Systems Research10.1016/j.epsr.2024.110740235(110740)Online publication date: Oct-2024
https://doi.org/10.1016/j.epsr.2024.110740
Trias FÁlvarez-Farré XAlsalti-Baldellou ÀGorobets AOliva A(2023)An Efficient Eigenvalue Bounding Method: Cfl Condition RevisitedSSRN Electronic Journal10.2139/ssrn.4353590Online publication date: 2023
https://doi.org/10.2139/ssrn.4353590
Orachev EKarpenko MAlimov PGrigorev S(2022)SPbLA: The Library of GPGPU-powered Sparse Boolean Linear Algebra OperationsJournal of Open Source Software10.21105/joss.037437:76(3743)Online publication date: Aug-2022
https://doi.org/10.21105/joss.03743
Psarras CBarthels HBientinesi P(2022)The Linear Algebra Mapping Problem. Current State of Linear Algebra Languages and LibrariesACM Transactions on Mathematical Software10.1145/354993548:3(1-30)Online publication date: 20-Jul-2022
https://dl.acm.org/doi/10.1145/3549935
Oh CSo JKim SYi Y(2021)Exploiting Activation Sparsity for Fast CNN Inference on Mobile GPUsACM Transactions on Embedded Computing Systems10.1145/347700820:5s(1-25)Online publication date: 17-Sep-2021
https://dl.acm.org/doi/10.1145/3477008
Niu WLi ZMa XDong PZhou GQian XLin XWang YRen B(2021)GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight SparsityIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.3089687(1-1)Online publication date: 2021
https://doi.org/10.1109/TPAMI.2021.3089687
Orachev EKarpenko MKhoroshev AGrigorev S(2021)SPbLA: The Library of GPGPU-Powered Sparse Boolean Linear Algebra Operations2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW52791.2021.00049(272-275)Online publication date: Jun-2021
https://doi.org/10.1109/IPDPSW52791.2021.00049
Valle NÁlvarez-Farré XGorobets ACastro JOliva AXavier Trias F(2021)On the implementation of flux limiters in algebraic frameworksComputer Physics Communications10.1016/j.cpc.2021.108230(108230)Online publication date: Nov-2021
https://doi.org/10.1016/j.cpc.2021.108230
Choi KYang H(2021)A GPU Architecture Aware Fine-Grain Pruning Technique for Deep Neural NetworksEuro-Par 2021: Parallel Processing10.1007/978-3-030-85665-6_14(217-231)Online publication date: 25-Aug-2021
https://doi.org/10.1007/978-3-030-85665-6_14
Lee KKim HLee HShin D(2020)Flexible Group-Level Pruning of Deep Neural Networks for On-Device Machine Learning2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE48585.2020.9116287(79-84)Online publication date: Mar-2020
https://doi.org/10.23919/DATE48585.2020.9116287
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing

Twin peaks: a software platform for heterogeneous computing on general-purpose and graphics processors

Caracal: dynamic translation of runtime environments for GPUs

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations