extended-abstract

Public Access

A Computational Model for Tensor Core Units

Authors:

Rezaul Chowdhury,

Francesco Silvestri,

Flavio VellaAuthors Info & Claims

SPAA '20: Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures

Pages 519 - 521

https://doi.org/10.1145/3350755.3400252

Published: 09 July 2020 Publication History

PDF eReader

Abstract

To respond to the need for efficient training and inference of deep neural networks, a plethora of domain-specific architectures have been introduced, such as Google Tensor Processing Units and NVIDIA Tensor Cores. A common feature of these architectures is the design for efficiently computing a dense matrix product of a given small size. In order to broaden the class of algorithms that exploit these systems, we propose a computational model, named the TCU model, that captures the ability to natively multiply small matrices. We then use the TCU model for designing fast algorithms for several problems, including dense and sparse matrix multiplication and the Discrete Fourier Transform. We finally highlight a relation between the TCU model and the external memory model.

References

[1]

G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Graph expansion and communication costs of fast matrix multiplication. J. ACM, 59(6):32:1--32:23, 2013.

Digital Library

Google Scholar

[2]

R. A. Chowdhury, F. Silvestri, and F. Vella. A computational model for tensor core units, 2020. Arxiv 1908.06649.

Google Scholar

[3]

A. Dakkak, C. Li, J. Xiong, I. Gelado, and W.-M. Hwu. Accelerating reduction and scan using tensor core units. In Proc. Int. Conf. on Supercomputing (ICS), pages 46--57, 2019.

Digital Library

Google Scholar

[4]

R. Jacob and M. Stöckel. Fast output-sensitive matrix multiplication. In Proc. European Symposium on Algorithms (ESA), pages 766--778, 2015.

Crossref

Google Scholar

[5]

N. P. Jouppi et al. In-datacenter performance analysis of a tensor processing unit. In Proc. 44th Int. Symposium on Computer Architecture (ISCA), pages 1--12, 2017.

Digital Library

Google Scholar

[6]

Nvidia Tesla V100 GPU architecture. http://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf.

Google Scholar

[7]

R. Raz. On the complexity of matrix product. SIAM Journal on Computing, 32(5):1356--1369, 2003.

Digital Library

Google Scholar

[8]

A. Sorna, X. Cheng, E. D'Azevedo, K. Won, and S. Tomov. Optimizing the fast fourier transform using mixed precision on tensor core hardware. In Proc. 25th Int. Conf. on High Performance Computing Workshops (HiPCW), pages 3--7, 2018.

Crossref

Google Scholar

[9]

J. S. Vitter. Algorithms and data structures for external memory. Foundations and Trends in Theoretical Computer Science, 2(4):305--474, 2006.

Digital Library

Google Scholar

Cited By

View all

Hanindhito BJohn LBalsamo SKnottenbelt WAbad CShang W(2024)Accelerating ML Workloads using GPU Tensor Cores: The Good, the Bad, and the UglyProceedings of the 15th ACM/SPEC International Conference on Performance Engineering10.1145/3629526.3653835(178-189)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3629526.3653835
Lu YLiu WMohror KArnold DBadia R(2023)DASP: Specific Dense Matrix Multiply-Accumulate Units Accelerated General Sparse Matrix-Vector MultiplicationProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607051(1-14)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607051
Zouzias AMcColl W(2023)A Parallel Scan Algorithm in the Tensor Core Unit ModelEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_33(489-502)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-39698-4_33
Show More Cited By

Index Terms

A Computational Model for Tensor Core Units
1. Theory of computation
  1. Design and analysis of algorithms
  2. Models of computation

Recommendations

Efficient tensor core-based GPU kernels for structured sparsity under reduced precision
SC '21: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

The success of DNN comes at the expense of excessive memory/computation cost, which can be addressed by exploiting reduced precision and sparsity jointly. Existing sparse GPU kernels, however, fail to achieve practical speedup over cuBLASHgemm under half-...
Toward accelerated stencil computation by adapting tensor core unit on GPU
ICS '22: Proceedings of the 36th ACM International Conference on Supercomputing

The Tensor Core Unit (TCU) has been increasingly adopted on modern high performance processors, specialized in boosting the performance of general matrix multiplication (GEMM). Due to its highly optimized hardware design, TCU can significantly ...
EGEMM-TC: accelerating scientific computing on tensor cores with extended precision
PPoPP '21: Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

Nvidia Tensor Cores achieve high performance with half-precision matrix inputs tailored towards deep learning workloads. However, this limits the application of Tensor Cores especially in the area of scientific computing with high precision requirements. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SPAA '20: Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures

July 2020

601 pages

ISBN:9781450369350

DOI:10.1145/3350755

General Chair:
Christian Scheideler
Institut fuer Informatik Universitaet Paderborn Fuerstenallee 11
,
Program Chair:
Michael Spear

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2020

Check for updates

Author Tags

Qualifiers

Extended-abstract

Funding Sources

UniBZ-CRC
Università degli Studi di Padova
National Science Foundation
INdAM-GNCS
Ministero dellðIstruzione, dellðUniversità e della Ricerca

Conference

SPAA '20

Sponsor:

SPAA '20: 32nd ACM Symposium on Parallelism in Algorithms and Architectures

July 15 - 17, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 447 of 1,461 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
714
Total Downloads

Downloads (Last 12 months)253
Downloads (Last 6 weeks)42

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hanindhito BJohn LBalsamo SKnottenbelt WAbad CShang W(2024)Accelerating ML Workloads using GPU Tensor Cores: The Good, the Bad, and the UglyProceedings of the 15th ACM/SPEC International Conference on Performance Engineering10.1145/3629526.3653835(178-189)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3629526.3653835
Lu YLiu WMohror KArnold DBadia R(2023)DASP: Specific Dense Matrix Multiply-Accumulate Units Accelerated General Sparse Matrix-Vector MultiplicationProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607051(1-14)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607051
Zouzias AMcColl W(2023)A Parallel Scan Algorithm in the Tensor Core Unit ModelEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_33(489-502)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-39698-4_33
Chowdhury RSilvestri FVella F(2021)Algorithm Design for Tensor UnitsEuro-Par 2021: Parallel Processing10.1007/978-3-030-85665-6_22(353-367)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-85665-6_22
D. Ahle TSilvestri F(2020)Similarity Search with Tensor Core UnitsSimilarity Search and Applications10.1007/978-3-030-60936-8_6(76-84)Online publication date: 14-Oct-2020
https://doi.org/10.1007/978-3-030-60936-8_6

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Efficient tensor core-based GPU kernels for structured sparsity under reduced precision

Toward accelerated stencil computation by adapting tensor core unit on GPU

EGEMM-TC: accelerating scientific computing on tensor cores with extended precision