Drishyam: An Image is Worth a Data Prefetcher
Pages 51 - 61
Abstract
Hardware prefetching is a latency-hiding technique that hides the costly off-chip DRAM accesses. Although hardware prefetching is an extensively researched topic with many state-of-the-art data prefetchers pushing the performance limits, prefetching for irregular applications with hard-to-predict access patterns is still a challenging problem to solve. The usage of neural networks for hardware prefetching is a promising direction, especially for predicting irregular memory access patterns. This paper presents Drishyam, a novel hardware prefetcher based on computer vision algorithms that use images to learn memory access patterns and predict future memory accesses with high accuracy and coverage. For hardware prefetching, an image is a graphical representation of memory accesses observed over time. For a sequence of memory addresses, Drishyam creates images that predict the future addresses by predicting the future OS page and a cache line offset within the OS page. Drishyam outperforms Voyager, the state-of-the-art machine learning (ML) based prefetcher, for a set of irregular benchmarks by an average of 4.7% with an average prefetch accuracy and prefetch coverage of 89.5% and 66.6%, respectively. In terms of training time, Drishyam outperforms Voyager by 225.5%.
References
[1]
A. Navarro-Torres, B. Panda, J. Alastruey-Benedé, P. Ibáñez, V. V. Yúfera, and A. Ros, "Berti: an accurate local-delta data prefetcher", in 55th IEEE/ACM International Symposium on Microarchitecture, MICRO 2022, Chicago, IL, USA, October 1-5, 2022, pp. 975--991, IEEE, 2022.
[2]
S. Pakalapati and B. Panda, "Bouquet of instruction pointers: Instruction pointer classifier-based spatial hardware prefetching", in 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pp. 118--131, IEEE, 2020.
[3]
J. Kim, S. H. Pugsley, P. V. Gratz, A. N. Reddy, C. Wilkerson, and Z. Chishti, "Path confidence based lookahead prefetching", in 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 1--12, IEEE, 2016.
[4]
M. Bakhshalipour, M. Shakerinava, P. Lotfi-Kamran, and H. Sarbazi-Azad, "Bingo spatial data prefetcher", in 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 399--411, IEEE, 2019.
[5]
M. Shevgoor, S. Koladiya, R. Balasubramonian, C. Wilkerson, S. H. Pugsley, and Z. Chishti, "Efficiently prefetching complex address patterns", in 2015 48th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 141--152, 2015.
[6]
L. Peled, S. Mannor, U. Weiser, and Y. Etsion, "Semantic locality and context-based prefetching using reinforcement learning", in Proceedings of the 42nd Annual International Symposium on Computer Architecture, ISCA '15, (New York, NY, USA), p. 285--297, Association for Computing Machinery, 2015.
[7]
L. Peled, U. Weiser, and Y. Etsion, "A neural network prefetcher for arbitrary memory access patterns", ACM Trans. Archit. Code Optim., vol. 16, oct 2019.
[8]
A. Srivastava, A. Lazaris, B. Brooks, R. Kannan, and V. K. Prasanna, "Predicting memory accesses: the road to compact ml-driven prefetcher", in Proceedings of the International Symposium on Memory Systems, MEMSYS 2019, Washington, DC, USA, September 30 - October 03, 2019, pp. 461--470, ACM, 2019.
[9]
M. Hashemi, K. Swersky, J. A. Smith, G. Ayers, H. Litz, J. Chang, C. Kozyrakis, and P. Ranganathan, "Learning memory access patterns", in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 (J. G. Dy and A. Krause, eds.), vol. 80 of Proceedings of Machine Learning Research, pp. 1924--1933, PMLR, 2018.
[10]
Z. Shi, A. Jain, K. Swersky, M. Hashemi, P. Ranganathan, and C. Lin, "A hierarchical neural model of data prefetching", ASPLOS '21, (New York, NY, USA), p. 861--873, Association for Computing Machinery, 2021.
[11]
T. D. Doudali, "A picture is worth a thousand... features! using computer vision alongside machine learning in computer systems", Wild and Crazy Ideas, ASPLOS 2022, pp. 1--2, 2022.
[12]
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, "Swin transformer: Hierarchical vision transformer using shifted windows", in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), (Los Alamitos, CA, USA), pp. 9992--10002, IEEE Computer Society, oct 2021.
[13]
S. Hochreiter and J. Schmidhuber, "Lstm can solve hard long time lag problems", in Proceedings of the 9th International Conference on Neural Information Processing Systems, (Cambridge, MA, USA), p. 473--479, MIT Press, 1996.
[14]
"The 1st ml-based data prefetching competition", 2021.
[15]
R. Bera, K. Kanellopoulos, A. Nori, T. Shahroodi, S. Subramoney, and O. Mutlu, "Pythia: A customizable hardware prefetching framework using online reinforcement learning", in MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO '21, (New York, NY, USA), p. 1121--1137, Association for Computing Machinery, 2021.
[16]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, "An image is worth 16x16 words: Transformers for image recognition at scale", in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.
[17]
B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, "Semantic understanding of scenes through the ade20k dataset", International Journal of Computer Vision, vol. 127, pp. 302--321, 2018.
[18]
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft coco: Common objects in context", in Computer Vision - ECCV 2014 (D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds.), (Cham), pp. 740--755, Springer International Publishing, 2014.
[19]
D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization", in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (Y. Bengio and Y. LeCun, eds.), 2015.
[20]
"Champsim simulator." Available at https://github.com/ChampSim/ChampSim.
[21]
"SunnyCove", May 2018.
[22]
"Spec cpu traces." Available at https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/.
[23]
"Gap traces." Available at https://utexas.app.box.com/s/2k54kp8zvrqdfaa8cdhfquvcxwh7yn85/folder/132804598561.
[24]
"Tensorflow." Available at https://github.com/tensorflow/tensorflow/releases.
[25]
N. S. Kalani and B. Panda, "Instruction criticality based energy-efficient hardware data prefetching", IEEE Computer Architecture Letters, vol. 20, no. 2, pp. 146--149, 2021.
[26]
"Voyager source code." Available at https://github.com/Quangmire/voyager.
[27]
Z. Liu, G. Li, and J. Cheng, "Hardware acceleration of fully quantized bert for efficient natural language processing", in 2021 Design, Automation Test in Europe Conference Exhibition (DATE), pp. 513--516, 2021.
[28]
H.-Y. Wang and T.-S. Chang, "Row-wise accelerator for vision transformer", 2022.
Index Terms
- Drishyam: An Image is Worth a Data Prefetcher
Index terms have been assigned to the content through auto-classification.
Recommendations
Increasing hardware data prefetching performance using the second-level cache
Techniques to reduce or tolerate large memory latencies are critical for achieving high processor performance. Hardware data prefetching is one of the most heavily studied solutions, but it is essentially applied to first-level caches where it can ...
A PAB-based multi-prefetcher mechanism
Aggressive prefetching mechanisms improve performance of some important applications, but substantially increase bus traffic and "pressure" on cache tag arrays. They may even reduce performance of applications that are not memory bounded. We introduce a ...
The migration prefetcher: Anticipating data promotion in dynamic NUCA caches
Special Issue on High-Performance Embedded Architectures and CompilersThe exponential increase in multicore processor (CMP) cache sizes accompanied by growing on-chip wire delays make it difficult to implement traditional caches with a single, uniform access latency. Non-Uniform Cache Architecture (NUCA) designs have been ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2023
355 pages
ISBN:9798350342543
Sponsors
- IFIP WG 10.3: IFIP WG 10.3
- SIGARCH: ACM Special Interest Group on Computer Architecture
- IEEE-CS\DATC: IEEE Computer Society
Publisher
IEEE Press
Publication History
Published: 26 November 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
PACT '23
Sponsor:
- IFIP WG 10.3
- SIGARCH
- IEEE-CS\DATC
PACT '23: International Conference on Parallel Architectures and Compilation Techniques
October 21 - 25, 2023
AE, Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 121 of 471 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 3Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)3
Reflects downloads up to 13 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in