Investigating optimal local memory performance

Recent work has demonstrated that, cache space is often poorly utilized. However, no previous work has yet demonstrated upper bounds on what a cache or local memory could achieve when exploiting both spatial and temporal locality. Belady's MIN algorithm does yield an upper bound, but exploits only temporal locality. In this article, we present an optimal replacement algorithm for local memory that exploits temporal locality and spatial locality simultaneously. This algorithm is an extension of Belady's algorithm. We prove the optimality of this new algorithm with respect to minimizing misses, and we show experimentally that the algorithm produces nearly minimum memory traffic on the SPEC95 benchmarks. Like Belady's algorithm, our algorithm requires the entire program trace. It selects replacement victims and the number of words it fetches at once based on future accesses. Many different spatial locality strategies can be implemented with this algorithm. With an optimal strategy, the algorithm yields an upper bound that enables us to evaluate alternative implementations to today's caches. We further demonstrate the utility of this algorithm as an analysis tool by evaluating several intermediate strategies between cache and optimal to highlight the limitations of the cache line paradigm using the SPEC95 benchmarks.