Layered Bottlenecks and Their Mitigation

Bottlenecks are a simple and well-understood phenomenon in service systems and queueing models. However in systems with layered resources bottlenecks are more complicated, because of simultaneous resource possession. Thus, the holding time of a higher-layer resource, such as a process thread, may include a small execution demand, but a large time to use other resources at a lower layer (such as a disk). A single saturation point may in fact saturate many other resources by push-back, making diagnosis of the problem difficult. This paper gives a new corrected definition of a layered bottleneck, and develops a framework for systematic detection of the source of a bottleneck, for applying improvements and for estimating their effectiveness. Many of the techniques are specific to layered bottlenecks.