Randomized load balancing with general service time distributions

Randomized load balancing greatly improves the sharing of resources in a number of applications while being simple to implement. One model that has been extensively used to study randomized load balancing schemes is the supermarket model. In this model, jobs arrive according to a rate-nλ Poisson process at a bank of n rate-1 exponential server queues. A notable result, due to Vvedenskaya et.al. (1996), showed that when each arriving job is assigned to the shortest of d ≥ 2 randomly chosen queues, the equilibrium queue sizes decay doubly exponentially in the limit as n to ∞. This is a substantial improvement over the case d=1, where queue sizes decay exponentially.

The method of analysis used in the above paper and in the subsequent literature applies to jobs with exponential service time distributions and does not easily generalize. It is desirable to study load balancing models with more general, especially heavy-tailed, service time distributions since such service times occur widely in practice.

This paper describes a modularized program for treating randomized load balancing problems with general service time distributions and service disciplines. The program relies on an ansatz which asserts that any finite set of queues in a randomized load balancing scheme becomes independent as n to ∞. This allows one to derive queue size distributions and other performance measures of interest. We establish the ansatz when the service discipline is FIFO and the service time distribution has a decreasing hazard rate (this includes heavy-tailed service times). Assuming the ansatz, we also obtain the following results: (i) as n to ∞, the process of job arrivals at any fixed queue tends to a Poisson process whose rate depends on the size of the queue, (ii) when the service discipline at each server is processor sharing or LIFO with preemptive resume, the distribution of the number of jobs is insensitive to the service distribution, and (iii) the tail behavior of the queue-size distribution in terms of the service distribution for the FIFO service discipline.