Rapid detection of significant spatial clusters

Given an N x N grid of squares, where each square has a count c_ij and an underlying population p_ij, our goal is to find the rectangular region with the highest density, and to calculate its significance by randomization. An arbitrary density function D, dependent on a region's total count C and total population P, can be used. For example, if each count represents the number of disease cases occurring in that square, we can use Kulldorff's spatial scan statistic D_K to find the most significant spatial disease cluster. A naive approach to finding the maximum density region requires O(N⁴) time, and is generally computationally infeasible. We present a multiresolution algorithm which partitions the grid into overlapping regions using a novel overlap-kd tree data structure, bounds the maximum score of subregions contained in each region, and prunes regions which cannot contain the maximum density region. For sufficiently dense regions, this method finds the maximum density region in O((N log N)²) time, in practice resulting in significant (20-2000x) speedups on both real and simulated datasets.