An experimental procedure for simulation response surface model identification

Reviewer: Kevin Denis Reilly

Response surface methodology is important in simulation, and frequency-based methods continue to play a significant role within it. The authors' years of experience in these methods help demonstrate how good they are. Many topics are broached, including tutorial descriptions, extensions of early proposals of the methodology, testing regimes, a stepwise approach to problems, and applications in queueing and inventory. In order, the paper includes an introduction to frequency domain experiments, an originating-domain problem (mathematical) statement, motivations, the design of experiments, analysis of these experiments, the experimental procedure for model identification, examples, and conclusions. If the reader's desire is to understand the problem-solving approach, observe demonstrations, and assess the merits of the methodology, the paper is at its best, because introductory material and examples constitute the bulk of the paper. A caveat on performance that the authors themselves provide is that “limited empirical” work has been performed to date. If the reader wishes to understand the foundations and analysis details, the authors give a good start, but the paper's references are needed. For example, in a numerical computing course, several students attempted to duplicate some of the calculations, but, though reinforced, they finished with doubts on a few details. An ideal for these students is probably access to actual demonstration code (and, perhaps, more results). Thus, on the surface the paper appears methodologically self-contained, but it could be improved. Moreover, the authors do not always clarify when they are reviewing and when they are proposing (or have proposed), making it difficult to trace the method's history and ascribe the authors' roles. The methodology's limitations and restrictions are dealt with adequately in some particulars, such as the happy circumstance that, though the method is not strictly applicable to transient systems analysis, in examples it seems robust enough to encourage its use in this case. For adaptive systems the method does not seem applicable, and the authors concentrate on areas where it works well, such as classical OR cases. Overall, the paper is quite successful in the face of the many tasks undertaken. It emerges as more than adequate fare for CACM, marred occasionally by uncompromising jargon and a tendency to be very brief on some fundamentals and on words for those wishing to implement the methodology faithfully and directly.