Analyzing transformation-based simulation metamodels

We present a technique for analyzing a simulation metamodel that has been constructed using a variance-stabilizing transformation. To compute a valid confidence interval for the expected value of the original simulation response at a selected factor-level combination (design point), we first compute the corresponding confidence interval for the transformed response at that factor-level combination and then untransform the endpoints of the resulting confidence interval. Taking the midpoint of the untransformed confidence interval as our point estimator of the expected simulation response at the selected factor-level combination and approximating the variance of this point estimator via the delta method, we formulate an approximate two-sample Student t-test for validating our metamodel-based estimator versus the results of making independent runs of the simulation at the selected factor-level combination. We illustrate this technique in a case study involving the design of a manufacturing cell, and we compare our results with those of a more conventional approach to analyzing transformed-based simulation metamodels. A Monte Carlo performance evaluation shows that significantly better confidence-interval coverage is maintained with the proposed procedure over a wide range of values for the residual variance of the transformed metamodel.