More Web Proxy on the site http://driver.im/

research-article

Tutorial: simulation metamodeling

Author:

Russell R. BartonAuthors Info & Claims

WSC '15: Proceedings of the 2015 Winter Simulation Conference

Pages 1765 - 1779

Published: 06 December 2015 Publication History

Abstract

The concept of a metamodel has been an important tool for simulation analysis for forty years. These models of simulation models have the advantage of faster execution, and they can (sometimes) provide insight on the nature of the simulation response as a function of design and input distribution parameters. This introductory tutorial will describe metamodeling uses and associated processes, survey commonly used metamodel types and associated experiment designs, and give a brief description of some recent developments and how they may affect future "mainstream" simulation metamodeling.

References

[1]

Banks, J., J. S. Carson, B. L. Nelson, and D. M. Nicol. 2010. Discrete-Event System Simulation. 5th ed. Upper Saddle River, New Jersey: Prentice-Hall, Inc.

[2]

Barton, R. R. 1992. "Metamodels for Simulation Input-Output Relations." In Proceedings of the 1992 Winter Simulation Conference, edited by J. J. Swain, D. Goldsman, R. C. Crain and J. R. Wilson, 289-299. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[3]

Barton, R. R. 1998. "Simulation Metamodels." In Proceedings of the 1998 Winter Simulation Conference, edited by D. J. Medeiros, E. F. Watson, J. S. Carson and M. S. Manivannan, 167-174. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[4]

Barton, R. R. 2009. "Simulation Optimization using Metamodels." In Proceedings of the 2009 Winter Simulation Conference, edited by M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin and R. G. Ingalls, 230-238. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[5]

Barton, R. R. 2010. "Simulation Experiment Design." In Proceedings of the 2010 Winter Simulation Conference, edited by B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, 75-86. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[6]

Barton, R. R. and M. Meckesheimer. 2006. "Metamodel-Based Simulation Optimization." Chapter 18 in Handbook in Operations Research and Management Science Vol. 13: Simulation, edited by S. G. Henderson and B. L. Nelson, 535-574. Amsterdam: North Holland.

[7]

Blanning, R. W. 1974. "The Sources and Uses of Sensitivity Information." Interfaces 4:32--38.

Digital Library

[8]

Blanning, R. W. 1975. "The Construction and Implementation of Metamodels." Simulation 24: 177--184.

[9]

Blanning, R. W. 1975. "Response to Michel, Kleijnen and Permut." Interfaces 5:24--25.

Digital Library

[10]

Burdick, D. S. and T. H. Naylor. 1966. "Design of Computer Simulation Experiments for Industrial Systems." Communications of the ACM 9:329--339.

Digital Library

[11]

Chen, V. C. P., K.-L. Tsui, R. R. Barton and M. Meckesheimer. 2006. "A Review on Design, Modeling and Applications of Computer Experiments." IIE Transactions 38, 273--291.

[12]

Chen X., B. E. Ankenman and B. L. Nelson. 2010. "Common Random Numbers and Stochastic Kriging. In Proceedings of the 2010 Winter Simulation Conference, edited by B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, 947-956. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[13]

Chen, X. and K. Kim. 2013. "Building Metamodels for Quantile-based Measures including Sectioning." In Proceedings of the 2013 Winter Simulation Conference, edited by R. Pasupathy, S.-H. Kim, A. Tolk, R. Hill, and M. E. Kuhl. 521--532. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[14]

Cheng, R. C. H. and C. S. M. Currie. 2004. "Optimization by Simulation Metamodeling Methods." In Proceedings of the 2004 Winter Simulation Conference, edited by R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, 485-490. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[15]

Dellino, G., J. P. C. Kleijnen and C. Meloni. 2009. "Robust Simulation-Optimization using Metamodels." In Proceedings of the 2009 Winter Simulation Conference, edited by M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin and R. G. Ingalls, 540-550. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[16]

Friedman, L. w. 1996. The Simulation Metamodel. Norwell, MA: Kluwer.

Digital Library

[17]

Hufschmidt, M. M. and M. B. Fiering. 1966. Simulation Techniques for Design of Water Resource Systems. Cambridge, MA: Harvard University Press.

[18]

Hunter, J. S. and T. H. Naylor. 1970. "Experimental Designs for Computer Experiments." Management Science 16:422--434.

Digital Library

[19]

Kleijnen, J. P. C. 1975. "A Comment on Blanning's Metamodel for Sensitivity Analysis: the Regression Metamodel in Simulation." Interfaces, 5:21--23.

Digital Library

[20]

Kleijnen, J. P. C. 2008. "Design of Experiments: Overview." In Proceedings of the 2008 Winter Simulation Conference, edited by S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler, 479-488. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[21]

Kleijnen, J. P. C. 2015. Design and Analysis of Simulation Experiments. 2nd ed. New York; London: Springer.

[22]

Law, A. W. 2014. "A Tutorial on Design of Experiments for Simulation Modeling." In Proceedings of the 2014 Winter Simulation Conference, edited by A. Tolk, S. Y. Diallo, I. O. Ryzhov, L. Yilmaz, S. Buckley, and J. A. Miller, 66-80. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Digital Library

[23]

Law, A. W. 2015. Simulation Modeling and Analysis. 5th ed. New York: McGraw-Hill.

[24]

Li, R. and D. K. J. Lin. 2003. "Analysis Methods for Supersaturated Design: Some Comparisons." Journal of Data Science 1:249--260.

[25]

Maass, A., M. M. Hufschmidt, R. Dorfman, H. A. Thomas, Jr., S. A. Marglin and G. M. Fair. 1962. Design of Water Resource Systems. Cambridge, MA: Harvard University Press.

[26]

Meckesheimer, M., R. R. Barton, T. Simpson, F. Limayem and B. Yannou. 2001. "Metamodeling of Combined Discrete/Continuous Responses." AIAA Journal 39:1950--1959.

[27]

Meckesheimer, M., A. J. Booker, R. R. Barton and T. W. Simpson. 2002. "Computationally Inexpensive Metamodel Assessment Strategies." AIAA Journal 40:2053--2060.

[28]

Minitab. 2015. "Minitab 17." Accessed May 7^th, 2015. http://www.minitab.com/en-us/.

[29]

Montgomery, D. C. 2012. Design and Analysis of Experiments. 7th ed. New York: Wiley.

[30]

Qu, H. and M. C. Fu. 2012. "On Direct Gradient Enhanced Metamodels." In Proceedings of the 2012 Winter Simulation Conference, edited by C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, 478-489. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[31]

Sanchez, S. M. and P. J. Sanchez. 2005. "Very Large Fractional Factorial and Central Composite Designs." ACM Transactions on Modeling and Computer Simulation 15, 362--377.

Digital Library

[32]

Sanchez, S. M. and H. Wan. 2012. "Work Smarter, Not Harder: a Tutorial on Designing and Conducting Simulation Experiments." In Proceedings of the 2012 Winter Simulation Conference, edited by C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, 1929-1943. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[33]

Schruben, L. W. 1979. "Designing Correlation Induction Strategies for Simulation Experiments." In Current Issues in Computer Simulation, edited by N. R. Adam and A. Dogramaci, 235--256. New York: Academic Press.

[34]

Staum. J. 2009. "Better Simulation Metamodeling: The Why, What, and How of Stochastic Kriging." In Proceedings of the 2009 Winter Simulation Conference, edited by M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin and R. G. Ingalls, 119-133. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[35]

Tew, J. D. and J. R. Wilson. 1987. "Metamodel Estimation using Integrated Correlation Methods." In Proceedings of the 1987 Winter Simulation Conference, edited by A. Thesen, H. Grant and W. D. Kelton, 409-418. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[36]

Tu, J. 2003. "Cross-Validated Multivariate Metamodeling Methods for Physics-based Computer Simulations." In Proceedings of the IMAC-XXI, A Conference on Structural Dynamics, The Hyatt Orlando, Kissimee, FL February 3-6, 2003.

[37]

Walsh, J. E. 1963. "Use of Linearized Nonlinear Regression for Simulations Involving Monte Carlo." Operations Research 11:228--235.

Digital Library

[38]

Xie, W., B. L. Nelson and R. R. Barton. 2014. "Statistical Uncertainty Analysis for Stochastic Simulation with Dependent Input Models." In Proceedings of the 2014 Winter Simulation Conference, edited by A. Tolk, S. Y. Diallo, I. O. Ryzhov, L. Yilmaz, S. Buckley, and J. A. Miller, 674-685. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Digital Library

Cited By

Sanchez SSanchez PWan H(2021)Work smarter, not harderProceedings of the Winter Simulation Conference10.5555/3522802.3522885(1-15)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522885
Jackson IVelázquez-Martínez J(2021)Automl approach to classification of candidate solutions for simulation models of logistic systemsProceedings of the Winter Simulation Conference10.5555/3522802.3522852(1-12)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522852
Zhang HHe JZhan DZheng Z(2021)Neural network-assisted simulation optimization with covariatesProceedings of the Winter Simulation Conference10.5555/3522802.3522817(1-12)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522817
Show More Cited By

Tutorial: simulation metamodeling
1. Computing methodologies
  1. Modeling and simulation

Recommendations

Tutorial: information and process modeling for simulation
WSC '14: Proceedings of the 2014 Winter Simulation Conference

Like a system model in Information Systems and Software Engineering (IS/SE), a system model in simulation engineering mainly consists of an information model and a process model. In the fields of IS/SE there are widely used standards such as the Class ...
Tutorial: an overview of UML 2
ICSE '06: Proceedings of the 28th international conference on Software engineering

This half-day tutorial covers the salient features of the first major revision of the Unified Modeling Language - UML 2. This short note summarizes the major topics covered by the tutorial.
Tutorial: metamodeling for simulation
WSC '20: Proceedings of the Winter Simulation Conference

Metamodels are simpler computer models that are designed to mimic the input-output behavior of discrete-event or other complex simulation models. They are models of models, thus the name (provided by Jack P. C. Kleijnen). This introductory tutorial will ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '15: Proceedings of the 2015 Winter Simulation Conference

December 2015

4051 pages

ISBN:9781467397414

Sponsors

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

IEEE Press

Publication History

Published: 06 December 2015

Check for updates

Qualifiers

Research-article

Conference

WSC '15

Sponsor:

SIGSIM

WSC '15: Winter Simulation Conference

December 6 - 9, 2015

California, Huntington Beach

Acceptance Rates

WSC '15 Paper Acceptance Rate 202 of 296 submissions, 68%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
69
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sanchez SSanchez PWan H(2021)Work smarter, not harderProceedings of the Winter Simulation Conference10.5555/3522802.3522885(1-15)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522885
Jackson IVelázquez-Martínez J(2021)Automl approach to classification of candidate solutions for simulation models of logistic systemsProceedings of the Winter Simulation Conference10.5555/3522802.3522852(1-12)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522852
Zhang HHe JZhan DZheng Z(2021)Neural network-assisted simulation optimization with covariatesProceedings of the Winter Simulation Conference10.5555/3522802.3522817(1-12)Online publication date: 13-Dec-2021
https://dl.acm.org/doi/10.5555/3522802.3522817
Sanchez SSanchez PWan H(2020)Work smarter, not harderProceedings of the Winter Simulation Conference10.5555/3466184.3466311(1128-1142)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466311
Sanchez SSánchez PWan HJohansson BJain S(2018)Work smarter, not harderProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320551(237-251)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320551
MacCalman ASanchez SMcDonald MGoerger SKarl AHuschka TChick S(2016)Tradespace analysis for multiple performance measuresProceedings of the 2016 Winter Simulation Conference10.5555/3042094.3042475(3063-3074)Online publication date: 11-Dec-2016
https://dl.acm.org/doi/10.5555/3042094.3042475
Sanchez SWan H(2015)Work smarter, not harderProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2888820(1795-1809)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2888820

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents