Cited By
View all- Feldkamp N(2021)Data farming output analysis using explainable AIProceedings of the Winter Simulation Conference10.5555/3522802.3522879(1-12)Online publication date: 13-Dec-2021
Allocation of simulation effort for neural network vs. regression metamodels
Article No.: 229, Pages 1 - 12
Abstract
The construction of a neural network simulation metamodel requires the generation of training data; design points (inputs) and the estimate of the corresponding output generated by the simulation model. A common methodology is to focus some simulation effort in obtaining accurate estimates of the expected output values by executing several simulation replications at each point and taking the average as the estimate. However, with a limited amount of simulation effort available and a rather large input space, this approach may not produce the best expected value approximations. An alternate approach is to distribute that same simulation effort over a larger sample of input points, even if it means the resulting estimates of the expected outputs at each point will be less accurate. We will show through several examples that this approach may result in better neural network metamodels; this conclusion differs from other studies involving regression metamodels.
References
[1]
Alam, F. M., K. R. McNaught, and T. J. Ringrose. 2004. "A Comparison of Experimental Designs in the Development of a Neural Network Simulation Metamodel." Simulation Modelling Practice and Theory 12: 559--578.
[2]
Barton, R. R. 1998. "Simulation Metamodels." In Proceedings of the 1998 Winter Simulation Conference, Edited by D. J. Medeiros, E. F. Watson, J. S. Carson, and M. S. Manivannan, 167--174. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[3]
Buzacott, J. A., and J. G. Shanthikumar. 1992. "A General Approach for Coordinating Production in Multiple-cell Manufacturing Systems." Production and Operations Management 1(1): 34--52.
[4]
Fonseca, D. J., D. O. Navaresse, and G. P. Moynihan. 2003. "Simulation Metamodeling through Artificial Neural Networks." Engineering Applications of Artificial Intelligence 16: 177--183.
[5]
Geman, S., E. Bienenstock, and R. Doursat. 1992. "Neural Networks and the Bias/Variance Dilemma." Neural Computation 4(1): 1--58.
[6]
Hagan, M. T. and M. B. Menhaj. 1994. "Training Feedforward Networks with the Marquardt Algorithm." IEEE Transactions on Neural Networks 5(6): 989--993.
[7]
Hurrion, R. D., and S. Birgil. 1999. "A Comparison of Factorial and Random Experimental Design Methods for the Development of Regression and Neural Network Simulation Metamodels." Journal of the Operational Research Society 50: 1018--1033.
[8]
Kilmer, R. A., and A. E. Smith. 1993. "Using Artificial Neural Networks to Approximate a Discrete Event Stochastic Simulation Model." Intelligent Engineering Systems Through Artificial Neural Networks 3: 631--636.
[9]
Kilmer, R. A., A. E. Smith, and L. J. Shuman. 1999. "Computing Confidence Intervals for Stochastic Simulation using Neural Network Metamodels." Computers and Industrial Engineering 36(2): 391--407.
[10]
Kleijnen, J. P. C. 2005. "An Overview of the Design and Analysis of Simulation Experiments for Sensitivity Analysis." European Journal of Operational Research 164: 287--300.
[11]
Kleijnen, J. P. C. 2007. "Regression Models and Experimental Designs: A Tutorial for Simulation Analysts." In Proceedings of the 2007 Winter Simulation Conference, Edited by S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton, 183--194. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[12]
Kleijnen, J. P. C., S. M. Sanchez, T. W. Lucas, and T. M. Cioppa. 2005. "A User's Guide to the Brave New World of Designing Simulation Experiments." INFORMS Journal on Computing 17(3): 263--289.
[13]
Law, A. M., and W. D. Kelton. 2000. Simulation Modeling & Analysis. 3rd ed. New York: McGraw-Hill, Inc.
[14]
MacDonald, C., and E. A. Gunn. 2007. "Analyzing Manufacturing Systems using Neural Network Metamodels." In Proceedings of the 2007 Industrial Engineering Research Conference (IERC), Edited by G. Bayraksan, W. Lin, Y. Son, and R. Wysk.
[15]
MacDonald, C., and E. A. Gunn. 2008. "A Simulation Based System for Analysis and Design of Production Control Systems." In Proceedings of the 2008 Winter Simulation Conference, Edited by S. J. Mason, R. Hill, L. Moench, and O. Rose, 1882--1890. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[16]
Saito, K., and R. Nakano. 2000. "Second-order Learning Algorithm with Squared Penalty Term." Neural Computation 12: 709--729.
[17]
Santos, P. R., and M. I. Santos. 2008. "Sequential Experimental Designs for Nonlinear Regression Metamodels in Simulation." Simulation Modelling Practice and Theory 16: 1365--1378.
[18]
Santos, P. R., and M. I. Santos. 2009a. "Construction of Stochastic Simulation Metamodels based on Un-replicated Smoothed Data." Simulation 85(6): 387--396.
[19]
Santos, P. R., and M. I. Santos. 2009b. "Design Experiments for the Construction of Simulation Metamodels." In Proceedings of the 23rd European Conference on Modelling and Simulation, Edited by J. Otamendi, A. Bargiela, J. L. Montes, and L. M. D. Pedrera, 338--344. Nottingham, UK: European Council for Modelling and Simulation.
[20]
Sjöberg, J., and L. Ljung. 1995. "Overtraining, Regularization and Searching for a Minimum, with Application to Neural Networks." International Journal of Control 62(6): 1391--1407.
[21]
Smith, M. 1993. Neural Networks for Statistical Modeling. New York: Van Nostrand Reinhold.
Index Terms
- Allocation of simulation effort for neural network vs. regression metamodels
Recommendations
Applying fuzzy neural network to estimate software development effort
The ability to accurately and consistently estimate software development efforts is required by the project managers in planning and conducting software development activities. Since software effort drivers are vague and uncertain, software effort ...
Comparison of regression model, feed-forward neural network and radial basis neural network for software development effort estimation
Accurate estimation of software development parameters such as effort, cost, and schedule is very important for effectively managing software development projects. Several software development effort estimation models have been developed in the last few ...
Neural network models for software development effort estimation: a comparative study
Software development effort estimation (SDEE) is one of the main tasks in software project management. It is crucial for a project manager to efficiently predict the effort or cost of a software project in a bidding process, since overestimation will ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sponsors
- IIE: Institute of Industrial Engineers
- INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
- SCS: Shanghai Computer Society
- SIGSIM: ACM Special Interest Group on Simulation and Modeling
Publisher
Winter Simulation Conference
Publication History
Published: 09 December 2012
Check for updates
Qualifiers
- Research-article
Conference
WSC '12
Sponsor:
- IIE
- INFORMS-SIM
- SCS
- SIGSIM
Acceptance Rates
WSC '12 Paper Acceptance Rate 189 of 384 submissions, 49%;
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 36Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Dec 2024
Other Metrics
Citations
Cited By
View all- Feldkamp N(2021)Data farming output analysis using explainable AIProceedings of the Winter Simulation Conference10.5555/3522802.3522879(1-12)Online publication date: 13-Dec-2021
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in