Challenges with verification, repeatability, and meaningful comparison in genetic programming: gibson's magic
Authors: 
Jason M. Daida, Derrick S. Ampy, Michael Ratanasavetavadhana, Hsiaolei Li, Omar A. ChaudhriAuthors Info & Claims
Pages 1851 - 1858
Abstract
This paper examines some of the reporting and research practices concerning empirical work in genetic programming. We describe several common loopholes and offer three case studies—two in data modeling and one in robotics—that illustrate each. We show that by exploiting these loopholes, one can achieve performance gains of up two orders of magnitude without any substantiative changes to G P. We subsequently offer several recommendations.
References
[1]
Compagner, A. (1995). "Operational Conditions for Random-Number Generation." Physical Review E 52: 5634-5645.
[2]
Daida, J. M., R. B. Bertram, J. A. Polito 2 and S. A. Stanhope (1999a). "Analysis of Single-Node (Building) Blocks in Genetic Programming." In L. Spector, W. B. Langdon, U.-M. O'Reilly and P. J. Angeline (Eds.), Advances in Genetic Programming 3. Cambridge: The MIT Press. (In press.)
[3]
Daida, J. M., J. A. Polito, 2, S. A. Stanhope, R. R. Bertram, J. Khoo and S. Chaudhary (1999b). "What Makes a Problem GP-Hard? Analysis of a Tunably Difficulty Problem in Genetic Programming." In J. K. Koza (Eds.), Proceedings of GECCO'99.
[4]
Daida, J. M., S. J. Ross, J. J. McClain, D. S. Ampy and M. Holczer (1997). "Challenges with Verification, Repeatability, and Meaningful Comparisons in Genetic Programming." In J. R. Koza, K. Deb, M. Dorigo, et al (Eds.), Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University. San Francisco: Morgan Kaufmann Publishers. pp. 64-69.
[5]
Evett, M. and T. Fernandez (1998). "Numeric Mutation Improves the Discovery of Numeric Constants in Genetic Programming." In J. R. Koza, W. Banzhaf, K. Chellapilla, et al (Eds.), Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison. San Francisco: Morgan Kaufmann Publishers. pp. 66-71.
[6]
Fitzkee, D. (1945). Magic by Misdirection. San Rafael.
[7]
Gathercole, C. and P. Ross (1996). "An Adverse Interaction Between Crossover and Restricted Tree Depth in Genetic Programming." In J. R. Koza, D. E. Goldberg, D. B. Fogel and R. L. Riolo (Eds.), Genetic Programming 1996: Proceedings of the First Annual Conference: July 28-31, 1996, Stanford University. Cambridge: The MIT Press. pp. 291-296.
[8]
Gibson, J. J. (1982). "Ecological Physics, Magic, and Reality." In E. Reed and R. Jones (Eds.), Reasons for Realism: Selected Essays of James J. Gibson. Hillsdale: L. Erlbaum.
[9]
Haynes, T. (1998). "Perturbing the Representation, Decoding, and Evaluation of Chromosomes." In J. R. Koza, W. Banzhaf, K. Chellapillaet al (Eds.), Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison. San Francisco: Morgan Kaufmann Publishers. pp. 122-127.
[10]
Hellekalek, P. (1997). "A Note on Pseudorandom Number Generators." Simulation Practice and Theory 5(6): 6-8.
[11]
Hellekalek, P. (1998). "Good Random Number Generators Are (Not So) Easy to Find." Mathematics and Computers in Simulation 46(5-6): 487-507.
[12]
Koza, J. R. (1992a). Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge: The MIT Press.
[13]
Koza, J. R. (1992b). "Evolution of Subsumption Using Genetic Programming." In F. J. Varela and P. Bourgine (Eds.), Proceedings of the First European Conference on Artificial Life: Towards a Practice of Autonomous Systems. Cambridge: The MIT Press. pp. 110-119.
[14]
Koza, J. R., W. Banzhaf, et al., Eds. (1998). Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison. San Francisco: Morgan Kaufmann Publishers.
[15]
Koza, J. R., K. Deb, et al., Eds. (1997). Genetic Programming 1997: Proceedings of the Second Annual Conference, July 13-16, 1997, Stanford University. San Francisco: Morgan Kaufmann Publishers.
[16]
Koza, J. R., D. E. Goldberg, D. B. Fogel and R. L. Riolo (1996). Genetic Programming 1996: Proceedings of the First Annual Conference: July 28-31, 1996, Stanford University. Cambridge: The MIT Press.
[17]
L'Ecuyer, P. (1997). "Random Number Generation." In J. Banks (Eds.), Handbook on Simulation. New York: Wiley.
[18]
Mataric, M. J. (1990). A Distributed Model for Mobile Robot Environment-Learning and Navigation. Cambridge, Massachusetts Institute of Technology Artificial Intelligence Laboratory.
[19]
Matsumoto, M. and T. Nishimura (1997). mt19937.c. Keio, Department of Mathematics, Keio University. http://www.math.keio.ac.jp/~matumoto/emt.html.
[20]
Matsumoto, M. and T. Nishimura (1998). "Mersenne Twister: A 623-Di-mensionally Equidistributed Uniform Pseudorandom Number Generator." ACM Transactions on Modeling and Computer Simulation 8(1): 3-30.
[21]
Meysenburg, M. and J. A. Foster (1999). "Random Generator Quality and GP Performance." In J. K. Koza (Eds.), Proceedings of GECCO'99.
[22]
Press, W. H., S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery (1992). Numerical Recipies in C: The Art of Scientific Computing: Second Edition. Cambridge: Cambridge University Press. pp. 274-304.
[23]
Raidl, G. R. (1998). "A Hybrid GP Approach for Numerically Robust Symbolic Regression." In J. R. Koza, W. Banzhaf, K. Chellapilla, et al. (Eds.), Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison. San Francisco: Morgan Kaufmann Publishers. pp. 323-28.
[24]
Ross, S. J., J. M. Daida, C. M. Doan, T. F. Bersano-Begey and J. J. McClain (1996). "Variations in Evolution of Subsumption Architectures Using Genetic Programming: The Wall Following Robot Revisited." In J. R. Koza, D. E. Goldberg, D. B. Fogel and R. L. Riolo (Eds.), Genetic Programming 1996: Proceedings of the First Annual Conference: July 28-31, 1996, Stanford University. Cambridge: The MIT Press. pp. 21-29.
[25]
Soule, T., J. A. Foster, and J. Dickinson (1996). "Using Genetic Programming to Approximate Maximum Cliques." In J. R. Koza, D. E. Goldberg, D. B. Fogel and R. L. Riolo (Eds.), Genetic Programming 1996: Proceedings of the First Annual Conference: July 28-31, 1996, Stanford University. Cambridge: The MIT Press. pp. 400-405.
[26]
Tufte, E. R. and J. I. Swiss (1997). "Explaining Magic: Pictorial Instructions and Disinformation Design." In E. R. Tufte (Eds.), Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire: Graphics Press. pp. 55-71.
[27]
Wolpert, D. H. and W. G. Macready (1997). "No Free Lunch Theorems for Optimization." IEEE Transactions on Evolutionary Computation 1(1): 67-82.
[28]
Zelkowitz, M. V. and D. R. Wallace (1998). "Experimental Models for Validating Technology." IEEE Computer 31(5): 23-31.
Recommendations
A comparison of Cartesian genetic programming and linear genetic programming
EuroGP'08: Proceedings of the 11th European conference on Genetic programmingTwo prominent genetic programming approaches are the graph-based Cartesian Genetic Programming (CGP) and Linear Genetic Programming (LGP). Recently, a formal algorithm for constructing a directed acyclic graph (DAG) from a classical LGP instruction ...
A Comparison of three evolutionary strategies for multiobjective genetic programming
We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-...
Genetic programming: a tutorial introduction
GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computationGenetic programming emerged in the early 1990's as one of the most exciting new evolutionary algorithm paradigms. It has rapidly grown into a thriving area of research and application. While sharing the evolutionary inspired algorithm principles of a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 1999
1867 pages
ISBN:1558606114
Publisher
Morgan Kaufmann Publishers Inc.
San Francisco, CA, United States
Publication History
Published: 13 July 1999
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 78Total Downloads
- Downloads (Last 12 months)29
- Downloads (Last 6 weeks)4
Reflects downloads up to 18 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in