Abstract
Over recent years, evolutionary computation research has begun to emphasize the issue of generalization. Instead of evolving solutions that are optimized for a particular problem instance, the goal is to evolve solutions that can generalize to various different scenarios. This paper compares objective-based search and novelty search on a set of generalization oriented experiments for a navigation task using grammatical evolution (GE). In particular, this paper studies the impact that the training set has on the generalization of evolved solutions, considering: (1) the training set size; (2) the manner in which the training set is chosen (random or manual); and (3) if the training set is fixed throughout the run or dynamically changed every generation. Experimental results suggest that novelty search outperforms objective-based search in terms of evolving navigation behaviors that are able to cope with different initial conditions. The traditional objective-based search requires larger training sets and its performance degrades when the training set is not fixed. On the other hand, novelty search seems to be robust to different training sets, finding general solutions in almost all of the studied conditions with almost perfect generalization in many scenarios.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Banzhaf W, Francone FD, Nordin P (1996) The effect of extensive use of the mutation operator on generalization in genetic programming using sparse data sets. In: In parallel problem solving from nature IV, proceedings of the international conference on evolutionary computation, edited by, Springer, Berlin, pp 300–309
Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, New York
Burke EK, Gustafson S, Kendall G, Krasnogor N (2004) Is increased diversity in genetic programming beneficial? an analysis of lineage selection. Ph.D. thesis, University of Nottingham, UK
Castelli M, Manzoni L, Silva S, Vanneschi L (2010) A comparison of the generalization ability of different genetic programming frameworks. In: Evolutionary Computation (CEC), 2010 IEEE Congress on, pp 1–8
Castelli M, Manzoni L, Silva S, Vanneschi L (2011) A quantitative study of learning and generalization in genetic programming. In: Silva S, Foster JA, Nicolau M, Machado P, Giacobini M (eds) EuroGP., Lecture notes in computer scienceSpringer, Berlin, pp 25–36
Dempsey I, O’Neill M, Brabazon A (2009) Foundations in grammatical evolution for dynamic environments, vol 194., Studies in computational intelligenceSpringer, Berlin
Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18
Doucette J, Heywood MI (2010) Novelty-Based Fitness: An Evaluation under the Santa Fe Trail. In: Esparcia-Alcázar AI, Ekárt A, Silva S, Dignum S, Uyar AS (eds) Genetic Programming: 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7-9, 2010. Proceedings. Springer, Berlin, Heidelberg, pp 50–61
Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley, New Jersey
Francone FD, Nordin P, Banzhaf W (1996) Benchmarking the generalization capabilities of a compiling genetic programming system using sparse data sets. Proceedings of the 1st annual conference on genetic programming. MIT Press, Cambridge, pp 72–80
Gathercole C, Ross P (1994) Dynamic training subset selection for supervised learning in genetic programming. In: Proceedings of the international conference on evolutionary computation. The third conference on parallel problem solving From nature: parallel problem solving from nature, PPSN III, Springer, London, pp 312–321
Georgiou L (2012) Constituent grammatical evolution. Ph.D. thesis, School of computer science, Bangor University, Bangor
Georgiou L, Teahan WJ (2006) jge–a java implementation of grammatical evolution. 10th WSEAS international conference on systems. Greece, Athens, pp 534–869
Georgiou L, Teahan WJ (2010) Grammatical evolution and the santa fe trail problem. International conference on evolutionary computation (ICEC). SciTePress, Valencia, Spain, pp 10–19
Goldberg DE (1987) Simple genetic algorithms and the minimal, deceptive problem. In: Davis L (ed) Genetic algorithms and simulated annealing., Research notes in artificial intelligencePitman, London, pp 74–88
Gomes J, Mariano P, Christensen AL (2015) Devising effective novelty search algorithms: a comprehensive empirical study. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation, GECCO’15, ACM, New York (2015), pp 943–950
Gomes J, Urbano P, Christensen A (2013) Evolution of swarm robotics systems with novelty search. Swarm Intell 7(2–3):115–144
Gonçalves I, Silva S (2011) Experiments on controlling overfitting in genetic programming. In: 15th Portuguese conference on artificial intelligence (EPIA 2011)
Gonçalves I, Silva S (2011) Experiments on controlling overfitting in genetic programming. In: 15th Portuguese conference on artificial intelligence. EPIA 2011
Gonçalves I, Silva S (2013) Balancing learning and overfitting in genetic programming with interleaved sampling of training data. In: Krawiec K, Moraglio A, Hu T, Etaner-Uyar A, Hu B (eds) Genetic programming. Lecture notes in computer science, vol 7831. Springer, Berlin, pp 73–84
Gonçalves I, Silva S, Fonseca C (2015) On the generalization ability of geometric semantic genetic programming. In: 18th European conference on genetic programming (EuroGP 2015). n/a
Gonçalves I, Silva S, Melo J, Carreiras JAMB (2012) Random sampling technique for overfitting control in genetic programming. In: Moraglio A, Silva S, Krawiec K, Machado P, Cotta C (eds) Genetic programming. Lecture notes in computer science, vol 7244. Springer, Berlin, pp 218–229
Kaelbling LP, Littman ML, Moore AP (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285
Koza JR (1992) Genetic programming–on the programming of computers by means of natural selection. MIT Press, Cambridge, MA, USA
Kushchu I (2002) An evaluation of evolutionary generalisation in genetic programming. Artif Intell Rev 18(1):3–14
Kushchu I (2002) Genetic programming and evolutionary generalization. IEEE Trans Evol Comput 6(5):431–442
Langdon W, Poli R (2001) Foundations of genetic programming. Springer, Berlin
Lehman J, Stanley K (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Bullock S, Noble J, Watson R, Bedau MA (eds) Artificial life XI: proceedings of the eleventh international conference on the simulation and synthesis of living systems. MIT Press, Cambridge, pp 329–336
Lehman J, Stanley KO (2010) Efficiently evolving programs through the search for novelty. In: Pelikan M, Branke J (eds) GECCO. ACM, New York, pp 837–844
Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223
Mahler S, Robilliard D, Fonlupt C (2005) Tarpeian bloat control and generalization accuracy. In: Keijzer M, Tettamanzi A, Collet P, van Hemert JI, Tomassini M (eds) Proceedings of the 8th European conference on genetic programming, vol 3447., Lecture notes in computer scienceSpringer, Lausanne, pp 203–214
Martinez Y, Trujillo L, Naredo E, Legrand P (2014) A comparison of fitness-case sampling methods for symbolic regression with genetic programming. In: EVOLVE 2014, Beijing, China
Martnez Y, Naredo E, Trujillo L, Lpez EG (2013) Searching for novel regression functions. In: IEEE congress on evolutionary computation, pp 16–23
Mouret JB, Doncieux S (2012) Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol Comput 20(1):91–133
Naik TR, Dabhi VK (2013) Improving generalization ability of genetic programming: comparative study. CoRR abs/1304.3779
Naredo E, Trujillo L (2013) Searching for novel clustering programs. Proceedings of the 15th annual conference on genetic and evolutionary computation. GECCO’13, ACM, New York, pp 1093–1100
Nelson AL, Barlow GJ, Doitsidis L (2009) Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57(4):345–370
Nicoară ES (2009) Mechanisms to avoid the premature convergence of genetic algorithms. Pet—Gas Univ Ploiesti Bull, Math-Inform-Phys Ser 61(1):87–96
Nolfi S, Floreano D (2000) Evolutionary robotics: the biology, intelligence, and technology. MIT Press, Cambridge
O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358
Robilliard D, Mahler S, Verhaghe D, Fonlupt C (2006) Santa fe trail hazards. In: Talbi EG, Liardet P, Collet P, Lutton E, Schoenauer M (eds) 7th international conference on artificial evolution EA 2005, vol 3871., Lecture notes in computer scienceSpringer, Lille, pp 1–12
Rosca J (1996) Generality versus size in genetic programming. In: Koza JR, Goldberg DE, Fogel DB, Riolo RL (eds) Genetic programming 1996: proceedings of the first annual conference. MIT Press, Stanford University, CA, pp 381–387
Shorten D, Nitschke G (2015) Evolving generalised maze solvers. In: Mora AM, Squillero G (eds) Applications of evolutionary computation. Lecture notes in computer science, vol 9028. Springer, Berlin, pp 783–794
Spector L (2012) Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. Proceedings of the 14th annual conference companion on genetic and evolutionary computation. GECCO’12, ACM, New York, pp 401–408
Trujillo L, Olague G, Lutton E, de Vega FF (2008) Behavior-based speciation for evolutionary robotics. In: GECCO, pp 297–298
Trujillo L, Olague G, Lutton E, de Vega FF, Dozal L, Clemente E (2011) Speciation in behavioral space for evolutionary robotics. J Intell Robot Syst 64(3–4):323–351
Trujillo L, Silva S, Legrand P, Vanneschi L (2011) An empirical study of functional complexity as an indicator of overfitting in genetic programming. In: Silva S, Foster JA, Nicolau M, Machado P, Giacobini M (eds) EuroGP. Lecture notes in computer science, vol 6621. Springer, Berlin, pp 262–273
Urbano P, Loukas G (2013) Improving grammatical evolution in santa fe trail using novelty search. In: Advances in artificial life, ECAL, pp 917–924
Urbano P, Naredo E, Trujillo L (2014) Generalization in maze navigation using grammatical evolution and novelty search. In: Dediu AH, Lozano M, Martn-Vide C (eds) Theory and practice of natural computing. Lecture notes in computer science, vol 8890. Springer, Berlin, pp 35–46
Uy NQ, Hien NT, Hoai NX, O’Neill M (2010) Improving the generalisation ability of genetic programming with semantic similarity based crossover. In: Proceedings of the 13th European conference on genetic programming. EuroGP’10, Springer, Berlin, pp 184–195
Vanneschi L, Castelli M, Silva S (2010) Measuring bloat, overfitting and functional complexity in genetic programming. Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO’10. ACM, New York, pp 877–884
Velez R, Clune J (2014) Novelty search creates robots with general skills for exploration. Proceedings of the 2014 conference on genetic and evolutionary computation, GECCO’14. ACM, New York, pp 737–744
Wilensky U (1999) Netlogo, Evanston, IL: Center for connected learning and computer-based modeling. http://ccl.northwestern.edu/netlogo. Accessed 27 Nov 2015
Acknowledgments
The authors acknowledge the following projects. First author was supported by CONACYT (México) scholarship No. 232288. Funding for this work was provided by FCT project EXPL/EEI-SII/1861/2013, and by CONACYT Basic Science Research Project No. 178323, DGEST (Mexico) Research Project 5414.14-P, and FP7-PEOPLE-2013-IRSES project ACOBSEC financed by the European Commission with contract No. 612689.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Naredo, E., Urbano, P. & Trujillo, L. The training set and generalization in grammatical evolution for autonomous agent navigation. Soft Comput 21, 4399–4416 (2017). https://doi.org/10.1007/s00500-016-2072-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2072-7