Abstract
Deep neural networks (DNNs), particularly convolutional neural networks (CNNs), have garnered significant attention in recent years for addressing a wide range of challenges in image processing and computer vision. Neural architecture search (NAS) has emerged as a crucial field aiming to automate the design and configuration of CNN models. In this paper, we propose a novel strategy to speed up the performance estimation of neural architectures by gradually increasing the size of the training set used for evaluation as the search progresses. We evaluate this approach using the CGP-NASV2 model, a multi-objective NAS method, on the CIFAR-100 dataset. Experimental results demonstrate a notable acceleration in the search process, achieving a speedup of 4.6 times compared to the baseline. Despite using limited data in the early stages, our proposed method effectively guides the search towards competitive architectures. This study highlights the efficacy of leveraging lower-fidelity estimates in NAS and paves the way for further research into accelerating the design of efficient CNN architectures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. Technical report 2 (2002)
Elsken, T., Metzen, J.H., Hutter, F.: Neural Architecture Search: A Survey. Technical report (2019). http://jmlr.org/papers/v20/18-598.html
Garcia-Garcia, C., Escalante, H.J., Morales-Reyes, A.: CGP-NAS. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, vol. 1, pp. 643–646. ACM, New York, NY, USA (2022). https://doi.org/10.1145/3520304.3528963
Garcia-Garcia, C., Morales-Reyes, A., Escalante, H.J.: Continuous cartesian genetic programming based representation for multi-objective neural architecture search. Appl. Soft Comput. 147, 110788 (2023). https://doi.org/10.1016/j.asoc.2023.110788
Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A survey of deep learning techniques for autonomous driving. J. Field Robot. 37(3), 362–386 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Kolbk, M., Tan, Z.H., Jensen, J., Kolbk, M., Tan, Z.H., Jensen, J.: Speech intelligibility potential of general and specialized deep neural network based speech enhancement systems. IEEE/ACM Trans. Audio Speech Lang. Proc. 25(1), 153–167 (2017). https://doi.org/10.1109/TASLP.2016.2628641
Liu, Q., Wang, X., Wang, Y., Song, X.: Evolutionary convolutional neural network for image classification based on multi-objective genetic programming with leader-follower mechanism. Complex Intell. Syst.(2022)
Liu, S., Zhang, H., Jin, Y.: A survey on computationally efficient neural architecture search. J. Autom. Intell. 1(1), 100002 (2022). https://doi.org/10.1016/j.jai.2022.100002
Lu, Z., Deb, K., Goodman, E., Banzhaf, W., Boddeti, V.N.: NSGANetV2: evolutionary multi-objective surrogate-assisted neural architecture search. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 35–51. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_3
Lu, Z., et al.: NSGA-Net. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 419–427. ACM, New York, NY, USA (2019)
Lu, Z., et al.: Multi-objective evolutionary design of deep convolutional neural networks for image classification. IEEE Trans. Evol. Comput., 1 (2020). https://doi.org/10.1109/TEVC.2020.3024708
Martinez, A.D., et al.: Lights and shadows in evolutionary deep learning: taxonomy, critical methodological analysis, cases of study, learned lessons, recommendations and challenges. Inf. Fusion 67, 161–194 (2021). https://doi.org/10.1016/j.inffus.2020.10.014
Miikkulainen, R., et al.: Evolving deep neural networks. In: Artificial Intelligence in the Age of Neural Networks and Brain Computing, pp. 293–312. Elsevier (2019)
Miller, J., Thomson, P., Fogarty, T., Ntroduction, I.: Designing electronic circuits using evolutionary algorithms. arithmetic circuits: a case study. Genet. Algorithms Evol. Strat. Eng. Comput. Sci. (1999)
Pinos, M., Mrazek, V., Sekanina, L.: Evolutionary approximation and neural architecture search. Genet. Programm. Evolvable Mach. (2022)
Real, E., et al.: Large-Scale Evolution of Image Classifiers (2017). http://arxiv.org/abs/1703.01041
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Suganuma, M., Kobayashi, M., Shirakawa, S., Nagao, T.: Evolution of deep convolutional neural networks using cartesian genetic programming (2020)
Sun, Y., Wang, H., Xue, B., Jin, Y., Yen, G.G., Zhang, M.: Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor. IEEE Trans. Evol. Comput. 24(2), 350–364 (2020). https://doi.org/10.1109/TEVC.2019.2924461
Termritthikun, C., Jamtsho, Y., Ieamsaard, J., Muneesawang, P., Lee, I.: EEEA-Net: an early exit evolutionary neural architecture search. Eng. Appl. Artif. Intell. 104, 104397 (2021). https://doi.org/10.1016/j.engappai.2021.104397
Torabi, A., Sharifi, A., Teshnehlab, M.: Using cartesian genetic programming approach with new crossover technique to design convolutional neural networks. Neural Process. Lett. (2022). https://doi.org/10.1007/s11063-022-11093-0
Xie, L., Yuille, A.: Genetic CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1388–1397. IEEE (2017)
Xue, Y., Jiang, P., Neri, F., Liang, J.: A Multi-objective evolutionary approach based on graph-in-graph for neural architecture search of convolutional neural networks. Int. J. Neural Syst. 31(9) (2021)
Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing [review article]. IEEE Comput. Intell. Mag. 13(3), 55–75 (2018). https://doi.org/10.1109/MCI.2018.2840738
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C., Fonseca, V.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7, 117–132 (2003)
Acknowledgements
Experiments presented in this paper were carried out using the Grid’5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Garcia-Garcia, C., Derbel, B., Morales-Reyes, A., Escalante, H.J. (2025). Speeding up the Multi-objective NAS Through Incremental Learning. In: Martínez-Villaseñor, L., Ochoa-Ruiz, G. (eds) Advances in Soft Computing. MICAI 2024. Lecture Notes in Computer Science(), vol 15247. Springer, Cham. https://doi.org/10.1007/978-3-031-75543-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-75543-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-75542-2
Online ISBN: 978-3-031-75543-9
eBook Packages: Computer ScienceComputer Science (R0)