More Web Proxy on the site http://driver.im/

poster

Efficient guided evolution for neural architecture search

Authors:

Bruno Degardin,

Luís A. AlexandreAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 655 - 658

https://doi.org/10.1145/3520304.3528936

Published: 19 July 2022 Publication History

Abstract

Neural Architecture Search methods have been successfully applied to image tasks with excellent results. However, NAS methods are often complex and tend to quickly converge for local minimas. In this paper, we propose G-EA, a novel approach for guided NAS. G-EA guides the evolution by exploring the search space by generating and evaluating several architectures in each generation at initialisation stage using a zero-proxy estimator, where only the highest-scoring architecture is trained and kept for the next generation. By generating several off-springs from an existing architecture at each generation, G-EA continuously extracts knowledge about the search space without added complexity. More, G-EA forces exploitation of the most performant architectures by descendant generation while at the same time forcing exploration by parent mutation and favouring younger architectures to the detriment of older ones. Experimental results demonstrate the effectiveness of the proposed method. Results show that G-EA achieves state-of-the-art results in NAS-Bench-101 and in all NAS-Bench-201 search space data sets: CIFAR-10, CIFAR-100 and ImageNet16-120, with mean accuracies of 93.99%, 72.62% and 46.04% respectively.

References

[1]

Pouya Bashivan, Mark Tensen, and James J. DiCarlo. 2019. Teacher Guided Architecture Search. In ICCV.

[2]

Fabio Maria Carlucci, Pedro M Esperança, Marco Singh, Victor Gabillon, Antoine Yang, Hang Xu, Zewei Chen, and Jun Wang. 2019. MANAS: Multi-agent neural architecture search. arXiv preprint arXiv:1909.01051 (2019).

[3]

François Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In CVPR. 1251--1258.

[4]

Alexis Conneau, Holger Schwenk, Loïc Barrault, and Yann LeCun. 2017. Very Deep Convolutional Networks for Text Classification. In EACL.

[5]

Li Deng, Dong Yu, et al. 2014. Deep learning: methods and applications. Foundations and Trends in Signal Processing (2014).

[6]

Xuanyi Dong and Yi Yang. 2019. One-Shot Neural Architecture Search via Self-Evaluated Template Network. In ICCV.

[7]

Xuanyi Dong and Yi Yang. 2019. Searching for a robust neural architecture in four gpu hours. In CVPR.

[8]

Xuanyi Dong and Yi Yang. 2020. NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search. In ICLR.

[9]

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR.

[10]

Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. Journal of Machine Learning Research (2019).

[11]

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning. Vol. 1. MIT press Cambridge.

Digital Library

[12]

Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR.

[13]

Liam Li and Ameet Talwalkar. 2020. Random search and reproducibility for neural architecture search. In UAI.

[14]

Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. 2018. Progressive neural architecture search. In ECCV.

[15]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. In ICLR.

[16]

Vasco Lopes and Luís A Alexandre. 2020. Auto-classifier: A robust defect detector based on an automl head. In ICONIP.

[17]

Vasco Lopes, Saeid Alirezazadeh, and Luís A Alexandre. 2021. EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search. In ICANN.

[18]

Vasco Lopes, António Gaspar, Luís A Alexandre, and João Cordeiro. 2021. An AutoML-based Approach to Multimodal Image Sentiment Analysis. In IJCNN.

[19]

Joseph Mellor, Jack Turner, Amos J. Storkey, and Elliot J. Crowley. 2021. Neural Architecture Search without Training. In ICML.

[20]

Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In ICML.

[21]

Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. 2019. Regularized Evolution for Image Classifier Architecture Search. In AAAI.

[22]

Mingxing Tan and Quoc V. Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In ICML.

[23]

Colin White, Arber Zela, Binxin Ru, Yang Liu, and Frank Hutter. 2021. How Powerful are Performance Predictors in Neural Architecture Search?. In NeurIPS.

[24]

Martin Wistuba, Ambrish Rawat, and Tejaswini Pedapati. 2019. A Survey on Neural Architecture Search. CoRR abs/1905.01392 (2019).

[25]

Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. 2020. PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. In ICLR.

[26]

Antoine Yang, Pedro M. Esperança, and Fabio M. Carlucci. 2020. NAS evaluation is frustratingly hard. In ICLR.

[27]

Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. 2019. NAS-Bench-101: Towards Reproducible Neural Architecture Search. In ICML.

[28]

Kaicheng Yu, Rene Ranftl, and Mathieu Salzmann. 2021. Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search. In CVPR.

[29]

Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. 2020. Understanding and Robustifying Differentiable Architecture Search. In ICLR.

[30]

Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, and Cheng-Lin Liu. 2018. Practical block-wise neural network architecture generation. In CVPR.

[31]

Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. In ICLR.

[32]

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2018. Learning Transferable Architectures for Scalable Image Recognition. CVPR (2018).

Cited By

Heuillet ANasser AArioui HTabia H(2024)Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture SearchACM Computing Surveys10.1145/366513856:11(1-36)Online publication date: 28-Jun-2024
https://doi.org/10.1145/3665138
Vo ALuong NLi XHandl J(2024)Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty SearchProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654064(1146-1155)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654064
Guo BXu LChen TYe PHe SLiu HChen J(2024)Latency-Aware Neural Architecture Performance Predictor With Query-to-Tier TechniqueIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328768434:7(5868-5883)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3287684
Show More Cited By

Index Terms

Efficient guided evolution for neural architecture search
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
    2. Search methodologies
  2. Machine learning
    1. Machine learning algorithms
    2. Machine learning approaches
      1. Neural networks

Recommendations

Auto-Keras: An Efficient Neural Architecture Search System
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a ...
Guided evolutionary neural architecture search with efficient performance estimation
Abstract
Neural Architecture Search (NAS) methods have been successfully applied to image tasks with excellent results. However, NAS methods are often complex and tend to converge to local minima as soon as generated architectures yield good results. This ...
Neural architecture search using progressive evolution
GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference

Vanilla neural architecture search using evolutionary algorithms (EA) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet to estimate the fitness of every ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2022

2395 pages

ISBN:9781450392686

DOI:10.1145/3520304

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Copyright © 2022 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

DeepNeuronic
NOVA Lincs

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
122
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)5

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Heuillet ANasser AArioui HTabia H(2024)Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture SearchACM Computing Surveys10.1145/366513856:11(1-36)Online publication date: 28-Jun-2024
https://doi.org/10.1145/3665138
Vo ALuong NLi XHandl J(2024)Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty SearchProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654064(1146-1155)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654064
Guo BXu LChen TYe PHe SLiu HChen J(2024)Latency-Aware Neural Architecture Performance Predictor With Query-to-Tier TechniqueIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328768434:7(5868-5883)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3287684
Le NVo ALuong N(2024)Zero-Cost Proxy-Based Hierarchical Initialization for Evolutionary Neural Architecture Search2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10611995(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/CEC60901.2024.10611995
Xiao SZhao BLiu D(2024)Semi-supervised accuracy predictor-based multi-objective neural architecture searchNeurocomputing10.1016/j.neucom.2024.128472(128472)Online publication date: Aug-2024
https://doi.org/10.1016/j.neucom.2024.128472
Lopes VSantos MDegardin BAlexandre L(2024)Guided evolutionary neural architecture search with efficient performance estimationNeurocomputing10.1016/j.neucom.2024.127509584(127509)Online publication date: Jun-2024
https://doi.org/10.1016/j.neucom.2024.127509
Luong NPhan QVo APham TBui D(2024)Lightweight multi-objective evolutionary neural architecture search with low-cost proxy metricsInformation Sciences: an International Journal10.1016/j.ins.2023.119856655:COnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.ins.2023.119856
Chen WGong XWu JWei YShi HYan ZYang YWang Z(2023)Understanding and Accelerating Neural Architecture Search With Training-Free and Theory-Grounded MetricsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.332834746:2(749-763)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3328347
Zhang YWang MZhao DLiu CLiu Z(2023)Early weed identification based on deep learning: A reviewSmart Agricultural Technology10.1016/j.atech.2022.1001233(100123)Online publication date: Feb-2023
https://doi.org/10.1016/j.atech.2022.100123
Lopes VCarlucci FEsperança PSingh MYang AGabillon VXu HChen ZWang J(2023)Manas: multi-agent neural architecture searchMachine Learning10.1007/s10994-023-06379-wOnline publication date: 12-Sep-2023
https://doi.org/10.1007/s10994-023-06379-w

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents