More Web Proxy on the site http://driver.im/

poster

Evolving deep autoencoders

Authors:

Suely Oliveira,

Xiaoyu XingAuthors Info & Claims

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

Pages 123 - 124

https://doi.org/10.1145/3377929.3390011

Published: 08 July 2020 Publication History

Abstract

Autoencoders have seen wide success in domains ranging from feature selection to information retrieval. Despite this success, designing an autoencoder for a given task remains a challenging undertaking due to the lack of firm intuition on how the backing neural network architectures of the encoder and decoder impact the overall performance of the autoencoder. In this work we present a distributed system that uses an efficient evolutionary algorithm to design a modular autoencoder. We demonstrate the effectiveness of this system on the tasks of manifold learning and image denoising. The system beats random search by nearly an order of magnitude on both tasks while achieving near linear horizontal scaling as additional worker nodes are added to the system.

References

[1]

Francisco Charte, Antonio J Rivera, Francisco Martínez, and María J del Jesus. 2019. Automating Autoencoder Architecture Configuration: An Evolutionary Approach. In International Work-Conference on the Interplay Between Natural and Artificial Computation. Springer, 339--349.

[2]

Jeff Hajewski and Suely Oliveira. 2020. A Scalable System for Neural Architecture Search. In IEEE Computing and Communication Workshop and Conference (CCWC'20).

[3]

Jeff Hajewski, Suely Oliveira, and Xiaoyu Xing. 2020. Distributed Evolution of Deep Autoencoders. (2020). arXiv:cs.NE/2004.07607

[4]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. IEEE Computer Society, 770--778.

[5]

Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. 2013. Evolving Large-scale Neural Networks for Vision-based Reinforcement Learning. In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (GECCO '13). ACM, New York, NY, USA, 1061--1068.

Digital Library

[6]

S. Lander and Y. Shang. 2015. EvoAE - A New Evolutionary Method for Training Autoencoders for Deep Learning Networks. In 2015 IEEE 39th Annual Computer Software and Applications Conference, Vol. 2. 790--795.

Digital Library

[7]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. DARTS: Differentiable Architecture Search. CoRR abs/1806.09055 (2018).

[8]

Yunqian Ma and Yun Fu. 2011. Manifold learning theory and applications. CRC press.

[9]

Risto Miikkulainen, Jason Zhi Liang, Elliot Meyerson, Aditya Rawal, Daniel Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and BabakHodjat. 2017. Evolving Deep Neural Networks. CoRR abs/1703.00548 (2017). arXiv:1703.00548

[10]

Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol. 80. PMLR, Stockholmsmässan, Stockholm Sweden, 4095--4104.

[11]

Masanori Suganuma, Mete Ozay, and Takayuki Okatani. 2018. Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol. 80. PMLR, Stockholmsmässan, Stockholm Sweden, 4771--4780. http://proceedings.mlr.press/v80/suganuma18a.html

[12]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2015. Rethinking the Inception Architecture for Computer Vision. CoRR abs/1512.00567 (2015).

[13]

Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. J. Mach. Learn. Res. 11 (2010), 3371--3408. http://portal.acm.org/citation.cfm?id=1953039

Digital Library

[14]

Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Computer Vision - ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer International Publishing, Cham, 818--833.

[15]

Chris Zhang, Mengye Ren, and Raquel Urtasun. 2018. Graph HyperNetworks for Neural Architecture Search. CoRR abs/1810.05749 (2018).

[16]

Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. https://arxiv.org/abs/1611.01578

Cited By

Hemberg EO'Reilly UToutouh JLi XHandl J(2024)Cooperative Coevolutionary Spatial Topologies for Autoencoder TrainingProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654127(331-339)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654127
Li NMa LXing TYu GWang CWen YCheng SGao S(2023)Automatic design of machine learning via evolutionary computation: A surveyApplied Soft Computing10.1016/j.asoc.2023.110412143(110412)Online publication date: Aug-2023
https://doi.org/10.1016/j.asoc.2023.110412
Vargas-Hakim GMezura-Montes EAcosta-Mesa H(2022)A Review on Convolutional Neural Network Encodings for NeuroevolutionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.308863126:1(12-27)Online publication date: Mar-2022
https://doi.org/10.1109/TEVC.2021.3088631

Index Terms

Evolving deep autoencoders
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Anomaly Detection with Robust Deep Autoencoders
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Deep autoencoders, and other deep neural networks, have demonstrated their effectiveness in discovering non-linear features across many problem domains. However, in many real-world problems, large outliers and pervasive noise are commonplace, and one ...
Winner-take-all autoencoders
NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2

In this paper, we propose a winner-take-all method for learning hierarchical sparse representations in an unsupervised fashion. We first introduce fully-connected winner-take-all autoencoders which use mini-batch statistics to directly enforce a ...
Hyper Autoencoders
Abstract
We introduce the hyper autoencoder architecture where a secondary, hypernetwork is used to generate the weights of the encoder and decoder layers of the primary, actual autoencoder. The hyper autoencoder uses a one-layer linear hypernetwork to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

July 2020

1982 pages

ISBN:9781450371278

DOI:10.1145/3377929

General Chair:
Carlos Artemio Coello Coello
CINVESTAV-IPN

Copyright © 2020 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2020

Check for updates

Qualifiers

Poster

Conference

GECCO '20

Sponsor:

SIGEVO

GECCO '20: Genetic and Evolutionary Computation Conference

July 8 - 12, 2020

Cancún, Mexico

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
105
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hemberg EO'Reilly UToutouh JLi XHandl J(2024)Cooperative Coevolutionary Spatial Topologies for Autoencoder TrainingProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654127(331-339)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654127
Li NMa LXing TYu GWang CWen YCheng SGao S(2023)Automatic design of machine learning via evolutionary computation: A surveyApplied Soft Computing10.1016/j.asoc.2023.110412143(110412)Online publication date: Aug-2023
https://doi.org/10.1016/j.asoc.2023.110412
Vargas-Hakim GMezura-Montes EAcosta-Mesa H(2022)A Review on Convolutional Neural Network Encodings for NeuroevolutionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.308863126:1(12-27)Online publication date: Mar-2022
https://doi.org/10.1109/TEVC.2021.3088631

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents