Abstract
Bayesian optimisation is an efficient method for global optimisation of expensive black-box functions. However, the current Gaussian process based methods cater to functions with arbitrary smoothness, and do not explicitly model the fact that most of the real world optimisation problems are well-behaved functions with only a few peaks. In this paper, we incorporate such shape constraints through the use of a derivative meta-model. The derivative meta-model is built using a Gaussian process with a polynomial kernel and derivative samples from this meta-model are used as extra observations to the standard Bayesian optimisation procedure. We provide a Bayesian framework to infer the degree of the polynomial kernel. Experiments on both benchmark functions and hyperparameter tuning problems demonstrate the superiority of our approach over baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning, vol. 1. MIT Press, Cambridge (2006)
Riihimäki, J., Vehtari, A.: Gaussian processes with monotonicity information. In: Proceedings of the Thirteenth International Conference on AIStat. (2010)
Jauch, M., Peña, V.: Bayesian optimization with shape constraints. arXiv preprint arXiv:1612.08915 (2016)
Mockus, J.: Application of bayesian approach to numerical methods of global and stochastic optimization. J. Global Optim. 4(4), 347–365 (1994)
Denison, D.G.T., Mallick, B.K., Smith, A.F.M.: Automatic Bayesian curve fitting. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 60(2), 333–350 (1998)
Abdolmaleki, A., Lioutikov, R., Peters, J.R., Lau, N., Reis, L.P., Neumann, G.: Model-based relative entropy stochastic search. In: NIPS (2015)
Solak, E., Murray-Smith, R., Leithead, W.E., Leith, D.J., Rasmussen, C.E.: Derivative observations in gaussian process models of dynamic systems. In: NIPS (2003)
Finkel, D.E.: Direct optimization algorithm user guide. CRSC (2003)
Dheeru, D., Karra Taniskidou, E.: UCI Machine Learning Repository (2017)
Acknowledgment
This research was partially funded by the Australian Government through the Australian Research Council (ARC) and the Telstra-Deakin Centre of Excellence in Big Data and Machine Learning. Professor Venkatesh is the recipient of an ARC Australian Laureate Fellowship (FL170100006).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Yang, A., Li, C., Rana, S., Gupta, S., Venkatesh, S. (2018). Efficient Bayesian Optimisation Using Derivative Meta-model. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11013. Springer, Cham. https://doi.org/10.1007/978-3-319-97310-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-97310-4_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97309-8
Online ISBN: 978-3-319-97310-4
eBook Packages: Computer ScienceComputer Science (R0)