More Web Proxy on the site http://driver.im/

Article

Free access

Deep lattice networks and partial monotonic functions

Authors:

Maya R. GuptaAuthors Info & Claims

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

Pages 2985 - 2993

Published: 04 December 2017 Publication History

PDF eReader Publisher Site

Abstract

We propose learning deep models that are monotonic with respect to a user-specified set of inputs by alternating layers of linear embeddings, ensembles of lattices, and calibrators (piecewise linear functions), with appropriate constraints for monotonicity, and jointly training the resulting network. We implement the layers and projections with new computational graph nodes in TensorFlow and use the Adam optimizer and batched stochastic gradients. Experiments on benchmark and real-world datasets show that six-layer monotonic deep lattice networks achieve state-of-the art performance for classification and regression with monotonicity guarantees.

References

[1]

M. R. Gupta, A. Cotter, J. Pfeifer, K. Voevodski, K. Canini, A. Mangylov, W. Moczydlowski, and A. Van Esbroeck. Monotonic calibrated interpolated look-up tables. Journal of Machine Learning Research, 17(109):1-47, 2016.

Digital Library

[2]

E. K. Garcia and M. R. Gupta. Lattice regression. In Advances in Neural Information Processing Systems (NIPS), 2009.

Digital Library

[3]

K. Canini, A. Cotter, M. M. Fard, M. R. Gupta, and J. Pfeifer. Fast and flexible monotonic functions with ensembles of lattices. Advances in Neural Information Processing Systems (NIPS), 2016.

Digital Library

[4]

A. Howard and T. Jebara. Learning monotonic transformations for classification. Advances in Neural Information Processing Systems (NIPS), 2007.

[5]

L. Lovész. Submodular functions and convexity. In Mathematical Programming The State of the Art, pages 235-257. Springer, 1983.

[6]

P. Groeneboom and G. Jongbloed. Nonparametric estimation under shape constraints. Cambridge Press, New York, USA, 2014.

[7]

N. P. Archer and S. Wang. Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems. Decision Sciences, 24(1):60-75, 1993.

[8]

S. Wang. A neural network method of density estimation for univariate unimodal data. Neural Computing & Applications, 2(3):160-167, 1994.

[9]

H. Kay and L. H. Ungar. Estimating monotonic functions and their bounds. AIChE Journal, 46(12):2426-2434, 2000.

[10]

C. Dugas, Y. Bengio, F. Bélisle, C. Nadeau, and R. Garcia. Incorporating functional knowledge in neural networks. Journal Machine Learning Research, 2009.

Digital Library

[11]

A. Minin, M. Velikova, B. Lang, and H. Daniels. Comparison of universal approximators incorporating partial monotonicity by structure. Neural Networks, 23(4):471-475, 2010.

Digital Library

[12]

H. Daniels and M. Velikova. Monotone and partially monotone neural networks. IEEE Trans. Neural Networks, 21(6):906-917, 2010.

Digital Library

[13]

J. Sill. Monotonic networks. Advances in Neural Information Processing Systems (NIPS), 1998.

Digital Library

[14]

W. W. Armstrong and M. M. Thomas. Adaptive logic networks. Handbook of Neural Computation, Section C1. 8, IOP Publishing and Oxford U. Press, ISBN 0 7503 0312, 3, 1996.

[15]

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.

[16]

D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

[17]

M. Ayer, H. D. Brunk, G. M. Ewing, W. T. William, E. Silverman, et al. An empirical distribution function for sampling with incomplete information. The annals of mathematical statistics, 26(4):641-647, 1955.

[18]

S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 3(1):1-122, 2011.

Digital Library

[19]

C. Blake and C. J. Merz. UCI repository of machine learning databases, 1998.

Cited By

Nanfack GTemple PFrénay B(2022)Constraint Enforcement on Decision Trees: A SurveyACM Computing Surveys10.1145/350673454:10s(1-36)Online publication date: 13-Sep-2022
https://dl.acm.org/doi/10.1145/3506734
Wei XChen RChen HZang B(2021) XStore: Fast RDMA-Based Ordered Key-Value Store Using Remote Learned CacheACM Transactions on Storage10.1145/346852017:3(1-32)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3468520
Qin JWang WXiao CZhang Y(2020)Similarity query processing for high-dimensional dataProceedings of the VLDB Endowment10.14778/3415478.341556413:12(3437-3440)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.14778/3415478.3415564
Show More Cited By

Recommendations

Research on Point-wise Gated Deep Networks

Display Omitted We introduce pgRBMs into DBNs and present Point-wise Gated Deep Belief Networks.Similar to pgDBNs, Point-wise Gated Deep Boltzmann Machines are presented.We introduce dropout and weight uncertainty methods into pgRBMs.We discuss the ...
Symmetric Power Activation Functions for Deep Neural Networks
LOPAL '18: Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and Applications

Common nonlinear activation functions with large saturation regions, like Sigmoid and Tanh, used for Deep Neural Networks (DNNs) can not guarantee useful and efficient training since they suffer from vanishing gradients problem. Rectified Linear Units ...
Chart classification by combining deep convolutional networks and deep belief networks
ICDAR '15: Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR)

Chart classification is the foundation of chart analysis and document understanding. In this paper, we propose a novel framework to classify charts by combining convolutional networks and deep belief networks. In the framework, we firstly extract deep ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

December 2017

7104 pages

ISBN:9781510860964

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 04 December 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
276
Total Downloads

Downloads (Last 12 months)153
Downloads (Last 6 weeks)13

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nanfack GTemple PFrénay B(2022)Constraint Enforcement on Decision Trees: A SurveyACM Computing Surveys10.1145/350673454:10s(1-36)Online publication date: 13-Sep-2022
https://dl.acm.org/doi/10.1145/3506734
Wei XChen RChen HZang B(2021) XStore: Fast RDMA-Based Ordered Key-Value Store Using Remote Learned CacheACM Transactions on Storage10.1145/346852017:3(1-32)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3468520
Qin JWang WXiao CZhang Y(2020)Similarity query processing for high-dimensional dataProceedings of the VLDB Endowment10.14778/3415478.341556413:12(3437-3440)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.14778/3415478.3415564
Meng CYang JRibeiro BNeville JTeredesai AKumar VLi YRosales RTerzi EKarypis G(2019)HATSProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3330876(783-792)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3292500.3330876
Gupta MBahri DCotter ACanini K(2018)Diminishing returns shape constraints for interpretability and regularizationProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3327757.3327788(6835-6845)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3327757.3327788

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents