Cited By
View all- Keskin CIzadi S(2018)SplineNetsProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3326943.3327128(1998-2008)Online publication date: 3-Dec-2018
Deep neural decision forests
Authors: 
Peter Kontschieder, Madalina Fiterau, Antonio Criminisi, Samuel Rota BulòAuthors Info & Claims
Pages 4190 - 4194
Abstract
We present a novel approach to enrich classification trees with the representation learning ability of deep (neural) networks within an end-to-end trainable architecture. We combine these two worlds via a stochastic and differentiable decision tree model, which steers the formation of latent representations within the hidden layers of a deep network. The proposed model differs from conventional deep networks in that a decision forest provides the final predictions and it differs from conventional decision forests by introducing a principled, joint and global optimization of split and leaf node parameters. Our approach compares favourably to other state-of-the-art deep models on a large-scale image classification task like ImageNet.
References
[1]
Y. Amit and D. Geman. Shape quantization and recognition with randomized trees. (NC) , 9(7):1545-1588, 1997.
[2]
Yoshua Bengio, Olivier Delalleau, and Clarence Simard. Decision trees do not generalize to new variations. Computational Intelligence , 26(4):449-467, 2010.
[3]
A. Bosch, A. Zisserman, and X. Muñoz. Image classification using random forests and ferns. In (ICCV) , 2007.
[4]
Leo Breiman. Random forests. Machine Learning , 45(1):5-32, 2001.
[5]
Gabriel J. Brostow, Jamie Shotton, Julien Fauqueur, and Roberto Cipolla. Segmentation and recognition using structure from motion point clouds. In (ECCV) . Springer, 2008.
[6]
R. Caruana, N. Karampatziakis, and A. Yessenalina. An empirical evaluation of supervised learning in high dimensions. In (ICML) , pages 96-103, 2008.
[7]
A. Criminisi and J. Shotton. Decision Forests in Computer Vision and Medical Image Analysis . Springer, 2013.
[8]
Andrej Karpathy Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In (CVPR) , 2015.
[9]
T. Hastie, R. Tibshirani, and J. H. Friedman. The Elements of Statistical Learning . Springer, 2009.
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. CoRR , abs/1502.01852, 2015.
[11]
David Heath, Simon Kasif, and Steven Salzberg. Induction of oblique decision trees. Journal of Artificial Intelligence Research , 2(2):1-32, 1993.
[12]
P. Kontschieder, P. Kohli, J. Shotton, and A. Criminisi. GeoF: Geodesic forests for learning coupled predictors. In (CVPR) , pages 65-72, 2013.
[13]
P. Kontschieder, M. Fiterau, A. Criminisi, and S. Rota Bul`o. Deep neural decision forests. In (ICCV) , 2015.
[14]
Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton. Imagenet classification with deep convolutional neural networks. In (NIPS) , 2012.
[15]
Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. CoRR , abs/1312.4400, 2013.
[16]
A. Montillo, J. Tu, J. Shotton, J. Winn, J. E. Iglesias, D. N. Metaxas, and A. Criminisi. Entangled forests and differentiable information gain maximization. In Decision Forests in Computer Vision and Medical Image Analysis . Springer, 2013.
[17]
Samuel Rota Bulò and Peter Kontschieder. Neural decision forests for semantic image labelling. In (CVPR) , 2014.
[18]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) , 2014.
[19]
Jamie Shotton, Ross Girshick, Andrew Fitzgibbon, Toby Sharp, Mat Cook, Mark Finocchio, Richard Moore, Pushmeet Kohli, Antonio Criminisi, Alex Kipman, and Andrew Blake. Efficient human pose estimation from single depth images. (PAMI) , 2013.
[20]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research , 15:1929-1958, 2014.
[21]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. CoRR , abs/1409.4842, 2014.
[22]
D. Yu and L. Deng. Automatic Speech Recognition: A Deep Learning Approach . Springer, 2014.
- Deep neural decision forests
Recommendations
Deep Neural Decision Forests
ICCV '15: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)We present Deep Neural Decision Forests - a novel approach that unifies classification trees with the representation learning functionality known from deep convolutional networks, by training them in an end-to-end manner. To combine these two worlds, we ...
Decision Boundary of Deep Neural Networks: Challenges and Opportunities
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data MiningOne crucial aspect that yet remains fairly unknown while can inform us about the behavior of deep neural networks is their decision boundaries. Trust can be improved once we understand how and why deep models carve out a particular form of decision ...
Edge-preserving image denoising using a deep convolutional neural network
Highlights- This paper makes use of a deep CNN for image denoising.
- The network is trained ...
AbstractThis paper introduces a novel denoising approach making use of a deep convolutional neural network to preserve image edges. The network is trained by using the edge map obtained from the well-known Canny algorithm and aims at ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sponsors
- Sony: Sony Corporation
- Arizona State University: Arizona State University
- Microsoft: Microsoft
- Facebook: Facebook
- AI Journal: AI Journal
Publisher
AAAI Press
Publication History
Published: 09 July 2016
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Dec 2024
Other Metrics
Citations
Cited By
View all- Keskin CIzadi S(2018)SplineNetsProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3326943.3327128(1998-2008)Online publication date: 3-Dec-2018