Unlearning from demonstration
Abstract
When doing learning from demonstration, it is often the case that the demonstrator provides corrective examples to fix errant behavior by the agent or robot. We present a set of algorithms which use this corrective data to identify and remove noisy examples in datasets which caused errant classifications, and ultimately errant behavior. The objective is to actually modify the source datasets rather than solely rely on the noise-insensitivity of the classification algorithm. This is particularly useful in the sparse datasets often found in learning from demonstration experiments. Our approach tries to distinguish between noisy misclassification and mere undersampling of the learning space. If errors are a result of misclassification, we potentially remove the responsible points and update the classifier. We demonstrate our method on UCI Machine Learning datasets at different levels of sparsity and noise, using decision trees, K-Nearest-Neighbor, and support vector machines.
References
[1]
Darrin C. Bentivegna, Christopher G. Atkeson, and Gordon Cheng. Learning tasks from observation and practice. Robotics and Autonomous Systems, 47(2-3):163-169, 2004.
[2]
Gert Cauwenberghs and Tomaso Poggio. Incremental and decremental support vector machine learning. In NIPS, 2001.
[3]
C. P. Diehl and Gert Cauwenberghs. SVM incremental learning, adaptation and optimization. In NIPS, volume 4, pages 2685-2690, 2003.
[4]
Jonathan Dinerstein, Parris K. Egbert, and Dan Ventura. Learning policies for embodied virtual agents through demonstration. In IJCAI, pages 1257-1252, 2007.
[5]
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental Bayesian approach testing on 101 object categories. Computer Vision and Image Understanding, 106(1):59-70, 2007.
[6]
Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. In Second European Conference on Computational Learning Theory, pages 23-37, 1995.
[7]
Terrence S. Furey, Nello Cristianini, Nigel Duffy, David W. Bednarski, Michél Schummer, and David Haussler. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics, 16(10):906-914, 2000.
[8]
Micheal Gamon. Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis. In International Conference on Computational Linguistics, 2004.
[9]
Johannes Gehrke, Venkatesh Ganti, Raghu Ramakrishnan, and Wei-Yin Loh. BOAT--optimistic decision tree construction. In International Conference on Management of Data, volume 28, pages 169-180, 1999.
[10]
Dani Goldberg and Maja J Mataric. Maximizing reward in a non-stationary mobile robot environment. Autonomous Agents and Multi-Agent Systems, 6, 2002.
[11]
Daniel H. Grollman and Aude G. Ballard. Robot learning from failed demonstrations. International Journal of Social Robotics, 4(4):331-342, Nov 2012.
[12]
Jiayuan Huang, Alexander J. Smola, Arthur Gretton, Karsten M. Borgwardt, and Bernhard Schölkopf. Correcting sample bias by unlabeled data. In NIPS, 2006.
[13]
José M. Jereza, Ignacio Molinab, Pedro J. García-Laencinac, Emilio Albad, Nuria Ribellesd, Miguel Martíne, and Leonardo Francoa. Missing data imputation using statistical and machine learning methods in a real breast cancer problem. Artificial Intelligence in Medicine, 50(2):105-115, 2010.
[14]
Elias Kalapanidas, Nikolaos Avouris, Marian Craciun, and Daniel Neagu. Machine learning algorithms: A study on noise sensitivity. In First Balkan Conference in Informatics, 2003.
[15]
Michael Kasper, Gernot Fricke, Katja Steuernagel, and Ewald von Puttkamer. A behavior-based mobile robot architecture for learning from demonstration. Robotics and Autonomous Systems, 34(2-3):153-164, 2001.
[16]
Kamakshi Lakshminarayan, Steven A. Harp, Robert Goldman, and Tariq Samad. Imputation of missing data using machine learning techniques. In KDD, 1996.
[17]
Junshui Ma, James Theiler, and Simon Perkins. Accurate on-line support vector regression. Neural Computation, 15:2683-2703, 2003.
[18]
Ryszard S Michalski, Igor Mozetic, Jiarong Hong, and Nada Lavrac. The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In AAAI, 1986.
[19]
Jun Nakanishi, Jun Morimoto, Gen Endo, Gordon Cheng, Stefan Schaal, and Mitsuo Kawato. Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47(2-3):79-91, 2004.
[20]
David F Nettleton, Albert Orriols-Puig, and Albert Fornells. A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial Intelligence Review, 33:275-306, 2010.
[21]
Monica N. Nicolescu and Maja J. Mataric. A hierarchical architecture for behavior-based robots. In AAMAS, pages 227-233. ACM, 2002.
[22]
Charles Parker, Prasad Tadepalli, Weng-Keen Wong, Thomas Dietterich, and Alan Fern. Learning from demonstrations via structured prediction. In AAAI, 2007.
[23]
Robi Polikar, Lalita Upda, Satish S. Upda, and Vasant Honavar. Learn++: An incremental learning algorithm for supervised neural networks. IEEE SMC, Part C, 31(4):497-508, Nov 2001.
[24]
David A Ross, Jongwoo Lim, Ruei-Sung Lin, and Ming-Hsuan Yang. Incremental learning for robust visual tracking. International Journal of Computer Vision, 77(1-3):125-141, 2008.
[25]
Nadeem Ahmed Syed, Huan Liu, and Kah Kay Sung. Handling concept drifts in incremental learning with support vector machines. In KDD, 1999.
[26]
Harini Veeraraghavan and Manuela M. Veloso. Learning task specific plans through sound and visually interpretable demonstrations. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2599-2604. IEEE, 2008.
- Unlearning from demonstration
Recommendations
Machine Unlearning in Gradient Boosting Decision Trees
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data MiningVarious machine learning applications take users' data to train the models. Recently enforced legislation requires companies to remove users' data upon requests, i.e.,the right to be forgotten. In the context of machine learning, the trained model ...
Introspective Reinforcement Learning and Learning from Demonstration
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent SystemsReinforcement learning is a paradigm to model how an autonomous agent learns to maximise its cumulative reward by interacting with the environment. One challenge faced by reinforcement learning is that in many environments the reward signal is sparse, ...
Active deep Q-learning with demonstration
AbstractReinforcement learning (RL) is a machine learning technique aiming to learn how to take actions in an environment to maximize some kind of reward. Recent research has shown that although the learning efficiency of RL can be improved with expert ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sponsors
- The International Joint Conferences on Artificial Intelligence, Inc. (IJCAI)
Publisher
AAAI Press
Publication History
Published: 03 August 2013
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 8Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Jan 2025