article

Adapting RBF Neural Networks to Multi-Instance Learning

Authors:

Min-Ling Zhang,

Zhi-Hua ZhouAuthors Info & Claims

Neural Processing Letters, Volume 23, Issue 1

Pages 1 - 26

https://doi.org/10.1007/s11063-005-2192-z

Published: 01 February 2006 Publication History

Abstract

In multi-instance learning, the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. In this paper, a neural network based multi-instance learning algorithm named RBF-MIP is presented, which is derived from the popular radial basis function (RBF) methods. Briefly, the first layer of an RBF-MIP neural network is composed of clusters of bags formed by merging training bags agglomeratively, where Hausdorff metric is utilized to measure distances between bags and between clusters. Weights of second layer of the RBF-MIP neural network are optimized by minimizing a sum-of-squares error function and worked out through singular value decomposition (SVD). Experiments on real-world multi-instance benchmark data, artificial multi-instance benchmark data and natural scene image database retrieval are carried out. The experimental results show that RBF-MIP is among the several best learning algorithms on multi-instance problems.

References

[1]

1. Alphonse, E. and Matwin, S.: Filtering multi-instance problems to reduce dimensionality in relational learning, Journal of Intelligent Information Systems , 22 (1) (2004), 23-40.]]

Digital Library

Google Scholar

[2]

2. Amar, R. A., Dooly, D. R., Goldman, S. A. and Zhang, Q.: Multiple-instance learning of real-valued data, In: Proceedings of the 18th International Conference on Machine Learning , pp. 3-10, Williamstown, MA, 2001. {http://www.cs.wustl.edu/~sg/multi-inst-data}]]

Digital Library

Google Scholar

[3]

3. Andrews, S., Tsochantaridis, I. and Hofmann, T.: Support vector machines for multiple-instance learning, In: Becker, S., Thrun, S. and Obermayer, K. (eds), Advances in Neural Information Processing Systems 15 , pp. 561-568, MIT Press, Cambridge, MA, 2003.]]

Google Scholar

[4]

4. Auer, P.: On learning from multi-instance examples: empirical evaluation of a theoretical approach, In: Proceedings of the 14th International Conference on Machine Learning , pp. 21-29, Nashville, TN, 1997.]]

Digital Library

Google Scholar

[5]

5. Auer, P., Long, P. M. and Srinivasan, A.: Approximating hyper-rectangles: learning and pseudo-random sets, Journal of Computer and System Sciences , 57 (3) (1998), 376-388.]]

Digital Library

Google Scholar

[6]

6. Bishop, C. M.: Neural Networks for Pattern Recognition , Oxford University Press, New York, 1995.]]

Digital Library

Google Scholar

[7]

7. Blake, C., Keogh, E. and Merz, C. J.: UCI repository of machine learning databases . Department of Information and Computer Science, University of California, Irvine, CA, 1998. {http://www.ics.uci.edu/~mlearn/MLRepository.html}]]

Google Scholar

[8]

8. Blum, A. and Kalai, A.: A note on learning from multiple-instance examples, Machine Learning , 30 (1) (1998), 23-29.]]

Digital Library

Google Scholar

[9]

9. Chevaleyre, Y. and Zucker, J.-D.: Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis problem, In: Stroulia, E. and Matwin, S. (eds), Lecture Notes in Artificial Intelligence 2056 , pp. 204-214, Springer, Berlin, 2001.]]

Digital Library

Google Scholar

[10]

10. Dempster, A. P., Laird, N. M. and Rubin, D. B.: Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistics Society, Series B , 39 (1) (1977), 1-38.]]

Google Scholar

[11]

11. De Raedt, L.: Attribute-value learning versus inductive logic programming: the missing links, In: Page, D. (ed.), Lecture Notes in Artificial Intelligence 1446 , pp. 1-8, Springer, Berlin, 1998.]]

Digital Library

Google Scholar

[12]

12. Dietterich, T. G., Lathrop, R. H. and Lozano-Pérez, T.: Solving the multiple-instance problem with axis-parallel rectangles, Artificial Intelligence , 89 (1-2) (1997), 31-71.]]

Digital Library

Google Scholar

[13]

13. Dooly, D. R., Goldman, S. A. and Kwek, S. S.: Real-valued multiple-instance learning with queries, In: Abe, N., Khardon, R. and Zeugmann, T. (eds), Lecture Notes in Artificial Intelligence 2225 , pp. 167-180, Springer, Berlin, 2001.]]

Digital Library

Google Scholar

[14]

14. Edgar, G. A.: Measure, Topology, and Fractal Geometry , 3rd print, Springer-Verlag, Berlin, 1995.]]

Google Scholar

[15]

15. Gärtner; T., Flach, P. A., Kowalczyk, A. and Smola, A. J.: Multi-instance kernels, In: Proceedings of the 19th International Conference on Machine Learning , pp. 179-186, Sydney, Australia, 2002.]]

Digital Library

Google Scholar

[16]

16. Goldman, S. A., Kwek, S. S. and Scott, S. D.: Agnostic learning of geometric patterns, Journal of Computer and System Sciences , 62 (1) (2001), 123-151.]]

Digital Library

Google Scholar

[17]

17. Goldman, S. A. and Scott, S. D.: Multiple-instance learning of real-valued geometric patterns, Annals of Mathematics and Artificial Intelligence , 39 (3) (2003), 259-290.]]

Digital Library

Google Scholar

[18]

18. Jollife, I. T.: Principle Component Analysis , Springer-Verlag, New York, 1986.]]

Crossref

Google Scholar

[19]

19. Kearns, M. J.: Efficient noise-tolerant learning from statistical queries, In: Proceedings of the 25th Annual ACM Symposium on Theory of Computing , pp. 392-401, San Diego, CA, 1993.]]

Digital Library

Google Scholar

[20]

20. Kearns, M. J. and Schapire, R. E.: Efficient distribution-free learning of probabilistic concepts, Journal of Computer and System Sciences , 48 (3) (1994), 464-497.]]

Digital Library

Google Scholar

[21]

21. Lindsay, R., Buchanan, B., Feigenbaum, E. and Lederberg, J.: Applications of Artificial Intelligence to Organic Chemistry: The DENDRAL Project , McGraw-Hill, New York, 1980.]]

Digital Library

Google Scholar

[22]

22. Long, P. M. and Tan, L.: PAC learning axis-aligned rectangles with respect to product distribution from multiple-instance examples, Machine Learning , 30 (1) (1998), 7-21.]]

Digital Library

Google Scholar

[23]

23. Maron, O.: Learning from Ambiguity, PhD dissertation, Department of Electronical Engineering and Computer Science, MIT, Cambridge, MA, Jun. 1998.]]

Digital Library

Google Scholar

[24]

24. Maron, O. and Lozano-Pérez, T.: A framework for multiple-instance learning, In: Jordan, M. I., Kearns, M. J. and Solla, S. A. (eds), Advances in Neural Information Processing Systems 10 , pp. 570-576, MIT Press, Cambridge, MA, 1998.]]

Digital Library

Google Scholar

[25]

25. Maron, O. and Ratan, A. L.: Multiple-instance learning for natural scene classification, In: Proceedings of the 15th International Conference on Machine Learning , pp. 341-349, Madison, WI, 1998.]]

Digital Library

Google Scholar

[26]

26. Press, W. H., Teukolsky, S. A., Vetterling, W. T. and Flannery, B. P.: Numerical Recipes in C: The Art of Scientific Computing , 2nd edn, Cambridge University Press, New York, 1992.]]

Digital Library

Google Scholar

[27]

27. Ray, S. and Page, D.: Multiple instance regression, In: Proceedings of the 18th International Conference on Machine Learning , pp. 425-432, Williamstown, MA, 2001.]]

Digital Library

Google Scholar

[28]

28. Ruffo, G.: Learning single and multiple decision trees for security applications, PhD dissertation, Department of Computer Science, University of Turin, Italy, 2000.]]

Google Scholar

[29]

29. Rumelhart, D. E., Hinton, G. E. and Williams, R. J.: Learning internal representations by error propagation, In: Rumelhart, D. E. and McClelland, J. L. (eds), Parallel Distributed Processing: Explorations in the Microstructure of Cognition , Vol. 1, pp. 318-362, MIT Press, Cambridge, MA, 1986.]]

Digital Library

Google Scholar

[30]

30. Sebag, M. and Rouveirol, C.: Tractable induction and classification in first order logic, In: Proceedings of the 15th International Joint Conference on Artificial Intelligence , pp. 888-893, Nagoya, Japan, 1997.]]

Google Scholar

[31]

31. Wang, J. and Zucker, J.-D.: Solving the multiple-instance problem: a lazy learning approach, In: Proceedings of the 17th International Conference on Machine Learning , pp. 1119-1125, San Francisco, CA, 2000.]]

Digital Library

Google Scholar

[32]

32. Yang, C. and Lozano- Pérez, T.: Image database retrieval with multiple-instance learning techniques, In: Proceedings of the 16th International Conference on Data Engineering , pp. 233-243, San Diego, CA, 2000.]]

Digital Library

Google Scholar

[33]

33. Zhang, Q. and Goldman, S. A.: EM-DD: an improved multiple-instance learning technique, In: Dietterich, T. G., Becker, S. and Ghahramani, Z. (eds), Advances in Neural Information Processing Systems 14 , pp. 1073-1080, MIT Press, Cambridge, MA, 2002.]]

Google Scholar

[34]

34. Zhang, Q., Yu, W., Goldman, S. A. and Fritts, J. E.: Content-based image retrieval using multiple-instance learning, In: Proceedings of the 19th International Conference on Machine Learning , pp. 682-689, Sydney, Australia, 2002.]]

Digital Library

Google Scholar

[35]

35. Zhang, M.-L. and Zhou, Z.-H.: Improve multi-instance neural network through feature selection, Neural Processing Letters , 19 (1) (2004), 1-10.]]

Digital Library

Google Scholar

[36]

36. Zhou, Z.-H., Wu, J. and Tang, W.: Ensembling neural networks: many could be better than all, Artificial Intelligence , 137 (1-2) (2002), 239-263.]]

Digital Library

Google Scholar

[37]

37. Zhou, Z.-H. and Zhang, M.-L.: Neural networks for multi-instance learning, Technical Report, AI Lab, Computer Science & Technology Department, Nanjing University, China, Aug. 2002.]]

Google Scholar

[38]

38. Zhou, Z.-H. and Zhang, M.-L.: Ensembles of multi-instance learners, In: Lavrač, N., Gamberger, D., Blockeel, H. and Todorovski, L. (eds), Lecture Notes in Artificial Intelligence 2837 , pp. 492-502, Springer-Verlag, Berlin, 2003.]]

Google Scholar

[39]

39. Zucker, J.-D. and Ganascia, J.-G.: Changes of representation for efficient learning in structural domains, In: Proceedings of the 13th International Conference on Machine Learning , pp. 543-551, Bary, Italy, 1996.]]

Google Scholar

[40]

40. Zucker, J.-D. and Ganascia, J.-G.: Learning structurally indeterminate clauses, In: Page, D. (ed.), Lecture Notes in Artificial Intelligence 1446 , pp. 235-244, Springer, Berlin, 1998.]]

Digital Library

Google Scholar

Cited By

View all

Zhao LYuan LHao KWen X(2022)Generalized attention-based deep multi-instance learningMultimedia Systems10.1007/s00530-022-00992-w29:1(275-287)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1007/s00530-022-00992-w
Yi JZhou B(2022)Attention Awareness Multiple Instance Neural NetworkArtificial Neural Networks and Machine Learning – ICANN 202210.1007/978-3-031-15934-3_48(581-592)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-15934-3_48
Yuan CHuang HFeng CLiu XWei X(2019)Distant supervision for relation extraction with linear attenuation simulation and non-IID relevance embeddingProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v33i01.33017418(7418-7425)Online publication date: 27-Jan-2019
https://dl.acm.org/doi/10.1609/aaai.v33i01.33017418
Show More Cited By

Index Terms

Adapting RBF Neural Networks to Multi-Instance Learning

Recommendations

Improve Multi-Instance Neural Networks through Feature Selection

Multi-instance learning is regarded as a new learning framework where the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. ...
Multi-instance clustering with applications to multi-instance prediction

In the setting of multi-instance learning, each object is represented by a bag composed of multiple instances instead of by a single instance in a traditional learning setting. Previous works in this area only concern multi-instance prediction problems ...
Ml-rbf: RBF Neural Networks for Multi-Label Learning

Multi-label learning deals with the problem where each instance is associated with multiple labels simultaneously. The task of this learning paradigm is to predict the label set for each unseen instance, through analyzing training instances with known ...

Reviews

Reviewer: Mario A. Aoun

A remarkable advancement in the latest research on multi-instance learning?a branch of the connectionist approach in machine learning?is brought to light in this paper. An insightful design modification in the structure of a radial basis function (RBF) neural network [1] to solve multi-instance problems [2], called radial basis function for multi-instance problems (RBF-MIP), is considered. The innovative method relies on input representation of the regular RBF neural network. Through the work of Bishop [3], clusters of instances were used to present first-layer input of an RBF neural network, but the authors have advanced the design to clusters of bags, where a bag is a set of instances and each instance is a numerical vector. Also, they enhanced "the adjustment of the second layer weights" in terms of speed, by minimizing the error function. Thus, they turned the glitch into a linear problem solved by linear matrix inversion techniques. In addition, the implementation of the Hausdorff metric to calculate differences between bags makes the work more recognizable in multi-instance learning approaches. Such modifications let RBF-MIP compete with previous algorithms in this arena like back propagation for MIP [4] and diverse density [5]. The paper presents a detailed bibliography on MIP research, and focuses on the problem of drug activity prediction and the work of Dietterich, the father of multi-instance learning. The new algorithm, RBF-MIP, is presented in a concrete mathematical model with further explanations. In addition, different experiments were conducted on artificial datasets, MUSK datasets, and a "natural scene image database retrieval" problem?an interesting problem of a system capable of retrieving images similar to an input image. All of these experiments provide statistical results that show the success of the RBF-MIP algorithm.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Neural Processing Letters Volume 23, Issue 1

February 2006

107 pages

ISSN:1370-4621

Issue’s Table of Contents

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 February 2006

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhao LYuan LHao KWen X(2022)Generalized attention-based deep multi-instance learningMultimedia Systems10.1007/s00530-022-00992-w29:1(275-287)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1007/s00530-022-00992-w
Yi JZhou B(2022)Attention Awareness Multiple Instance Neural NetworkArtificial Neural Networks and Machine Learning – ICANN 202210.1007/978-3-031-15934-3_48(581-592)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-15934-3_48
Yuan CHuang HFeng CLiu XWei X(2019)Distant supervision for relation extraction with linear attenuation simulation and non-IID relevance embeddingProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v33i01.33017418(7418-7425)Online publication date: 27-Jan-2019
https://dl.acm.org/doi/10.1609/aaai.v33i01.33017418
Guo JRen THuang LBei J(2019)Saliency detection on sampled images for tag rankingMultimedia Systems10.1007/s00530-017-0546-925:1(35-47)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s00530-017-0546-9
Abpeykar SGhatee M(2019)An ensemble of RBF neural networks in decision tree structure with knowledge transferring to accelerate multi-classificationNeural Computing and Applications10.1007/s00521-018-3543-931:11(7131-7151)Online publication date: 1-Nov-2019
https://dl.acm.org/doi/10.1007/s00521-018-3543-9
Chai JLiu WTsang IShen XLim EWinslett MSanderson MFu ASun JCulpepper SLo EHo JDonato DAgrawal RZheng YCastillo CSun ATseng VLi C(2017)Compact Multiple-Instance LearningProceedings of the 2017 ACM on Conference on Information and Knowledge Management10.1145/3132847.3133070(2007-2010)Online publication date: 6-Nov-2017
https://dl.acm.org/doi/10.1145/3132847.3133070
Zhang HWu WWang D(2017)Multi‐instance multi‐label learning of natural scene imagesIET Computer Vision10.1049/iet-cvi.2016.033812:3(305-311)Online publication date: 18-Dec-2017
https://dl.acm.org/doi/10.1049/iet-cvi.2016.0338
Xu TGondra IChiu D(2017)A maximum partial entropy-based method for multiple-instance concept learningApplied Intelligence10.1007/s10489-016-0873-046:4(865-875)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s10489-016-0873-0
Zafra APechenizkiy MVentura S(2013)HyDR-MIInformation Sciences: an International Journal10.1016/j.ins.2011.01.034222(282-301)Online publication date: 1-Feb-2013
https://dl.acm.org/doi/10.1016/j.ins.2011.01.034
Zhou ZZhang MHuang SLi Y(2012)Multi-instance multi-label learningArtificial Intelligence10.1016/j.artint.2011.10.002176:1(2291-2320)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.1016/j.artint.2011.10.002
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Improve Multi-Instance Neural Networks through Feature Selection

Multi-instance clustering with applications to multi-instance prediction

Ml-rbf: RBF Neural Networks for Multi-Label Learning

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations