Multi‐instance multi‐label learning of natural scene images: : via sparse coding and multi‐layer neural network
Authors: 
Hu Zhang, Wei Wu, Ding WangAuthors Info & Claims
Abstract
The classification of natural scene images is multi‐instance multi‐label (MIML) for many labels that exist in a natural scene image. The traditional method of solving MIML is to degenerate it into single‐instance single‐label learning (SISL). However, the precision of the method could decrease due to information loss during the degeneration process. How to reasonably solve the MIML problem is key to obtaining high accuracy in this research area. An MIML algorithm based on instances via combining sparse coding with a deep neural network is proposed. First, an instance‐based sparse representation with dictionary learning is adopted. Second, an MIML description model based on a deep network is proposed, which can realise parameter self‐learning in combination with sparse representations. Third, the residuals of the sparse representation are introduced to the deep neural network. The results of the experiments show that the method outperforms a number of state‐of‐the‐art approaches.
7 References
[1]
Zhou, Z., Zhang, M.: ‘Multi‐instance multi‐label learning with application to scene classification’, Adv. Neural Inf. Process. Syst., 2007, 19, pp. 1609–1616
[2]
Zhou, Z.H., Zhang, M.L., Huang, S.J., et al: ‘Multi‐instance multi‐label learning’, Artif. Intell., 2012, 176, (1), pp. 2291–2320
[3]
Dietterich, T.G., Lathrop, R.H., Lozano‐Pérez, T.: ‘Solving the multiple instance problem with axis‐parallel rectangles’, Artif. Intell., 1997, 89, pp. 31–71
[4]
Zhang, M.‐L., Zhang, K.: ‘Multi‐label learning by exploiting label dependency’. Proc. 16th ACM SIGKDD Int. Conf. Knowledge discovery and data mining ‐ KDD'10, 2010, pp. 999–1008
[5]
Maron, O., Ratan, A.L.: ‘Multiple‐instance learning for natural scene classification’. Proc. 15th Int. Conf. Machine Learning, 1998, pp. 341–349
[6]
Zhang, Q., Goldman, S.A.: ‘EM‐DD: an improved multiple‐instance learning technique’, NIPS, 2002, 14, pp. 1073–1080
[7]
Mason, L., Baxter, J., Bartlett, P., et al: ‘Boosting algorithms as gradient descent in function space’, Adv. Neural Inf. Process. Syst., 1999, 12, (April), pp. 512–518
[8]
Wang, J., Zucker, J.‐D.: ‘Solving multiple‐instance problem: a lazy learning approach’. Proc. 17th Int. Conf. Machine Learning, 2000, (1994), pp. 1119–1125
[9]
Xi, I., Bergadano, F.: ‘Learning single and multiple instance decision trees for computer security applications’ (2000, February)
[10]
Blockeel, H., Page, D., Srinivasan, A.: ‘Multi‐instance tree learning’, Proc. 22nd Int. Conf. Machine Learning, 2005, pp. 57–64
[11]
Zhang, M.L., Zhou, Z.H.: ‘Improve multi‐instance neural networks through feature selection’, Neural Process. Lett., 2004, 19, (1), pp. 1–10
[12]
Zhou, Z., Zhang, M.: ‘Neural networks for multi‐instance learning’. Int. Conf. Intelligent Information Technologies, 2002, pp. 455–459
[13]
Zhang, M., Zhou, Z.: ‘Adapting RBF neural networks to multi‐instance learning’, Neural Process. Lett., 2006, 23, (1), pp. 1–26
[14]
Chevaleyre, Y., Zucker, J.‐D.: ‘A framework for learning rules from multiple instance data’, 2003, pp. 49–60
[15]
Andrews, S., Hofmann, T., Tsochantaridis, I.: ‘Multiple instance learning with generalized support vector machines’, AAAI, 2002, 15, (2), pp. 561–568
[16]
Chen, Y., Wang, J.Z.: ‘Image categorization by learning and reasoning with regions’, J. Mach. Learn. Res., 2004, 5, pp. 913–939
[17]
Zhou, Z.: ‘On the relation between multi‐instance learning and semi‐supervised learning’, 2007, (1997)
[18]
Gärtner, T., Flach, P.A., Kowalczyk, A., et al: ‘Multi‐instance kernels’. Proc. Nineteenth Int. Conf. Machine Learning, 2002, (Mi), pp. 179–186
[19]
Cheung, P.‐M., Kwok, J.T.: ‘A regularization framework for multiple‐instance learning’, Proc. 23rd Int. Conf. Mach. Learn. ACM, 2006, 23, (148), pp. 193–200
[20]
Kwok, J.T., Cheung, P.M.: ‘Marginalized multi‐instance kernels’. Int. Joint Conf. Artificial Intelligence (IJCAI), 2007, pp. 901–906
[21]
Fung, G., Dundar, M., Krishnapuram, B., et al: ‘Multiple instance learning for computer aided diagnosis’, 2006
[22]
Zhou, Z.‐H., Zhang, M.: ‘Ensembles of multi‐instance learners’, Lect. Notes Artif. Intell. (Subseries Lect. Notes Comput. Sci.), 2003, 2837, pp. 492–502
[23]
Xu, X., Frank, E.: ‘Logistic regression and boosting for labeled bags of instances’. Proc. PacificAsia Conf. Knowledge Discovery and Data Mining, 2004, pp. 272–281
[24]
Schapire, R.E., Singer, Y.: ‘BoosTexter: a boosting‐based system for text categorization’, Mach. Learn, 2000, 39, (2–3), pp. 135–168
[25]
Chen, Y., Bi, J., Wang, J.Z., et al: ‘MILES: multiple‐instance learning via embedded instance selection’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (12), pp. 1–18
[26]
Boutell, M.R., Luo, J., Shen, X., et al: ‘Learning multi‐label scene classification’, Pattern Recognit., 2004, 37, (9), pp. 1757–1771
[27]
Nguyen, N.: ‘A new SVM approach to multi‐instance multi‐label learning’. Proc. ‐ IEEE Int. Conf. Data Mining (ICDM), 2010, pp. 384–392
[28]
Li, Y.‐X., Ji, S., Kumar, S., et al: ‘Drosophila gene expression pattern annotation through multi‐instance multi‐label learning’, IEEE/ACM Trans. Comput. Biol. Bioinforma., 2012, 9, (1), pp. 98–112
[29]
Elisseeff, A.: ‘Kernel methods for multi‐labelled classification and categorical regression problems’, Adv. Neural Inf. Process., 2001, 2, pp. 681–687
[30]
Godbole, S., Sarawagi, S.: ‘Discriminative methods for multi‐labeled classification’, Lect. Notes Comput. Sci., 2004, 3056, pp. 22–30
[31]
Cesa‐Bianchi, N., Gentile, C., Zaniboni, L.: ‘Hierarchical classification: combining Bayes with SVM’. Proc. 23rd Int. Conf. Machine Learning – ICML ‘06, 2006, pp. 177–184
[32]
Liu, Y., Jin, R., Yang, L.: ‘Semi‐supervised multi‐label learning by constrained non‐negative matrix factorization’. Proc. Natl. Conf. Artificial Intelligence, 2006, vol. 21, (1), p. 421
[33]
Yu, K., Yu, S., Tresp, V.: ‘Multi‐label informed latent semantic indexing’. Proc. 28th Annual Int. ACM, 2005, pp. 258–265
[34]
Zhang, Y., Zhou, Z.‐H.: ‘Multilabel dimensionality reduction via dependence maximization’, ACM Trans. Knowl. Discov. Data, 2010, 4, (3), pp. 1–21
[35]
Zhang, M.L., Zhou, Z.H.: ‘Multi‐instance clustering with applications to multi‐instance prediction’, Appl. Intell., 2009, 31, (1), pp. 47–68
[36]
Zhang, M.: ‘M3MIML: a maximum margin method for multi‐instance multi‐label learning’. IEEE/Int. Conf. Data Mining, 2008, pp. 688–697
[37]
Zhang, M.L.: ‘A k‐nearest neighbor based multi‐instance multi‐label learning algorithm’. Proc. – Int. Conf. Tools with Artificial Intelligence (ICTAI), 2010, vol. 2, pp. 207–212
[38]
Yang, C., Lozano‐Perez, T.: ‘Image database retrieval with multiple‐instance learning techniques’. Proc. 16th Int. Conf. Data Engineering 2000, 2000, pp. 233–243
[39]
Jiang, Y., Zhou, Z. H.: ‘SOM ensemble‐based image segmentation’, Neural Process. Lett., 2004, 20, (3), pp. 171–178
[40]
Kohonen, T.: ‘The self‐organizing map’, Neurocomputing, 1998, 21, (1–3), pp. 1–6
[41]
Zhou, Z.‐H., Zhang, M., Chen, K.‐J.: ‘A novel bag generator for image database retrieval with multi‐instance learning techniques’. Proc. Int. Conf. Tools with Artificial Intelligence, 2003, pp. 565–569
[42]
Aharon, M., Elad, M., Bruckstein, A.: ‘K‐SVD: an algorithm for designing overcomplete dictionaries for sparse representation’, IEEE Trans. Signal Process., 2006, 54, (11), pp. 4311–4322
[43]
Sajjad, M., Mehmood, I., Baik, S.W.: ‘Sparse coded image super‐resolution using K‐SVD trained dictionary based on regularized orthogonal matching pursuit’, Biomed. Mater. Eng., 2015, 26, (September), pp. S1399–S1407
[44]
Song, X., Liu, Z., Yang, X., et al: ‘A parameterized fuzzy adaptive K‐SVD approach for the multi‐classes study of pursuit algorithms’, Neurocomputing, 2014, 123, pp. 131–139
[45]
Wang, S., Jiao, L., Yang, S., et al: ‘SAR image target recognition via complementary spatial pyramid coding’, Neurocomputing, 2016, 196, pp. 125–132
[46]
Chen, Z., Chi, Z., Fu, H., et al: ‘Multi‐instance multi‐label image classification: a neural approach’, Neurocomputing, 2013, 99, pp. 298–306
[47]
Wu, J.S., Huang, S.J., Zhou, Z.H.: ‘Genome‐wide protein function prediction through multi‐instance multi‐label learning’, IEEE/ACM Trans. Comput. Biol. Bioinform., 2014, 11, (5), pp. 891–902
[48]
Zhang, M., Wang, Z.: ‘MimlRbf: RBF neural networks for multi‐instance multi‐label learning’, Neurocomputing., 2009, 72, (16), pp. 3951–3956
[49]
Glorot, X., Bengio, Y.: ‘Understanding the difficulty of training deep feedforward neural networks’. Proc. 13th Int. Conf. Artificial Intelligence and Statistics, 2010, vol. 9, pp. 249–256
[50]
Salton, G.: ‘Automatic text processing: the transformation’, Anal. Retr. Inf. Comput., 1989, 14, p. 15
[51]
Yang, Y., Pedersen, J.O.: ‘A comparative study on feature selection in text categorization’. Proc. 14th Int. Conf. Machine Learning, 1997, pp. 412–420
[52]
Lahrache, S., El Qadi, A., El Ouazzani, R.: ‘Bag‐of‐features for image memorability evaluation’, IET Comput. Vis., 2016, 10, (6), pp. 577–584
[53]
Triguero, I., Vens, C.: ‘Labelling strategies for hierarchical multi‐label classification techniques’, Pattern Recognit., 2015, 56, pp. 170–183
Recommendations
Diversified dictionaries for multi-instance learning
Multiple-instance learning (MIL) has been a popular topic in the study of pattern recognition for years due to its usefulness for such tasks as drug activity prediction and image/text classification. In a typical MIL setting, a bag contains a bag-level ...
Generalized Dictionaries for Multiple Instance Learning
We present a multi-class multiple instance learning (MIL) algorithm using the dictionary learning framework where the data is given in the form of bags. Each bag contains multiple samples, called instances, out of which at least one belongs to the class ...
Multi-instance multi-label learning
In this paper, we propose the MIML (Multi-Instance Multi-Label learning) framework where an example is described by multiple instances and associated with multiple class labels. Compared to traditional learning frameworks, the MIML framework is more ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2018 The Institution of Engineering and Technology.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 18 December 2017
Author Tags
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Jan 2025