Abstract
Data-intensive applications have achieved great success in the field of machine learning. How to ensure that the machine can still learn correctly in the absence of labeled samples is the next challenging problem to be solved. This paper first introduces the problem definition of few-shot learning. Secondly, the existing small few-shot learning methods based on meta-learning are comprehensively summarized. Specifically, they are divided into three categories: metric-based learning methods, optimization-based learning methods and model-based learning methods. We conducted a series of comparisons among various methods in each category to show the advantages and disadvantages of each method. Finally, the limitations of existing methods are analyzed, and the future development direction of few-shot learning research is prospected.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Andrychowicz, M., et al.: Learning to learn by gradient descent by gradient descent. arXiv:1606.04474 [cs], November 2016
Baltrušaitis, T., Ahuja, C., Morency, L.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 423–443 (2019). https://doi.org/10.1109/TPAMI.2018.2798607
Caruana, R.: Multitask Learning. Mach. Learn. 28(1), 41–75 (1997). https://doi.org/10.1023/A:1007379606734
Chen, Y., Wang, X., Liu, Z., Xu, H., Darrell, T.: A new meta-baseline for few-shot learning. arXiv:2003.04390 [cs], April 2020
Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A baseline for few-shot image classification. arXiv:1909.02729 [cs, stat], October 2020
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR, July 2017
Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv:1410.5401 [cs], December 2014
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR, June 2015
Jamal, M.A., Qi, G.J.: Task agnostic meta-learning for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11719–11727 (2019)
Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, MM 2014, pp. 675–678. Association for Computing Machinery, New York, November 2014. https://doi.org/10.1145/2647868.2654889
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 [cs], January 2017
Koch, G.: Siamese neural networks for one-shot image recognition, p. 30
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386
Lake, B.M., Salakhutdinov, R., Gross, J., Tenenbaum, J.B.: One shot learning of simple visual concepts, p. 7
Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350(6266), 1332–1338 (2015). https://doi.org/10.1126/science.aab3050
Munkhdalai, T., Yu, H.: Meta networks. In: International Conference on Machine Learning, pp. 2554–2563. PMLR, July 2017
Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. arXiv:1803.02999 [cs], October 2018
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). https://doi.org/10.1109/TKDE.2009.191
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning, p. 11 (2017)
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp. 1842–1850. PMLR, June 2016
Schaul, T., Schmidhuber, J.: Metalearning. Scholarpedia 5(6), 4650 (2010). https://doi.org/10.4249/scholarpedia.4650
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Settles, B.: Active learning. In: Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 6, no. 1, pp. 1–114, June 2012. https://doi.org/10.2200/S00429ED1V01Y201207AIM018
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. arXiv:1703.05175 [cs, stat], June 2017
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208 (2018)
Tseng, H.Y., Lee, H.Y., Huang, J.B., Yang, M.H.: Cross-domain few-shot classification via learned feature-wise transformation. arXiv:2001.08735 [cs], March 2020
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, vol. 29, pp. 3630–3638 (2016)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv:1212.5701 [cs], December 2012
Zhou, Z.H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5(1), 44–53 (2018). https://doi.org/10.1093/nsr/nwx106
Zhu, X., Goldberg, A.B.: Introduction to semi-supervised learning. In: Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 3, no. 1, pp. 1–130, January 2009. https://doi.org/10.2200/S00196ED1V01Y200906AIM006
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Huang, W., He, M., Wang, Y. (2022). A Survey on Meta-learning Based Few-Shot Classification. In: Jiang, X. (eds) Machine Learning and Intelligent Communications. MLICOM 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 438. Springer, Cham. https://doi.org/10.1007/978-3-031-04409-0_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-04409-0_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04408-3
Online ISBN: 978-3-031-04409-0
eBook Packages: Computer ScienceComputer Science (R0)