Abstract
The problem of deep long-tailed learning, a prevalent challenge in the realm of generic visual recognition, persists in a multitude of real-world applications. To tackle the heavily-skewed dataset issue in long-tailed classification, prior efforts have sought to augment existing deep models with the elaborate class-balancing strategies, such as class rebalancing, data augmentation, and module improvement. Despite the encouraging performance, the limited class knowledge of the tailed classes in the training dataset still bottlenecks the performance of the existing deep models. In this paper, we propose an innovative long-tailed learning paradigm that breaks the bottleneck by guiding the learning of deep networks with external prior knowledge. This is specifically achieved by devising an elaborated “prophetic” teacher, termed as “Propheter”, that aims to learn the potential class distributions. The target long-tailed prediction model is then optimized under the instruction of the well-trained “Propheter”, such that the distributions of different classes are as distinguishable as possible from each other. Experiments on eight long-tailed benchmarks across three architectures demonstrate that the proposed prophetic paradigm acts as a promising solution to the challenge of limited class knowledge in long-tailed datasets. The developed code is publicly available at https://github.com/tcmyxc/propheter.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cao, Y., Long, M., Wang, J., Zhu, H., Wen, Q.: Deep quantization network for efficient image retrieval. In: AAAI (2016)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique.J. Artif. Intell. Res. 16, 321–357 (2002)
Chu, P., Bian, X., Liu, S., Ling, H.: Feature space augmentation for long-tailed data. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12374, pp. 694–710. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58526-6_41
Cui, Y., Jia, M., Lin, T.Y., Song, Y., Belongie, S.: Class-balanced loss based on effective number of samples. In: CVPR (2019)
Feng, C., Zhong, Y., Huang, W.: Exploring classification equilibrium in long-tailed object detection. In: ICCV (2021)
Feng, M., et al.: Exploring hierarchical spatial layout cues for 3D point cloud based scene graph prediction. IEEE Trans. Multimedia 99, 1–13 (2023)
Feng, Z., Jing, Y., Zhang, C., Xu, R., Lei, J., Song, M.: Graph-based color gamut mapping using neighbor metric. In: ICME (2017)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NIPS Deep Learning and Representation Learning Workshop (2015)
Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Learning a unified classifier incrementally via rebalancing. In: CVPR (2019)
Iscen, A., Araujo, A., Gong, B., Schmid, C.: Class-balanced distillation for long-tailed visual recognition. In: BMVC (2021)
Jamal, M.A., Brown, M., Yang, M.H., Wang, L., Gong, B.: Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective. In: CVPR (2020)
Jing, Y., Yuan, C., Ju, L., Yang, Y., Wang, X., Tao, D.: Deep graph reprogramming. In: CVPR (2023)
Kang, B., et al.: Decoupling representation and classifier for long-tailed recognition. In: ICLR (2019)
Liang, H., et al.: Training interpretable convolutional neural networks by differentiating class-specific filters. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 622–638. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_37
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: ICCV (2017)
Liu, S., Garrepalli, R., Dietterich, T.G., Fern, A., Hendrycks, D.: Open category detection with PAC guarantees. In: ICML (2018)
Liu, S., Wang, K., Yang, X., Ye, J., Wang, X.: Dataset distillation via factorization. NeurIPS (2022)
Liu, S., Ye, J., Yu, R., Wang, X.: Slimmable dataset condensation. In: CVPR (2023)
Luo, B., et al.: Learning deep hierarchical features with spatial regularization for one-class facial expression recognition. In: AAAI (2023)
Mengke Li, Yiu-ming Cheung, Y.L.: Long-tailed visual recognition via gaussian clouded logit adjustment. In: CVPR, pp. 6929–6938 (2022)
Ren, J., et al.: Balanced meta-softmax for long-tailed visual recognition. In: NeurIPS (2020)
Su, X., et al.: Prioritized architecture sampling with monto-carlo tree search. In: CVPR (2021)
Su, X., et al.: Locally free weight sharing for network width search. arXiv preprint arXiv:2102.05258 (2021)
Su, X., You, S., Wang, F., Qian, C., Zhang, C., Xu, C.: BCNet: searching for network width with bilaterally coupled network. In: CVPR (2021)
Su, X., et al.: ViTAS: vision transformer architecture search. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol. 13681. Springer, Cham. https://doi.org/10.1007/978-3-031-19803-8_9
Xi, H.: Data-driven optimization technologies for MaaS. In: Big Data and Mobility as a Service (2022)
Xi, H., Liu, W., Waller, S.T., Hensher, D.A., Kilby, P., Rey, D.: Incentive-compatible mechanisms for online resource allocation in mobility-as-a-service systems. Trans. Res. Part B Methodol. 170, 119-147 (2023)
Xi, H., Tang, Y., Waller, S.T., Shalaby, A.: Modeling, equilibrium, and demand management for mobility and delivery services in mobility-as-a-service ecosystems. Comput-Aided Civ. Infrastruct. Eng. 38(11), 1403–1423 (2023)
Xi, H., Zhang, Y., Zhang, Y.: Detection of safety features of drivers based on image processing. In: 18th COTA International Conference of Transportation Professionals (2018)
Yang, X., Ye, J., Wang, X.: Factorizing knowledge in neural networks. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol. 13694. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19830-4_5
Yang, X., Zhou, D., Feng, J., Wang, X.: Diffusion probabilistic model made slim. In: CVPR (2023)
Yang, X., Zhou, D., Liu, S., Ye, J., Wang, X.: Deep model reassembly. NeurIPS (2022)
Yu, R., Liu, S., Wang, X.: Dataset distillation: a comprehensive review. arXiv preprint arXiv:2301.07014 (2023)
Zhai, W., Cao, Y., Zhang, J., Zha, Z.J.: Exploring figure-ground assignment mechanism in perceptual organization. NeurIPS (2022)
Zhai, W., Luo, H., Zhang, J., Cao, Y., Tao, D.: One-shot object affordance detection in the wild. Int. J. Comput. Vis. 130, 2472–2500 (2022). https://doi.org/10.1007/s11263-022-01642-4
Zhao, H., Bian, W., Yuan, B., Tao, D.: Collaborative learning of depth estimation, visual odometry and camera relocalization from monocular videos. In: IJCAI (2020)
Zhao, H., Zhang, J., Zhang, S., Tao, D.: JPerceiver: joint perception network for depth, pose and layout estimation in driving scenes. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol. 13698. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19839-7_41
Zhao, H., Zhang, Q., Zhao, S., Zhang, J., Tao, D.: BEVSimDet: simulated multi-modal distillation in bird’s-eye view for multi-view 3D object detection. arXiv preprint arXiv:2303.16818 (2023)
Zhu, J., Luo, B., Yang, T., Wang, Z., Zhao, X., Gao, Y.: Knowledge conditioned variational learning for one-class facial expression recognition. IEEE Trans. Image Process. 32, 4010–4023 (2023)
Acknowledgements
This work is funded by National Key Research and Development Project (Grant No: 2022YFB2703100), National Natural Science Foundation of China (61976186, U20B2066), Zhejiang Province High-Level Talents Special Support Program “Leading Talent of Technological Innovation of Ten-Thousands Talents Program” (No. 2022R52046), Ningbo Natural Science Foundation (2022J182), Basic Public Welfare Research Project of Zhejiang Province (LGF21F020020), and the Fundamental Research Funds for the Central Universities (2021FZZX001-23, 226-2023-00048). This work is partially supported by the National Natural Science Foundation of China (Grant No. 62106235), the Exploratory Research Project of Zhejiang Lab (2022PG0AN01), and the Zhejiang Provincial Natural Science Foundation of China (LQ21F020003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xu, W. et al. (2024). Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learning. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14450. Springer, Singapore. https://doi.org/10.1007/978-981-99-8070-3_17
Download citation
DOI: https://doi.org/10.1007/978-981-99-8070-3_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8069-7
Online ISBN: 978-981-99-8070-3
eBook Packages: Computer ScienceComputer Science (R0)