Abstract
Recently, representation learning with contrastive learning algorithms has been successfully applied to challenging unlabeled datasets. However, these methods are unable to distinguish important features from unimportant ones under simply unsupervised settings, and definitions of importance vary according to the type of downstream task or analysis goal, such as the identification of objects or backgrounds. In this paper, we focus on unsupervised image clustering as the downstream task and propose a representation learning method that enhances features critical to the clustering task. We extend a clustering-friendly contrastive learning method and incorporate a contrastive analysis approach, which utilizes a reference dataset to separate important features from unimportant ones, into the design of loss functions. Conducting an experimental evaluation of image clustering for three datasets with characteristic backgrounds, we show that for all datasets, our method achieves higher clustering scores compared with conventional contrastive analysis and deep clustering methods.
T. Oshima and K. Takagi—First two authors have equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abid, A., Zhang, M.J., Bagaria, V.K., Zou, J.: Exploring patterns enriched in a dataset with contrastive principal component analysis. Nat. Commun. 9, 2134 (2018)
Abid, A., Zou, J.: Contrastive variational autoencoder enhances salient features. arXiv:1902.04601 (2019)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: NeurIPS, pp. 9912–9924. Curran Associates, Inc. (2020)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: ICML, pp. 1597–1607. PMLR (2020)
Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR, pp. 15750–15758. IEEE (2021)
Dang, Z., Deng, C., Yang, X., Wei, K., Huang, H.: Nearest neighbor matching for deep clustering. In: CVPR, pp. 13693–13702 (2021)
Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Process. Mag. 29(6), 141–142 (2012)
Dirie, A.H., Abid, A., Zou, J.: Contrastive multivariate singular spectrum analysis. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1122–1127 (2019)
Ge, R., Zou, J.: Rich component analysis. In: ICML, pp. 1502–1510. PMLR (2016)
Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J.T., Peng, X.: Contrastive clustering. In: AAAI, pp. 8547–8555. AAAI Press (2021)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)
McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426 (2018)
Niu, C., Shan, H., Wang, G.: SPICE: semantic pseudo-labeling for image clustering. arXiv:2103.09382 (2022)
Park, S., et al.: Improving unsupervised image clustering with robust learning. In: CVPR (2021)
Piosenka, G.: Birds 400 - species image classification (2022)
Rougetet, A.: Landscape pictures (2020)
Shota, M., Yukako, T.: Application of contrastive representation learning to unsupervised defect classification in semiconductor manufacturing. In: AEC/APC Symposium Asia 2021 (2021)
Tao, Y., Takagi, K., Nakata, K.: Clustering-friendly representation learning via instance discrimination and feature decorrelation. In: ICLR (2021)
Tsai, T.W., Li, C., Zhu, J.: Mice: mixture of contrastive experts for unsupervised image clustering. In: ICLR (2021)
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: SCAN: learning to classify images without labels. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12355, pp. 268–285. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58607-2_16
Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: CVPR, pp. 3733–3742. IEEE (2018)
Li, Y., Yang, M., Peng, D., Li, T., Huang, J., Peng, X.: Twin contrastive learning for online clustering. Int. J. Comput. Vision 130(9), 2205–2221 (2022)
Zhou, S., et al.: A comprehensive survey on deep clustering: taxonomy, challenges, and future directions (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Oshima, T., Takagi, K., Nakata, K. (2024). Clustering-Friendly Representation Learning for Enhancing Salient Features. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14645. Springer, Singapore. https://doi.org/10.1007/978-981-97-2242-6_17
Download citation
DOI: https://doi.org/10.1007/978-981-97-2242-6_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2241-9
Online ISBN: 978-981-97-2242-6
eBook Packages: Computer ScienceComputer Science (R0)