More Web Proxy on the site http://driver.im/

research-article

Tri-Clustered Tensor Completion for Social-Aware Image Tag Refinement

Authors:

Ramesh JainAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 39, Issue 8

Pages 1662 - 1674

https://doi.org/10.1109/TPAMI.2016.2608882

Published: 01 August 2017 Publication History

Abstract

Social image tag refinement, which aims to improve tag quality by automatically completing the missing tags and rectifying the noise-corrupted ones, is an essential component for social image search. Conventional approaches mainly focus on exploring the visual and tag information, without considering the user information, which often reveals important hints on the (in)correct tags of social images. Towards this end, we propose a novel tri-clustered tensor completion framework to collaboratively explore these three kinds of information to improve the performance of social image tag refinement. Specifically, the inter-relations among users, images and tags are modeled by a tensor, and the intra-relations between users, images and tags are explored by three regularizations respectively. To address the challenges of the super-sparse and large-scale tensor factorization that demands expensive computing and memory cost, we propose a novel tri-clustering method to divide the tensor into a certain number of sub-tensors by simultaneously clustering users, images and tags into a bunch of tri-clusters. And then we investigate two strategies to complete these sub-tensors by considering (in)dependence between the sub-tensors. Experimental results on a real-world social image database demonstrate the superiority of the proposed method compared with the state-of-the-art methods.

References

[1]

J. Z. Wang, D. Geman, J. Luo, and R. M. Gray, “Real-world image annotation and retrieval: An introduction to the special section,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1873–1876, Nov. 2008.

Digital Library

[2]

C. Wang, F. Jing, L. Zhang, and H.-J. Zhang, “Content-based image annotation refinement,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1–8.

[3]

H. Xu, J. Wang, X.-S. Hua, and S. Li, “Tag refinement by regularized LDA,” in Proc. 17th ACM Int. Conf. Multimedia, 2009, pp. 573– 576.

Digital Library

[4]

G. Zhu, S. Yan, and Y. Ma, “Image tag refinement towards low-rank, content-tag prior and error sparsity,” in Proc. 18th ACM Int. Conf. Multimedia , 2010, pp. 461–470.

Digital Library

[5]

X. Li, C. G. Snoek, and M. Worring, “Learning social tag relevance by neighbor voting,” IEEE Trans. Multimedia, vol. 11, no. 7, pp. 1310–1322, Nov. 2009.

Digital Library

[6]

G.-J. Qi, C. Aggarwal, J. Han, and T. Huang, “Mining collective intelligence in diverse groups,” in Proc. 22nd Int. Conf. World Wide Web, 2012, pp. 1041–1052.

[7]

Z.-H. Deng, H. Yu, and Y. Yang, “Image tagging via cross-modal semantic mapping,” in Proc. 23rd ACM Int. Conf. Multimedia, 2015, pp. 1143–1146.

Digital Library

[8]

L. Wu, R. Jin, and A. K. Jain, “Tag completion for image retrieval,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 3, pp. 716–727, Mar. 2013.

Digital Library

[9]

Z. Lin, G. Ding, M. Hu, J. Wang, and X. Ye, “Image tag completion via image-specific and tag-specific linear sparse reconstructions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 1618–1625.

[10]

X. Li, Y.-J. Zhang, B. Shen, and B.-D. Liu, “Image tag completion by low-rank factorization with dual reconstruction structure preserved,” in Proc. IEEE Int. Conf. Image Process., 2014, pp. 3062–3066.

[11]

L. Chen, D. Xu, I. W. Tsang, and J. Luo, “Tag-based web photo retrieval improved by batch mode re-tagging,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 3440–3446.

[12]

N. Zhou, W. K. Cheung, G. Qiu, and X. Xue, “A hybrid probabilistic model for unified collaborative and content-based image tagging,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1281– 1294, Jul. 2011.

Digital Library

[13]

D. Liu, X.-S. Hua, M. Wang, and H.-J. Zhang, “Image retagging,” in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 491–500.

Digital Library

[14]

J. Tang, R. Hong, S. Yan, T.-S. Chua, G.-J. Qi, and R. Jain, “ Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images,” ACM Trans. Intell. Syst. Technol., vol. 2, no. 2, 2011, Art. no.

[15]

Y. Jin, L. Khan, L. Wang, and M. Awad, “Image annotations by combining multiple evidence & wordNet,” in Proc. 13th Annu. ACM Int. Conf. Multimedia, 2005, pp. 706–715.

Digital Library

[16]

C. Wang, F. Jing, L. Zhang, and H.-J. Zhang, “Image annotation refinement using random walk with restarts,” in Proc. 14th ACM Int. Conf. Multimedia, 2006, pp. 647–650.

Digital Library

[17]

D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang, “Tag ranking,” in Proc. 18th Int. Conf. World Wide Web, 2009, pp. 351–360.

Digital Library

[18]

J. Liu, et al., “Dual cross-media relevance model for image annotation,” in Proc. 15th ACM Int. Conf. Multimedia, 2007, pp. 605 –614.

Digital Library

[19]

Z. Li, J. Liu, J. Tang, and H. Lu, “Robust structured subspace learning for data representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 10, pp. 2085–2098, Oct. 2015.

Digital Library

[20]

E. J. Candes and Y. Plan, “Matrix completion with noise,” Proc. IEEE , vol. 98, no. 6, pp. 925–936, Jun. 2010.

[21]

Z. Li, J. Liu, X. Zhu, T. Liu, and H. Lu, “Image annotation using multi-correlation probabilistic matrix factorization,” in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 1187– 1190.

Digital Library

[22]

X. Liu, S. Yan, T.-S. Chua, and H. Jin, “Image label completion by pursuing contextual decomposability,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 8, no. 2, 2012, Art. no.

[23]

J. Zhuang and S. C. Hoi, “A two-view learning approach for image tag ranking,” in Proc. 4th ACM Int. Conf. Web Search Data Mining, 2011, pp. 625–634.

[24]

P. Cui, S.-W. Liu, W.-W. Zhu, H.-B. Luan, T.-S. Chua, and S.-Q. Yang, “ Social-sensed image search,” ACM Trans. Inf. Syst., vol. 32, no. 2, 2014, Art. no.

[25]

G.-J. Qi, C. C. Aggarwal, Q. Tian, H. Ji, and T. S. Huang, “Exploring context and content links in social media: A latent space method,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 5, pp. 850–862, May 2012.

Digital Library

[26]

J. Sang, J. Liu, and C. Xu, “Exploiting user information for image tag refinement,” in Proc. 19th ACM Int. Conf. Multimedia, 2011, pp. 1129–1132.

Digital Library

[27]

J. Sang, C. Xu, and J. Liu, “User-aware image tag refinement via ternary semantic analysis,” IEEE Trans. Multimedia, vol. 14, no. 3, pp. 883–895, Jun. 2012.

Digital Library

[28]

P. E. Crandall and M. J. Quinn, “Block data decomposition for data-parallel programming on a heterogeneous workstation network,” in Proc. 2nd Int. Symp. High Performance Distrib. Comput., 1993, pp. 42–49.

[29]

R. M. Czekster, C. A. De Rose, P. Fernandes, A. M. de Lima, and T. Webber, “Kronecker descriptor partitioning for parallel algorithms,” in Proc. Spring Simul. Multiconference, 2010, Art. no.

[30]

A. Benoit, B. Plateau, and W. J. Stewart, “ Memory-efficient Kronecker algorithms with applications to the modelling of parallel systems,” Future Generation Comput. Syst., vol. 22, no. 7, pp. 838 –847, 2006.

Digital Library

[31]

J. Tang, Z. Li, M. Wang, and R. Zhao, “Neighborhood discriminant hashing for large-scale image retrieval,” IEEE Trans. Image Process., vol. 24, no. 9, pp. 2827–2840, Sep. 2015.

Digital Library

[32]

E. Papalexakis, C. Faloutsos, and N. Sidiropoulos, “ ParCube: Sparse parallelizable tensor decompositions,” in Proc. Eur. Conf. Mach. Learn. Knowl. Discovery Databases, 2012, pp. 521–536.

[33]

I. S. Dhillon, “Co-clustering documents and words using bipartite spectral graph partitioning,” in Proc. 7th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2001, pp. 269–274.

[34]

X. He, D. Cai, H. Liu, and J. Han, “Image clustering with tensor representation,” in Proc. 13th Annu. ACM Int. Conf. Multimedia, 2005, pp. 132–140.

Digital Library

[35]

L. R. Tucker, “Some mathematical notes on three-mode factor analysis,” Psychometrika, vol. 31, no. 3, pp. 279 –311, 1966.

[36]

L. Wu, X.-S. Hua, N. Yu, W.-Y. Ma, and S. Li, “Flickr distance: A relationship measure for visual concepts,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 5, pp. 863–875, May 2012.

Digital Library

[37]

P. Resnik, “Using information content to evaluate semantic similarity in a taxonomy,” in Proc. 14th Int. Joint Conf. Artif. Intell.—Vol. 1, 1995, pp. 448–453.

[38]

D. Lin, “Using syntactic dependency as local context to resolve word sense ambiguity,” in Proc. 35th Annu. Meeting Assoc. Comput. Linguistics, 1997, pp. 64–71.

[39]

L. Zhao and M. J. Zaki, “TRICLUSTER: An effective algorithm for mining coherent clusters in 3D microarray data,” in Proc. ACM SIGMOD Int. Conf. Manage. Data, 2005, pp. 694–705.

[40]

Q. Zhou, G. Xu, and Y. Zong, “Web co-clustering of usage network using tensor decomposition,” in Proc. IEEE/WIC/ACM Int. Joint Conf. Web Intell. Intell. Agent Technol., 2009, pp. 311 –314.

[41]

E. J. Candès and B. Recht, “Exact matrix completion via convex optimization,” Found. Comput. Math., vol. 9, no. 6, pp. 717–772, 2009.

[42]

Z. Li, J. Liu, Y. Yang, X. Zhou, and H. Lu, “Clustering-guided sparse structural learning for unsupervised feature selection,” IEEE Trans. Knowl. Data Eng., vol. 26, no. 9, pp. 2138–2150, Sep. 2014.

[43]

T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, “ NUS-WIDE: A real-world web image database from National University of Singapore,” in Proc. ACM Int. Conf. Image Video Retrieval, 2009, Art. no.

Digital Library

Cited By

Zhang ZChang DZhu RLi XMa ZXue J(2025)Query-Aware Cross-Mixup and Cross-Reconstruction for Few-Shot Fine-Grained Image ClassificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.348453035:2(1276-1286)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1109/TCSVT.2024.3484530
Shi HLi LXiao JZhuang YChen L(2025)From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph GenerationInternational Journal of Computer Vision10.1007/s11263-024-02190-9133:1(489-508)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11263-024-02190-9
Li FXu QBao SYang ZCong RCao XHuang QSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Size-invariance mattersProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693236(28989-29021)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693236
Show More Cited By

Index Terms

Tri-Clustered Tensor Completion for Social-Aware Image Tag Refinement
1. Computing methodologies
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Folded-concave penalization approaches to tensor completion

The existing studies involving matrix or tensor completion problems are commonly under the nuclear norm penalization framework due to the computational efficiency of the resulting convex optimization problem. Folded-concave penalization methods have ...
Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train

This paper proposes a novel approach to tensor completion, which recovers missing entries of data represented by tensors. The approach is based on the tensor train (TT) rank, which is able to capture hidden information from tensors thanks to its ...
Automatic Abstract Tag Detection for Social Image Tag Refinement and Enrichment

Collaborative image tagging systems, such as Flickr, are very attractive for supporting keyword-based image retrieval, but some user-provided tags of collaboratively-tagged social images might be imprecise. Some people may use general or high-level ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 39, Issue 8

Aug. 2017

208 pages

ISSN:0162-8828

Issue’s Table of Contents

0162-8828 © 2016 IEEE.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 August 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZChang DZhu RLi XMa ZXue J(2025)Query-Aware Cross-Mixup and Cross-Reconstruction for Few-Shot Fine-Grained Image ClassificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.348453035:2(1276-1286)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1109/TCSVT.2024.3484530
Shi HLi LXiao JZhuang YChen L(2025)From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph GenerationInternational Journal of Computer Vision10.1007/s11263-024-02190-9133:1(489-508)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11263-024-02190-9
Li FXu QBao SYang ZCong RCao XHuang QSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Size-invariance mattersProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693236(28989-29021)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693236
Hua CXu QBao SYang ZHuang QSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)ReconBoostProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692859(19573-19597)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692859
Fu PLiang XLuo TGuo QZhang YQian YLarson K(2024)Core-structures-guided multi-modal classification neural architecture searchProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/440(3980-3988)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/440
Guo QLiang XQian YCui ZWen JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)A Progressive Skip Reasoning Fusion Method for Multi-Modal ClassificationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681437(429-437)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681437
Fu PLiang XQian YGuo QWei ZLi WCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)CoMO-NAS: Core-Structures-Guided Multi-Objective Neural Architecture Search for Multi-Modal ClassificationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681351(9126-9135)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681351
Hao XJiang XNi WTan WYan B(2024)Prompt-Guided Semantic-Aware Distillation for Weakly Supervised Incremental Semantic SegmentationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.341299634:11_Part_1(10632-10645)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3412996
Cui XLi ZLi PHuang HLiu XHe Z(2024)INSTASTYLE: Inversion Noise of a Stylized Image is Secretly a Style AdviserComputer Vision – ECCV 202410.1007/978-3-031-72983-6_26(455-472)Online publication date: 29-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72983-6_26
Jang JKang U(2023)Static and Streaming Tucker Decomposition for Dense TensorsACM Transactions on Knowledge Discovery from Data10.1145/356868217:5(1-34)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3568682
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents