More Web Proxy on the site http://driver.im/

research-article

Social event detection using multimodal clustering and integrating supervisory signals

Authors:

Georgios Petkos,

Symeon Papadopoulos,

Yiannis KompatsiarisAuthors Info & Claims

ICMR '12: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval

Article No.: 23, Pages 1 - 8

https://doi.org/10.1145/2324796.2324825

Published: 05 June 2012 Publication History

Abstract

A large variety of features can be extracted from raw multimedia items. Moreover, in many contexts, like in the case of multimedia uploaded by users of social media platforms, items may be linked to metadata that can be very useful for a variety of analysis tasks. Nevertheless, such features are typically heterogeneous and are difficult to combine in a unified representation that would be suitable for analysis. In this paper, we discuss the problem of clustering collections of multimedia items with the purpose of detecting social events. In order to achieve this, a novel multimodal clustering algorithm is proposed. The proposed method uses a known clustering in the currently examined domain, in order to supervise the multimodal fusion and clustering procedure. It is tested on the MediaEval social event detection challenge data and is compared to a multimodal spectral clustering approach that uses early fusion. By taking advantage of the explicit supervisory signal, it achieves superior clustering accuracy and additionally requires the specification of a much smaller number of parameters. Moreover, the proposed approach has wider scope; it is not only applicable to the task of social event detection, but to other multimodal clustering problems as well.

References

[1]

David Arthur and Sergei Vassilvitskii. k-means++: the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, SODA '07, pages 1027--1035, Philadelphia, PA, USA, 2007. Society for Industrial and Applied Mathematics.

Digital Library

[2]

Ron Bekkerman and Jiwoon Jeon. Multi-modal clustering for multimedia collections. In CVPR. IEEE Computer Society, 2007.

[3]

Markus Brenner and Ebroul Izquierdo. Mediaeval benchmark: Social event detection in collaborative photo collections. In Larson et al. {8}.

[4]

Xiao Cai, Feiping Nie, Heng Huang, and F. Kamangar. Heterogeneous image feature integration via multi-modal spectral clustering. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1977--1984, June 2011.

Digital Library

[5]

Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011.

Digital Library

[6]

Vasil Khalidov, Florence Forbes, and Radu P. Horaud. Conjugate mixture models for clustering multimodal data. Neural Computation, 23(2):517--557, February 2011.

Digital Library

[7]

Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Trans. Knowl. Discov. Data, 3:1:1--1:58, March 2009.

Digital Library

[8]

Martha Larson, Adam Rae, Claire-Hélène Demarty, Christoph Kofler, Florian Metze, Raphaël Troncy, Vasileios Mezaris, and Gareth J. F. Jones, editors. Working Notes Proceedings of the MediaEval 2011 Workshop, Santa Croce in Fossabanda, Pisa, Italy, September 1--2, 2011, volume 807 of CEUR Workshop Proceedings. CEUR-WS.org, 2011.

[9]

Xueliang Liu, Benoit Huet, and Raphaël Troncy. Eurecom @ mediaeval 2011 social event detection task. In Larson et al. {8}.

[10]

Xueliang Liu, Raphael Troncy, and Benoit Huet. Finding media illustrating events. In ICMR'11, 1st ACM International Conference on Multimedia Retrieval, April 17--20, 2011, Trento, Italy, 04 2011.

Digital Library

[11]

Xueliang Liu, Raphael Troncy, and Benoit Huet. Using social media to identify events. In WSM'11, ACM Multimedia 3rd Workshop on Social Media, November 18--December 1st, 2011, Scottsdale, Arizona, USA, 11 2011.

Digital Library

[12]

David G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60:91--110, November 2004.

Digital Library

[13]

Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems (NIPS), pages 849--856. MIT Press, 2001.

Digital Library

[14]

S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, and A. Vakali. Cluster-based landmark and event detection for tagged photo collections. Multimedia, IEEE, 18(1):52--63, jan. 2011.

Digital Library

[15]

Symeon Papadopoulos, Raphaël Troncy, Vasileios Mezaris, Benoit Huet, and Ioannis Kompatsiaris. Social event detection at mediaeval 2011: Challenges, dataset and evaluation. In Larson et al. {8}.

[16]

Symeon Papadopoulos, Christos Zigkolis, Yiannis Kompatsiaris, and Athena Vakali. Certh @ mediaeval 2011 social event detection task. In Larson et al. {8}.

[17]

Massimiliano Ruocco and Heri Ramampiaro. Ntnu@mediaeval 2011 social event detection task. In Larson et al. {8}.

[18]

T. Sakaki, M. Okazaki, and Y. Matsuo. Earthquake shakes twitter users: real-time event detection by social sensors. In Proceedings of the 19th international conference on World wide web, pages 851--860. ACM, 2010.

Digital Library

[19]

Hassan Sayyadi, Matthew Hurst, and Alexey Maykov. Event detection and tracking in social streams. In Eytan Adar, Matthew Hurst, Tim Finin, Natalie S. Glance, Nicolas Nicolov, and Belle L. Tseng, editors, ICWSM. The AAAI Press, 2009.

[20]

Cees G. M. Snoek, Marcel Worring, and Arnold W. M. Smeulders. Early versus late fusion in semantic video analysis. In Proceedings of the 13th annual ACM international conference on Multimedia, MULTIMEDIA '05, pages 399--402, New York, NY, USA, 2005. ACM.

Digital Library

[21]

Yanxiang Wang, Lexing Xie, and Hari Sundaram. Social event detection with clustering and filtering. In Larson et al. {8}.

[22]

Jianshu Weng and Bu-Sung Lee. Event detection in twitter. In Proceedings of the Fifth International Conference on Weblogs and Social Media, Barcelona, Catalonia, Spain. The AAAI Press, 2011.

Cited By

Kovalchuk MFilatova AKorneev AKoreneva MNasonov DVoskresenskii ABoukhanovsky A(2024)SemConvTree: Semantic Convolutional Quadtrees for Multi-Scale Event Detection in Smart CitySmart Cities10.3390/smartcities70501077:5(2763-2780)Online publication date: 28-Sep-2024
https://doi.org/10.3390/smartcities7050107
He ZWan SZappatore MLu H(2024)A Similarity Matrix Low-Rank Approximation and Inconsistency Separation Fusion Approach for Multiview ClusteringIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.32719645:2(868-881)Online publication date: Feb-2024
https://doi.org/10.1109/TAI.2023.3271964
Mondal MKhayati MSandlin HCudré-Mauroux P(2024)A survey of multimodal event detection based on data fusionThe VLDB Journal10.1007/s00778-024-00878-534:1Online publication date: 13-Dec-2024
https://doi.org/10.1007/s00778-024-00878-5
Show More Cited By

Index Terms

Social event detection using multimodal clustering and integrating supervisory signals
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding
  2. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis

Recommendations

Multimodal Clustering via Deep Commonness and Uniqueness Mining
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Deep multimodal clustering have shown their competitiveness among different multimodal clustering algorithms. Existing algorithms usually boost the multimodal clustering by exploring the common knowledge among multiple modalities, which underutilizes ...
Social event detection with interaction graph modeling
MM '12: Proceedings of the 20th ACM international conference on Multimedia

This paper focuses on detecting social, physical-world events from photos posted on social media sites. The problem is important: cheap media capture devices have significantly increased the number of photos shared on these sites. The main contribution ...
Graph-based multimodal clustering for social multimedia

Real world datasets often consist of data expressed through multiple modalities. Clustering such datasets is in most cases a challenging task as the involved modalities are often heterogeneous. In this paper we propose a graph-based multimodal ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '12: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval

June 2012

489 pages

ISBN:9781450313292

DOI:10.1145/2324796

Conference Chairs:
Horace H. S. Ip
City University of Hong Kong
,
Yong Rui
Microsoft, China

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

ICMR '12

Sponsor:

SIGMM

ICMR '12: International Conference on Multimedia Retrieval

June 5 - 8, 2012

Hong Kong, China

Acceptance Rates

ICMR '12 Paper Acceptance Rate 50 of 145 submissions, 34%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

60
Total Citations
View Citations
1,022
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)2

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kovalchuk MFilatova AKorneev AKoreneva MNasonov DVoskresenskii ABoukhanovsky A(2024)SemConvTree: Semantic Convolutional Quadtrees for Multi-Scale Event Detection in Smart CitySmart Cities10.3390/smartcities70501077:5(2763-2780)Online publication date: 28-Sep-2024
https://doi.org/10.3390/smartcities7050107
He ZWan SZappatore MLu H(2024)A Similarity Matrix Low-Rank Approximation and Inconsistency Separation Fusion Approach for Multiview ClusteringIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.32719645:2(868-881)Online publication date: Feb-2024
https://doi.org/10.1109/TAI.2023.3271964
Mondal MKhayati MSandlin HCudré-Mauroux P(2024)A survey of multimodal event detection based on data fusionThe VLDB Journal10.1007/s00778-024-00878-534:1Online publication date: 13-Dec-2024
https://doi.org/10.1007/s00778-024-00878-5
Rasool SJain SMoon A(2024)A Graph Based-Novel Framework for Social Synchrony Detection Using Influential User and Event Detection ApproachCongress on Smart Computing Technologies10.1007/978-981-97-5081-8_37(491-506)Online publication date: 30-Oct-2024
https://doi.org/10.1007/978-981-97-5081-8_37
Peng HZhang RLi SCao YPan SYu P(2023)Reinforced, Incremental and Cross-Lingual Event Detection From Social MessagesIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.314499345:1(980-998)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TPAMI.2022.3144993
Shi SNie FWang RLi X(2023)Multi-View Clustering via Nonnegative and Orthogonal Graph ReconstructionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.309329734:1(201-214)Online publication date: Jan-2023
https://doi.org/10.1109/TNNLS.2021.3093297
Zhang ZHe W(2023)Tensorized topological graph learning for generalized incomplete multi-view clusteringInformation Fusion10.1016/j.inffus.2023.101914(101914)Online publication date: Jul-2023
https://doi.org/10.1016/j.inffus.2023.101914
Liang YBai WJiang Y(2023)A Unified Spectral Rotation Framework Using a Fused Similarity GraphMachine Learning and Knowledge Discovery in Databases: Research Track10.1007/978-3-031-43418-1_13(209-225)Online publication date: 17-Sep-2023
https://doi.org/10.1007/978-3-031-43418-1_13
Komecoglu BYilmaz B(2022)Topic Detection and Tracking Towards Determining Public Agenda ItemsMachine Learning for Societal Improvement, Modernization, and Progress10.4018/978-1-6684-4045-2.ch008(158-179)Online publication date: 24-Jun-2022
https://doi.org/10.4018/978-1-6684-4045-2.ch008
TANKO DTUNCER TDOGAN SAKBAL E(2022)REGRESSION METHODS FOR SOCIAL MEDIA DATA ANALYSISMugla Journal of Science and Technology10.22531/muglajsci.10282998:1(31-40)Online publication date: 28-Jun-2022
https://doi.org/10.22531/muglajsci.1028299
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents