Concept Preserving Visual Summarization of Social Image Search Results

Existing tag based social media search engines present search results as a ranked list of images. But, they fail to identify visual, textual and geographical concepts present in query results. In this paper, we present an approach for automatic generation of visual, textual and geographical concept preserving summary of social image search results. For user specified query, search results are collected from popular content-sharing websites such as Flickr. Aim of the algorithm is, to generate representative but diverse summary having a set of images, information about locations-of-interest (LOI) associated with the query, and a set of tags, describing the context of images. The proposed scheme exploits multiple modalities in order to understand context and content of geotagged social images. We formulate the problem as a graph clustering problem, where nodes are images and edge weight is computed as geo-graphical distance, tag-based similarity between images and visual similarity between images. In order to reduce the computational overhead, we implement late fusion of three different edge weight parameters. An innovative Graph based clustering algorithm using Haversine distance formula is proposed for geo-clustering of images. Performance evaluation is based on intrinsic and extrinsic methods. We also present an evaluation protocol having no human intervention for evaluating coverage of geographical spread of images in the final result and cluster coherence. Through empirical study, we demonstrate the effectiveness of our algorithm against state-of-the-art image search result summarization methods.