Abstract
Large-scale dynamic relational data visualization has attracted considerable research attention recently. We introduce dynamic data visualization into the multimedia domain, and present an interactive and scalable system, VideoMap, for exploring large-scale video content. A long video or movie has much content; the associations between the content are complicated. VideoMap uses new visual representations to extract meaningful information from video content. Map-based visualization naturally and easily summarizes and reveals important features and events in video. Multi-scale descriptions are used to describe the layout and distribution of temporal information, spatial information, and associations between video content. Firstly, semantic associations are used in which map elements correspond to video contents. Secondly, video contents are visualized hierarchically from a large scale to a fine-detailed scale. VideoMap uses a small set of sketch gestures to invoke analysis, and automatically completes charts by synthesizing visual representations from the map and binding them to the underlying data. Furthermore, VideoMap allows users to use gestures to move and resize the view, as when using a map, facilitating interactive exploration. Our experimental evaluation of VideoMap demonstrates how the system can assist in exploring video content as well as significantly reducing browsing time when trying to understand and find events of interest.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Avoid common mistakes on your manuscript.
References
Mashima, D.; Kobourov, S.; Hu, Y. Visualizing dynamic data with maps. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 9, 1424–1437, 2012.
Gansner, E. R.; Hu, Y.; North, S. Visualizing streaming text data with dynamic graphs and maps. Lecture Notes in Computer Science, Vol. 7704. Didimo, W.; Patrignani, M. Eds. Springer Berlin Heidelberg, 439–450, 2013.
Information on https://ia.net/know-how/iatrendmap-2007v2.
Fels, S.; Mase, K. Interactive video cubism. In: Proceedings of the 1999 Workshop on New Paradigms in Information Visualization and Manipulation in Conjunction with the 8th ACM International Conference on Information and Knowledge Management, 78–82, 1999.
Daniel, G.; Chen, M. Video visualization. In: Proceedings of IEEE Visualization, 409–416, 2003.
Yeung, M. M.; Yeo, B.-L. Video visualization for compact presentation and fast browsing of pictorial content. IEEE Transactions on Circuits and Systems for Video Technology Vol. 7, No. 5, 771–785, 1997.
Taniguchi, Y.; Akutsu, A.; Tonomura, Y. Panorama Excerpts: Extracting and packing panoramas for video browsing. In: Proceedings of the 5th ACM International Conference on Multimedia, 427–436, 1997.
Hua, X.-S.; Li, S.; Zhang, H.-J. Video booklet. 2010. Available at http://dent.cecs.uci.edu/papers/icme05/defevent/papers/cr1126.pdf.
Goldman, D. B.; Curless, B.; Salesin, D.; Seitz, S. M. Schematic storyboarding for video visualization and editing. ACM Transactions on Graphics Vol. 25, No. 3, 862–871, 2006.
Nguyen, C.; Niu, Y.; Liu, F. Video summagator: An interface for video summarization and navigation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 647–650, 2012.
Shah, R.; Narayanan, P. J. Interactive video manipulation using object trajectories and scene backgrounds. IEEE Transactions on Circuits and Systems for Video Technology Vol. 23, No. 9, 1565–1576, 2013.
Ma, C.-X.; Liu, Y.-J.; Wang, H.-A.; Teng, D.-X.; Dai, G.-Z. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia Vol. 14, No. 4, 1153–1165, 2012.
Truong, B. T.; Venkatesh, S. Video abstraction: A systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 3, No. 1, Article No. 3, 2007.
Viaud, M.-l.; Buisson, O.; Saulnier, A.; Guenais, C. Video exploration: From multimedia content analysis to interactive visualization. In: Proceedings of the 18th ACM International Conference on Multimedia, 1311–1314, 2010.
Höferlin, M.; Höferlin, B.; Weiskopf, D. Video visual analytics of tracked moving objects. 2012. Available at http://www.vis.uni-stuttgart.de/uploads/tx vispublications/Hoeferlin2009b.pdf.
Meghdadi, A. H.; Irani, P. Interactive exploration of surveillance video through action shot summarization and trajectory visualization. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 12, 2119–2128, 2013.
Walton, S.; Berger, K.; Ebert, D.; Chen, M. Vehicle object retargeting from dynamic traffic videos for realtime visualisation. The Visual Computer Vol. 30, No. 5, 493–505, 2014.
Höferlin, B.; Höferlin, M.; Heidemann, G.; Weiskopf, D. Scalable video visual analytics. Information Visualization Vol. 14, No. 1, 10–26, 2013.
Caspi, Y.; Axelrod, A.; Matsushita, Y.; Gamliel, A. Dynamic stills and clip trailers. The Visual Computer Vol. 22, No. 9, 642–652, 2006.
Correa, C. D.; Ma, K.-L. Dynamic video narratives. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 88, 2010.
Tang, A.; Greenberg, S.; Fels, S. Exploring video streams using slit-tear visualizations. In: Proceedings of Extended Abstracts on Human Factors in Computing Systems, 3509–3510, 2009.
Tanahashi, Y.; Ma, K.-L. Design considerations for optimizing storyline visualizations. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 12, 2679–2688, 2012.
Li, F. C.; Gupta, A.; Sanocki, E.; He, L.-w.; Rui, Y. Browsing digital video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 169–176, 2000.
Liu, Y.-J.; Ma, C.-X.; Fu, Q.; Fu, X.; Qin, S.-F.; Xie, L. A sketch-based approach for interactive organization of video clips. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 11, No. 1, Article No. 2, 2014.
Jawaheer, G.; Weller, P.; Kostkova, P. Modeling user preferences in recommender systems: A classification framework for explicit and implicit user feedback. ACM Transactions on Interactive Intelligent Systems Vol. 4, No. 2, Article No. 8, 2014.
Zhang, X.; Furnas, G. W. mCVEs: Using crossscale collaboration to support user interaction with multiscale structures. Presence Vol. 14, No. 1, 31–46, 2005.
McCarthy, J. F.; Meidel, E. S. ActiveMap: A visualization tool for location awareness to support informal interactions. Lecture Notes in Computer Science, Vol. 1707. Gellersen, H.-W. Ed. Springer Berlin Heidelberg, 158–170, 2000.
Nesbitt, K. V. Getting to more abstract places using the metro map metaphor. In: Proceedings of the 8th International Conference on Information Visualisation, 488–493, 2004.
Gansner, E. R.; Hu, Y.; Kobourov, S. GMap: Visualizing graphs and clusters as maps. In: Proceedings of IEEE Pacific Visualization Symposium, 201–208, 2010.
Fu, X. L.; Cai, L. H.; Liu, Y.; Jia, J.; Chen, W. F.; Yi, Z.; Zhao, G. Z.; Liu, Y. J.; Wu, C. X. A computational cognition model of perception, memory, and judgment. Science China Information Sciences Vol. 57, No. 3, 1–15, 2014.
Solway, A.; Botvinick, M. M. Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates. Psychological Review Vol. 119, No. 1, 120–154, 2012.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Cui-Xia Ma received her Ph.D. degree from the Institute of Software, Chinese Academy of Sciences, Beijing, China, in 2003. She is now a professor with the Institute of Software, Chinese Academy of Sciences. Her research interests include human–computer interaction and multimedia computing.
Yang Guo started studying in the Institute of Software, Chinese Academy of Sciences, Beijing, China, in 2013. He is now pursuing a master degree in the Institute of Software, Chinese Academy of Sciences. His research interests include human–computer interaction and multimedia visualization.
Hong-An Wang received his Ph.D. degree from the Institute of Software, Chinese Academy of Sciences, Beijing, China, in 1999. He is now a professor with the Institute of Software, Chinese Academy of Sciences. His research interests include real-time intelligence and user interface.
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Ma, CX., Guo, Y. & Wang, HA. VideoMap: An interactive and scalable visualization for exploring video content. Comp. Visual Media 2, 291–304 (2016). https://doi.org/10.1007/s41095-016-0049-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-016-0049-1