[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2072298.2072306acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Combining content-based analysis and crowdsourcing to improve user interaction with zoomable video

Published: 28 November 2011 Publication History

Abstract

This paper introduces a new paradigm for interacting with zoomable video. Our interaction technique reduces the number of zooms and pans required by providing recommended viewports to the users, and replaces multiple zoom and pan actions with a simple click on the recommended viewport. The usefulness of our technique is visible in the quality of the recommended viewport, which needs to match the user intention, track movement in the scene, and properly frame the scene in the video. To this end, we propose a hybrid method where content analysis is complimented by the implicit feedback of a community of users in order to recommend viewports. We first compute preliminary sets of recommended viewports by analyzing the content of the video. These viewports allow tracking of moving objects in the scene, and are framed without violating basic aesthetic rules. To improve the relevance of the recommended viewports, we collect viewing statistics as users view a video, and use the viewports they select to reinforce the importance of certain recommendations and penalize others. New recommendations that are not previously recognized by content analysis may also emerge. The resulting recommended viewports converge towards the regions in the video that are relevant to users. A user study involving 70 participants shows that an user interface incorporating with our paradigm leads to more number of zooms, into more informative regions with fewer interactions.

References

[1]
C. Beleznai, B. Fruhstuck, and H. Bischof. Human tracking by fast mean shift mode seeking. Journal of Multimedia, 1(1):1--8, 2006.
[2]
A. Carlier, V. Charvillat, W. T. Ooi, R. Grigoras, and G. Morin. Crowdsourced automatic zoom and scroll for video retargeting. In Proceedings of MULTIMEDIA'10, pages 201--210, Florence, Italy, 2010.
[3]
A. Carlier, R. Guntur, and W. T. Ooi. Towards characterizing users' interaction with zoomable video. In Proceedings of the 2010 ACM workshop on Social,adaptive and personalized multimedia interaction and access, pages 21--24, Florence, Italy, 2010.
[4]
K.-Y. Cheng, S.-J. Luo, B.-Y. Chen, and H.-H. Chu. Smartplayer: User-centric video fast-forwarding. In Proceedings of CHI'09, 2009.
[5]
D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell., 24(5):603--619, 2002.
[6]
A. Doan, R. Ramakrishnan, and A. Y. Halevy. Crowdsourcing systems on the world-wide web. Commun. ACM, 54:86--96, April 2011.
[7]
P. Dragicevic, G. Ramos, J. Bibliowitcz, D. Nowrouzezahrai, R. Balakrishnan,and K. Singh. Video browsing by direct manipulation. In Proceedings of CHI'08, pages 237'246, Florence, Italy, 2008.
[8]
H. El-Alfy, D. Jacobs, and L. Davis. Multi-scale video cropping. In Proceedings of MULTIMEDIA '07, pages 97--106, Augsburg, Germany, 2007.
[9]
D. B. Goldman, C. Gonterman, B. Curless, D. Salesin, and S. M. Seitz. Video object annotation, navigation, and composition. In Proceedings of UIST'08, pages 3--12, 2008.
[10]
J. Han, K. N. Ngan, M. Li, and H. Zhang. Unsupervised extraction of visual attention objects in color images. IEEE Trans. Circuits Syst. Video Techn.,16(1):141--145, 2006.
[11]
L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(11):1254--1259, 1998.
[12]
F. Liu and M. Gleicher. Video retargeting: automating pan and scan. In Proceedings of MULTIMEDIA '06, pages 241--250, Santa Barbara, CA, USA, 2006.
[13]
S. Montabone and A. Soto. Human detection using a mobile platform and novel features derived from a visual saliency mechanism. Image and Vision Computing, 28(3):391--402, 2010.
[14]
K. Q. M. Ngo, R. Guntur, A. Carlier, and W. T. Ooi. Supporting zoomable video streams via dynamic region-of-interest cropping. In Proceedings of MMSys'10, pages 259--270, Scottsdale, AZ, USA, 2010.
[15]
M. Rubinstein, A. Shamir, and S. Avidan. Multi-operator media retargeting. ACM Trans. Graph., 28(3), 2009.
[16]
D. A. Shamma, R. Shaw, P. L. Shafton, and Y. Liu. Watch what i watch: using community activity to understand content. In Multimedia Information Retrieval, pages 275--284, 2007.
[17]
F. Shipman, A. Girgensohn, and L. Wilcox. Authoring, viewing, and generating hypervideo: An overview of hyper-hitchcock. ACM Trans.Multimedia Comput. Commun. Appl., 5(2):1--19, 2008.
[18]
T. Syeda-Mahmood and D. Ponceleon. Learning video browsing behavior and its application in the generation of video previews. In Proceedings of MULTIMEDIA'01, pages 119--128, Ottawa, Canada, 2001.
[19]
N. Ukita, T. Ono, and M. Kidode. Region extraction of a gaze object using the gaze point and view image sequences. In Proceedings of ICMI'05, pages 129--136, Torento, Italy, 2005.
[20]
P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR'01, 1:511--518, 2001.
[21]
Y.-S. Wang, H. Fu, O. Sorkine, T.-Y. Lee, and H.-P. Seidel. Motion-aware temporal coherence for video resizing. ACM Trans. Graph., 28:127:1--127:10,December 2009.
[22]
X. Xie, H. Liu, S. Goumaz, and W.-Y. Ma. Learning user interest for image browsing on small-form-factor devices. In Proceedings of CHI'05, pages 671--680, Portland, Oregon, USA, 2005.

Cited By

View all
  • (2022)Exploring jump back behavior patterns and reasons in e-book systemSmart Learning Environments10.1186/s40561-021-00183-69:1Online publication date: 4-Jan-2022
  • (2022)FitVid: Responsive and Flexible Video Content AdaptationProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501948(1-16)Online publication date: 29-Apr-2022
  • (2021)Mobile link prediction: Automated creation and crowdsourced validation of knowledge graphsMicroprocessors and Microsystems10.1016/j.micpro.2021.104335(104335)Online publication date: Oct-2021
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '11: Proceedings of the 19th ACM international conference on Multimedia
November 2011
944 pages
ISBN:9781450306164
DOI:10.1145/2072298
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. content-analysis
  2. crowdsourcing
  3. interaction techniques
  4. zoomable video

Qualifiers

  • Research-article

Conference

MM '11
Sponsor:
MM '11: ACM Multimedia Conference
November 28 - December 1, 2011
Arizona, Scottsdale, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)0
Reflects downloads up to 04 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Exploring jump back behavior patterns and reasons in e-book systemSmart Learning Environments10.1186/s40561-021-00183-69:1Online publication date: 4-Jan-2022
  • (2022)FitVid: Responsive and Flexible Video Content AdaptationProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501948(1-16)Online publication date: 29-Apr-2022
  • (2021)Mobile link prediction: Automated creation and crowdsourced validation of knowledge graphsMicroprocessors and Microsystems10.1016/j.micpro.2021.104335(104335)Online publication date: Oct-2021
  • (2018)A robust video watermarking based on feature regions and crowdsourcingMultimedia Tools and Applications10.1007/s11042-018-5888-677:20(26769-26791)Online publication date: 1-Oct-2018
  • (2017)Smart JumpProceedings of the 26th International Conference on World Wide Web Companion10.1145/3041021.3054166(331-339)Online publication date: 3-Apr-2017
  • (2016)Interaction-Enhancing Strategies for Educational VideoProceedings of HCI Korea10.17210/hcik.2016.01.356(356-363)Online publication date: 27-Jan-2016
  • (2016)Towards hybrid cloud-assisted crowdsourced live streamingProceedings of the 26th International Workshop on Network and Operating Systems Support for Digital Audio and Video10.1145/2910642.2910644(1-6)Online publication date: 10-May-2016
  • (2015)Automated Link Generation for Sensor-Enriched Smartphone ImagesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/280820912:1s(1-25)Online publication date: 21-Oct-2015
  • (2015)A Video Timeline with Bookmarks and Prefetch State for Faster Video BrowsingProceedings of the 23rd ACM international conference on Multimedia10.1145/2733373.2806376(967-970)Online publication date: 13-Oct-2015
  • (2015)Automatic multi-camera remix from single videoProceedings of the 30th Annual ACM Symposium on Applied Computing10.1145/2695664.2695881(1270-1277)Online publication date: 13-Apr-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media