short-paper

Dynamic Adjustment of Subtitles Using Audio Fingerprints

Authors:

Lucas C. Villa Real,

Rodrigo Laiola Guimarães,

Priscilla AveglianoAuthors Info & Claims

MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Pages 975 - 978

https://doi.org/10.1145/2733373.2806378

Published: 13 October 2015 Publication History

Get Access

Abstract

Anyone who ever downloaded subtitle files from the Internet has faced problems synchronizing them with the associated media files. Even with the efforts of communities on reviewing user-contributed subtitles and with mechanisms in movie players to automate the discovery of subtitles for a given media, users still face lip synchronization issues. In this work we conduct a study on several subtitle files associated with popular movies and TV series and analyze their differences. Based on that, we propose a two-phase subtitle synchronization method that annotates subtitles with audio fingerprints, which serve as synchronization anchors to the media player. Preliminary results obtained with our prototype suggest that our technique is effective and has minimal impact on the extension of subtitle formats and on media playback performance.

References

[1]

A. Brown, R. Jones, et al. Dynamic subtitles: the user experience. In Proceedings of the 2015 ACM International Conference on Interactive Experiences for TV and Online Video. ACM, 2015.

Digital Library

Google Scholar

[2]

D. C. Bulterman, A. Jansen, et al. An efficient, streamable text format for multimedia captions and subtitles. In Proceedings of the 2007 ACM symposium on Document engineering, pages 101--110. ACM, 2007.

Digital Library

Google Scholar

[3]

C. Concolato and J. Le Feuvre. Live http streaming of video and subtitles within a browser. In Proceedings of the 4th ACM Multimedia Systems Conference, pages 146--150. ACM, 2013.

Digital Library

Google Scholar

[4]

M. Federico and M. Furini. An automatic caption alignment mechanism for off-the-shelf speech recognition technologies. Multimedia tools and applications, 72(1):21--40, 2014.

Digital Library

Google Scholar

[5]

R. Hong, M. Wang, et al. Dynamic captioning: video accessibility enhancement for hearing impairment. In Proceedings of the international conference on Multimedia, pages 421--430. ACM, 2010.

Digital Library

Google Scholar

[6]

G. Kovacs and R. C. Miller. Smart subtitles for vocabulary learning. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, pages 853--862. ACM, 2014.

Digital Library

Google Scholar

[7]

X. Liu and W. Wang. Robustly extracting captions in videos based on stroke-like edges and spatio-temporal analysis. Multimedia, IEEE Transactions on, 14(2):482--489, 2012.

Digital Library

Google Scholar

[8]

K. Rooney. The impact of keyword caption ratio on foreign language listening comprehension. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 4(2):11--28, 2014.

Digital Library

Google Scholar

[9]

J. Tiedemann. Synchronizing translated movie subtitles. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC'2008), 2008.

Google Scholar

Cited By

View all

Borges TGuimarães Rde Salles Soares Neto C(2020)Toward Assessing the Quality of Subtitles based on SRT Files ProcessingProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3428658.3431758(153-156)Online publication date: 30-Nov-2020
https://dl.acm.org/doi/10.1145/3428658.3431758
Guimarães RBrito JSantos CMota MSerique BPrates RCandello H(2018)Investigating the Influence of Subtitles Synchronization in the Viewer's Quality of ExperienceProceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems10.1145/3274192.3274222(1-10)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1145/3274192.3274222
Brito JGuimaraes RSantos CRoesler VValdeni de Lima JSaibel Santos CWillrich R(2017)Investigating the Collaborative Process of Subtitles Creation and Sharing for Videos on the WebProceedings of the 23rd Brazillian Symposium on Multimedia and the Web10.1145/3126858.3131592(69-72)Online publication date: 17-Oct-2017
https://dl.acm.org/doi/10.1145/3126858.3131592
Show More Cited By

Recommendations

Adaptive Subtitles: Preferences and Trade-Offs in Real-Time Media Adaption
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Subtitles can help improve the understanding of media content. People enable subtitles based on individual characteristics (e.g., language or hearing ability), viewing environment, or media context (e.g., drama, quiz show). However, some people find ...
Dynamic Subtitles in Cinematic Virtual Reality
TVX '18: Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video

Cinematic Virtual Reality has been increasing in popularity in recent years. Watching 360° movies with a Head Mounted Display, the viewer can freely choose the direction of view, and thus the visible section of the movie. Therefore, a new approach for ...
Dynamic Subtitles: The User Experience
TVX '15: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video

Subtitles (closed captions) on television are typically placed at the bottom-centre of the screen. However, placing subtitles in varying positions, according to the underlying video content (`dynamic subtitles'), has the potential to make the overall ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MM '15: Proceedings of the 23rd ACM international conference on Multimedia

October 2015

1402 pages

ISBN:9781450334594

DOI:10.1145/2733373

General Chairs:
Xiaofang Zhou
The University of Queensland, Australia
,
Alan F. Smeaton
Dublin City University, Ireland
,
Qi Tian
The University of Texas at San Antonio, USA
,
Program Chairs:
Dick C.A. Bulterman
FXPAL, USA
,
Heng Tao Shen
The University of Queensland, Australia
,
Ketan Mayer-Patel
The University of North Carolina, USA
,
Shuicheng Yan
National University of Singapore, Singapore

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Brazilian Ministry of Science and Technology

Conference

MM '15

Sponsor:

SIGMM

MM '15: ACM Multimedia Conference

October 26 - 30, 2015

Brisbane, Australia

Acceptance Rates

MM '15 Paper Acceptance Rate 56 of 252 submissions, 22%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
261
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)5

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Borges TGuimarães Rde Salles Soares Neto C(2020)Toward Assessing the Quality of Subtitles based on SRT Files ProcessingProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3428658.3431758(153-156)Online publication date: 30-Nov-2020
https://dl.acm.org/doi/10.1145/3428658.3431758
Guimarães RBrito JSantos CMota MSerique BPrates RCandello H(2018)Investigating the Influence of Subtitles Synchronization in the Viewer's Quality of ExperienceProceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems10.1145/3274192.3274222(1-10)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1145/3274192.3274222
Brito JGuimaraes RSantos CRoesler VValdeni de Lima JSaibel Santos CWillrich R(2017)Investigating the Collaborative Process of Subtitles Creation and Sharing for Videos on the WebProceedings of the 23rd Brazillian Symposium on Multimedia and the Web10.1145/3126858.3131592(69-72)Online publication date: 17-Oct-2017
https://dl.acm.org/doi/10.1145/3126858.3131592
Laiola Guimarães RAvegliano PVilla Real LSablatnig RHassan T(2016)A Lightweight and Efficient Mechanism for Fixing the Synchronization of Misaligned Subtitle DocumentsProceedings of the 2016 ACM Symposium on Document Engineering10.1145/2960811.2960812(175-184)Online publication date: 13-Sep-2016
https://dl.acm.org/doi/10.1145/2960811.2960812

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Adaptive Subtitles: Preferences and Trade-Offs in Real-Time Media Adaption

Dynamic Subtitles in Cinematic Virtual Reality

Dynamic Subtitles: The User Experience