More Web Proxy on the site http://driver.im/

short-paper

Laugh at Your Own Pace: Basic Performance Evaluation of Language Learning Assistance by Adjustment of Video Playback Speeds Based on Laughter Detection

Authors:

Buntarou ShizukiAuthors Info & Claims

L@S '22: Proceedings of the Ninth ACM Conference on Learning @ Scale

Pages 368 - 373

https://doi.org/10.1145/3491140.3528299

Published: 01 June 2022 Publication History

Abstract

Among various methods to learn a second language (L2), such as listening and shadowing, Extensive Viewing involves learning L2 by watching many videos. However, it is difficult for many L2 learners to smoothly and effortlessly comprehend video contents made for native speakers at the original speed. Therefore, we developed a language learning assistance system that automatically adjusts the playback speed according to the learner's comprehension. Our system judges that learners understand the contents if they laugh at the punchlines of comedy dramas, and vice versa. Experimental results show that this system supports learners with relatively low L2 ability (under 700 in TOEIC Score in the experimental condition) to understand video contents. Our system can widen learners' possible options of native speakers' videos as Extensive Viewing material.

Supplementary Material

MP4 File (L-at-S22-iswp118_PresentationVideo.mp4)

Presentation Video

Download
48.78 MB

References

[1]

Riku Arakawa and Hiromu Yakura. 2021. Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 99, 15 pages. https://doi.org/10.1145/3411764.3445339

Digital Library

[2]

Octavio Arriaga. 2021. Perception for Autonomous Systems. https://github.com/oarriaga/paz (Last Access Date:2022/4/6).

[3]

Betul Bal-Gezegin. 2014. An Investigation of Using Video vs. Audio for Teaching Vocabulary. Procedia - Social and Behavioral Sciences 143, 3 (2014), 450--457. https://doi.org/10.1016/j.sbspro.2014.07.516

[4]

Veronica Benigno and John De Jong. 2019. Linking vocabulary to the CEFR and the Global Scale of English: A psychometric model. In Developments in Language Education: A Memorial Volume in Honour of Sauli Takala, Ari Huhta, Gudrun Erickson, and Neus Figueras (Eds.). EALTA-European Association for Language Testing and Assessment, University of Jyvaskyla, Centre for Applied Language Studies, 8--29.

[5]

Eileen Kay Blau. 1990. The Effect of Syntax, Speed, and Pauses on Listening Comprehension. TESOL Quarterly 24, 4 (1990), 746--753. https://doi.org/10.2307/3587129

[6]

John Brooke. 1996. SUS: A "Quick and Dirty" Usability Scale. In Usability Evaluation In Industry. London: Taylor and Francis, 189--194.

[7]

Anna CS Chang. 2016. Teaching L2 Listening: In and Outside the Classroom. In English Language Teaching Today: Linking Theory and Practice. Springer, 111--125.

[8]

Hung chun Wang and Cheryl Wei yu Chen. 2020. Learning English From Youtubers: English L2 Learners' Self-regulated Language Learning on Youtube. Innovation in Language Learning and Teaching 14, 4 (2020), 333--346. https://doi.org/10.1080/17501229.2019.1607356

[9]

Jeffrey Cohn, Zara Ambadar, and Paul Ekman. 2007. Observer-Based Measurement of Facial Expression with the Facial Action Coding System. In The Handbook of Emotion Elicitation and Assessment. Oxford University Press Series in Affective Science, 2--53.

[10]

Software Freedom Conservancy. 2021. Selenium. https://www.selenium.dev/ja/ (Last Access Date:2022/4/6).

[11]

Dioco. 2022. Language Reactor. https://www.languagereactor.com/.

[12]

Paul Ekman. 1993. Facial Expression and Emotion. American psychologist 48, 4 (1993), 384--392. https://doi.org/10.1037/0003-066X.48.4.384

[13]

Paul Ekman and Wallace V Friesen. 1978. Facial Action Coding System. Palo Alto, CA: Consulting Psychologists Press.

[14]

English-Corpora.org. 2021. Corpus of Contemporary American English. https://www.english-corpora.org/coca/(Last Access Date:2022/4/6).

[15]

Yanxue Feng and Stuart Webb. 2020. Learning Vocabulary Through Reading, Listening, and Viewing: Which Mode of Input Is Most Effective? Studies in Second Language Acquisition 42, 3 (2020), 499--523. https://doi.org/10.1017/S0272263119000494

[16]

The Institute for International Business Communication. 2022. TOEIC Listening & Reading Test. https://www.iibc-global.org/toeic/test/lr.html (Last Access Date:2022/4/6).

[17]

Katsuya Fujii and Jun Rekimoto. 2019. SubMe: An Interactive Subtitle System with English Skill Estimation Using Eye Tracking. In Proceedings of the 10th Augmented Human International Conference 2019. Association for Computing Machinery, New York, NY, USA, Article 23, 9 pages. https://doi.org/10.1145/3311823.3311865

Digital Library

[18]

Mehdi Ghayoumi and Arvind Bansal. 2015. Unifying Geometric Features and Facial Action Units for Improved Performance of Facial Expression Analysis. In New Developments in Circuits, Systems, Signal Processing, Communications and Computers. 259--266.

[19]

Jon Gillick. 2021. Laughter-Detection. https://github.com/jrgillick/laughter-detection (Last Access Date:2022/4/6).

[20]

Sandra Hart and Lowell Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter Hancock and Najmedin Meshkati (Eds.). Vol. 52. North--Holland, 139--183. https://doi.org/10.1016/S0166--4115(08)62386--9

[21]

Abdolmajid Hayati and Firooz Mohmedi. 2011. The Effect of Films With and Without Subtitles on Listening Comprehension of EFL Learners. British Journal of Educational Technology 42, 1 (2011), 181--192. https://doi.org/10.1111/j.1467--8535.2009.01004.x

[22]

Sathaporn "Hubert" Hu and Wesley J. Willett. 2018. Kalgan: Video Player for Casual Language Learning. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3170427.3188498

Digital Library

[23]

Gwan-Hyeok Im and Liying Cheng. 2019. The Test of English for International Communication (TOEIC). Language Testing 36, 2 (2019), 315--324. https://doi.org/10.1177/0265532219828252

[24]

Francisca Ivone and Willy Renandya. 2019. Extensive Listening and Viewing in ELT. Teflin Journal 30, 2 (2019), 237--256. https://doi.org/10.15639/teflinjournal.v30i2/237--256

[25]

Chen-Tai Kao, Yen-Ting Liu, and Alexander Hsu. 2014. Speeda: Adaptive Speed-up for Lecture Videos. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 97--98. https://doi.org/10.1145/2658779.2658794

Digital Library

[26]

Frederic Kuder and Marion Richardson. 1937. The Theory of The Estimation of Test Reliability. Psychometrika 2, 3 (1937), 151--160. https://doi.org/10.1007/BF02288391

[27]

David Lang, Guanling Chen, Kathy Mirzaei, and Andreas Paepcke. 2020. Is Faster Better? A Study of Video Playback Speed. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. Association for Computing Machinery, New York, NY, USA, 260--269. https://doi.org/10.1145/3375462.3375466

Digital Library

[28]

Mina Lee, Beverly Roskos, and David R. Ewoldsen. 2013. The Impact of Subtitles on Comprehension of Narrative Film. Media Psychology 16, 4 (2013), 412--440. https://doi.org/10.1080/15213269.2013.826119

[29]

Wang Li and Willy Renandya. 2012. Effective Approaches to Teaching Listening: Chinese EFL Teachers' Perspectives. The Journal of Asia TEFL 9, 4 (2012), 79--111. https://doi.org/10.1177/2158244020917393

[30]

Ahmed Masrai. 2020. Can L2 Phonological Vocabulary Knowledge and Listening Comprehension be Developed Through Extensive Movie Viewing? The Case Of Arab EFL Learners. International Journal of Listening 34, 1 (2020), 54--69. https://doi.org/10.1080/10904018.2019.1582346

[31]

James Milton and Thomai Alexiou. 2009. Vocabulary Size and the Common European Framework of Reference for Languages. In Vocabulary Studies in First and Second Language Acquisition. Springer, 194--211. https://doi.org/10.1057/9780230242258

[32]

Setsuko Mori. 2015. If You Build It, They Will Come: From a "Field of Dreams" to a More Realistic View of Extensive Reading in an EFL Context. Reading in a foreign language 27 (2015), 129--135.

[33]

Naoto Nishida, Hinako Nozaki, and Buntarou Shizuki. 2021. Language Learning Assistance by Automatic Adjustment of Playback Speed Based on Facial Expression. In 29th Workshop on Interactive Systems and Software. Japan Society for Software Science and Technology, Hamamatsu, Shizuoka, Japan. (in Japanese, non-archival article).

[34]

Council of Europe. 2001. Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press. https://www.coe.int/en/web/common-european-framework-reference-languages.

[35]

The College of Optometrists. 2022. Look After Your Eye. https://lookafteryoureyes.org/eye-care/screen-use/ (Last Access Date:2022/4/6).

[36]

Elke Peters, Eva Heynen, and Eva Puimège. 2016. Learning Vocabulary Through Audiovisual Input: the Differential Effect of L1 Subtitles and Captions. System 63 (2016), 134--148. https://doi.org/10.1016/j.system.2016.10.002

[37]

Pallavi Powale. 2017. Variable speed playback on mobile. https://www.english-corpora.org/coca/ (Last Access Date:2022/4/6).

[38]

Georgia Pujadas and Carmen Munoz. 2020. Examining Adolescent EFL Learners' TV Viewing Comprehension Through Captions And Subtitles. Studies in Second Language Acquisition 42, 3 (2020), 551--575. https://doi.org/10.1017/S0272263120000042

[39]

Willy Renandya and George Jacobs. 2016. Extensive Reading and Listening in the L2 Classroom. In English Language Teaching Today: Linking Theory and Practice. Springer, Chapter 8, 97--110.

[40]

Willy A Renandya and Thomas SC Farrell. 2011. "Teacher, the tape is too fast!' Extensive Listening in ELT. ELT journal 65, 1 (2011), 52--59. https://doi.org/10.1093/elt/ccq015

[41]

Tannenbaum Richard and Wylie Caroline. 2008. Linking English-Language Test Scores Onto the Common European Framework of Reference: An Application of Standard-Setting Methodology. ETS Research Report Series 1 (2008), 1--75. https://doi.org/10.1002/j.2333--8504.2008.tb02120.x

[42]

Michael Rodgers. 2013. English Language Learning Through Viewing Television: an Investigation of Comprehension, Incidental Vocabulary Acquisition, Lexical Coverage, Attitudes, and Captions. Ph.D. Dissertation. Victoria University of Wellington.

[43]

Nathan Sakunkoo and Pattie Sakunkoo. 2013. GliFlix: Using Movie Subtitles For Language Learning. In The Adjunct Publication of the 26th Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 7--8.

[44]

Educational Testing Service. 2022. TOEFL. https://www.ets.org/toefl (Last Access Date:2022/4/6).

[45]

Sunghyun Song, Jeong-ki Hong, Ian Oakley, Jun Dong Cho, and Andrea Bianchi. 2015. Automatically Adjusting the Speed of E-Learning Videos. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1451--1456. https://doi.org/10.1145/2702613.2732711

Digital Library

[46]

Lars Stenius Staehr. 2008. Vocabulary Size and the Skills of Listening, Reading and Writing. The Language Learning Journal 36, 2 (2008), 139--152. https://doi.org/10.1080/09571730802389975

[47]

Webb Stuart. 2015. Extensive Viewing: Language Learning through Watching Television. In Language Learning Beyond the Classroom. 159--168. https://doi.org/10.4324/9781315883472--24

[48]

subtitlesfll.com. 2022. Subtitles for Language Learning (Prime Video). https://www.subtitlesfll.com/en.

[49]

Kousuke Sugai, Shigeru Yamane, and Kazuo Kanzaki. 2016. The Time Domain Factors Affecting EFL Learners' Listening Comprehension: a Study on Japanese EFL Learners. ARELE: Annual Review of English Language Education in Japan 27 (2016), 97--108. https://doi.org/10.20581/arele.27.0_97

[50]

Ying-li Tian, Takeo Kanade, and Jeffrey Cohn. 2001. Recognizing Action Units for Facial Expression Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 2 (2001), 97--115. https://doi.org/10.1109/34.908962

Digital Library

[51]

Stuart Webb and Michael Rodgers. 2009. Vocabulary Demands of Television Programs. Language Learning 59 (2009), 335--366. https://doi.org/10.1111/j.1467--9922.2009.00509.x

[52]

Mehmet Ozgen and Nazli Gunduz. 2020. Authentic Captioned Sitcom as Listening Comprehension Material in English Language Teaching. ELT Research Journal 9, 2 (2020), 167--193.

Cited By

Sechayk YShamir AIgarashi T(2024)SmartLearn: Visual-Temporal Accessibility for Slide-based e-learning VideosExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650883(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650883
Chung YHsu CChan MCherng F(2024)Enhancing ESL Learners' Experience and Performance through Gradual Adjustment of Video Speed during Extensive ViewingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642366(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642366
Varnavsky A(2024)The Three-Stage Hierarchical Logistic Model Controlling Personalized Playback of Audio Information for Intelligent Tutoring SystemsIEEE Transactions on Learning Technologies10.1109/TLT.2024.343947017(2005-2019)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3439470
Show More Cited By

Index Terms

Laugh at Your Own Pace: Basic Performance Evaluation of Language Learning Assistance by Adjustment of Video Playback Speeds Based on Laughter Detection
1. Applied computing
  1. Education
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Internet-Based Technology Use in Second Language Learning: A Systematic Review

Ever since computer technologies were accessible to second language learners and teachers, various types of computer-assisted language learning (CALL) have been harnessed in the service of teaching languages. Most recently, the advent of online ...
Supportiveness of language MOOCs for self-regulated learning: a review of commercial language MOOCs on the market

MOOCs have been frequently applied as an effective approach to language education, especially when they can support self-regulated learning. However, few studies have discussed the supportiveness of using MOOCs for language education (i.e. language MOOCs) ...
A fully-automated computer-assisted language learning courseware for beginner-level Korean language learners
AIC'09: Proceedings of the 9th WSEAS international conference on Applied informatics and communications

Recently, the progress of information and communication technologies enables computer-assisted language learning. Conventional computer-assisted language learning courseware hardly improves learner's speaking skill, since it is difficult for the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

L@S '22: Proceedings of the Ninth ACM Conference on Learning @ Scale

June 2022

491 pages

ISBN:9781450391580

DOI:10.1145/3491140

General Chair:
René F. Kizilcec
Cornell University, USA
,
Program Chairs:
Katie Davis
University of Washington, USA
,
Xavier Ochoa
New York University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

L@S '22

L@S '22: Ninth (2022) ACM Conference on Learning @ Scale

June 1 - 3, 2022

NY, New York City, USA

Acceptance Rates

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
149
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)5

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sechayk YShamir AIgarashi T(2024)SmartLearn: Visual-Temporal Accessibility for Slide-based e-learning VideosExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650883(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650883
Chung YHsu CChan MCherng F(2024)Enhancing ESL Learners' Experience and Performance through Gradual Adjustment of Video Speed during Extensive ViewingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642366(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642366
Varnavsky A(2024)The Three-Stage Hierarchical Logistic Model Controlling Personalized Playback of Audio Information for Intelligent Tutoring SystemsIEEE Transactions on Learning Technologies10.1109/TLT.2024.343947017(2005-2019)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3439470
Kawamura KRekimoto JWard JMcGill MMarky K(2023)AIx Speed: Playback Speed Optimization Using Listening Comprehension of Speech Recognition ModelsProceedings of the Augmented Humans International Conference 202310.1145/3582700.3582722(200-208)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3582722
Kawamura KRekimoto J(2022)AIx speed: Playback Speed Optimization using Listening Comprehension of Speech Recognition ModelsAdjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526114.3558727(1-3)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526114.3558727

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents