Miruoto : Sports event atmosphere visual rendering through real-time image and sound processing system
Authors: 
Guillaume Gourmelen, Shutaro Toriya, Eiko Miya, Naohisa Shioura, Hiroyasu IwataAuthors Info & Claims
Article No.: 14, Pages 1 - 2
Abstract
Did you already imagine how would it be to watch a sport match without sounds? You would miss all this specific sport related sounds but also mostly miss a big part of the atmosphere present in the stadium, that is particular to live events. This is what happens to most Deaf and Hard of Hearing persons. Towards Tokyo 2025 Deaflympics, we developed an AI-based system able to recognize sounds and players motion to render in real time sound related Onomatopoeia over the match video as one could see in Comics or Manga.
References
[1]
Akita. 2009. A grammar of sound-symbolic words in Japanese : theoretical approaches to iconic and lexical properties of mimetics. Ph.D. Dissertation. Kobe Univ.
[2]
Jort F.2017. Audio Set: An ontology and human-labeled dataset for audio events. In Proc. IEEE ICASSP 2017.
[3]
Tsukasa Fukusato and Shigeo Morishima. 2014. Automatic depiction of onomatopoeia in animation considering physical phenomena. In Proceedings of the 7th International Conference on Motion in Games. 161–169. https://doi.org/10.1145/2668064.2668096
[4]
Andrew G.2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arxiv:cs.CV/1704.04861
[5]
Carling Gerd. 2014. Motivated language change: processes involved in the growth and conventionalization of onomatopoeia and sound symbolism. Acta Linguistica Hafniensia 46, 2 (2014), 199–217. https://doi.org/10.1080/03740463.2014.990293
[6]
Hamano. 2000. The Sound-Symbolic System of Japanese. Language 76, 1 (March 2000), 174. https://doi.org/10.2307/417404
[7]
Keisuke Kambara and Koji Tsukada. 2010. Onomatopen: Painting Using Onomatopoeia. Springer Berlin Heidelberg, 43–54. https://doi.org/10.1007/978-3-642-15399-0_5
[8]
M. Manar. 2021. Onomatopoeia and Direct Speech on the Narratives of Deaf and Hearing Students. In Proceedings of the Tenth International Conference on Languages and Arts (ICLA 2021). 85–91. https://doi.org/10.2991/assehr.k.211129.015
[9]
M. Namatame. 2019. Effectiveness of Multisensory Methods in Learning Onomatopoeia for the Hearing-Impaired. https://doi.org/10.1007/978-3-030-21151-6_1
[10]
Statista. 2022. Japan: manga reading frequency. http://tinyurl.com/4jwsw68b.
[11]
Jan‐Olof Svantesson. 2017. Sound symbolism: the role of word sound in meaning. WIREs Cognitive Science 8, 5 (March 2017). https://doi.org/10.1002/wcs.1441
[12]
Lei Ye. 2023. Onomatopoeia, Sound Effects and Humour in Japanese Anime and US Animation Films and Television. Animation 18, 3 (Nov. 2023), 273–291. https://doi.org/10.1177/17468477231206681
Recommendations
Sound image externalization for headphone based real-time 3D audio
3D audio effects can provide immersive auditory experience, but we often face the so-called in-head localization (IHL) problem in headphone sound reproduction. To address this problem, we propose an effective sound image externalization approach. ...
Real-time rendering of decorative sound textures for soundscapes
Audio recordings contain rich information about sound sources and their properties such as the location, loudness, and frequency of events. One prevalent component in sound recordings is the sound texture, which contains a massive number of events. In ...
Virtual Sound Rendering in a Stereophonic Loudspeaker Setup
This paper presents a mathematical analysis of the effects of interchannel amplitude and time differences in two channel (stereophonic) sound systems. The analysis is developed by computing the acoustic conditions at the listener's ears as a function of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 July 2024
Check for updates
Qualifiers
- Abstract
- Research
- Refereed limited
Conference
SIGGRAPH '24
Sponsor:
SIGGRAPH '24: Special Interest Group on Computer Graphics and Interactive Techniques Conference
July 27 - August 1, 2024
CO, Denver, USA
Acceptance Rates
Overall Acceptance Rate 1,822 of 8,601 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 85Total Downloads
- Downloads (Last 12 months)85
- Downloads (Last 6 weeks)12
Reflects downloads up to 13 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format