More Web Proxy on the site http://driver.im/

short-paper

Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging

Authors:

Kenny Q. ZhuAuthors Info & Claims

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Pages 3652 - 3656

https://doi.org/10.1145/3459637.3482097

Published: 30 October 2021 Publication History

Abstract

Audio tagging aims at predicting sound events occurred in a recording. Traditional models require enormous laborious annotations, otherwise performance degeneration will be the norm. Therefore, we investigate robust audio tagging models in low-resource scenarios with the enhancement of knowledge graphs. Besides existing ontological knowledge, we further propose a semi-automatic approach that can construct temporal knowledge graphs on diverse domain-specific label sets. Moreover, we leverage a variant of relation-aware graph neural network, D-GCN, to combine the strength of the two knowledge types. Experiments on AudioSet and SONYC urban sound tagging datasets suggest the effectiveness of the introduced temporal knowledge, and the advantage of the combined KGs with D-GCN over single knowledge source.

Supplementary Material

MP4 File (zhiling_cikm_processed.mp4)

Presentation video for the CIKM '21 paper: Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging

Download
42.02 MB

References

[1]

Juan P Bello, Claudio Silva, Oded Nov, R Luke Dubois, Anish Arora, Justin Salamon, Charles Mydlarz, and Harish Doraiswamy. 2019. Sonyc: A system for monitoring, analyzing, and mitigating urban noise pollution. Commun. ACM, Vol. 62, 2 (2019), 68--77.

Digital Library

[2]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.

[3]

Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren. 2020. Scalable multi-hop relational reasoning for knowledge-aware question answering. arXiv preprint arXiv:2005.00646 (2020).

[4]

Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel PW Ellis, Aren Jansen, and R Channing Moore. 2020. Addressing missing labels in large-scale sound event recognition using a teacher-student framework with loss masking. IEEE Signal Processing Letters, Vol. 27 (2020), 1235--1239.

[5]

Frederic Font, Gerard Roma, and Xavier Serra. 2018. Sound sharing and retrieval. In Computational analysis of sound scenes and events. Springer, 279--301.

[6]

Zhouyu Fu, Guojun Lu, Kai Ming Ting, and Dengsheng Zhang. 2010. A survey of audio-based music classification and annotation. IEEE transactions on multimedia, Vol. 13, 2 (2010), 303--319.

Digital Library

[7]

Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. 2017. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 776--780.

Digital Library

[8]

Arindam Jati, Naveen Kumar, Ruxin Chen, and Panayiotis Georgiou. 2019. Hierarchy-aware loss function on a tree structured label space for audio event detection. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6--10.

[9]

Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, and Minlie Huang. 2020. Language generation with multi-hop reasoning on commonsense knowledge graph. arXiv preprint arXiv:2009.11692 (2020).

[10]

Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D Plumbley. 2020. Panns: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 28 (2020), 2880--2894.

Digital Library

[11]

Qiuqiang Kong, Yin Cao, Turab Iqbal, Yong Xu, Wenwu Wang, and Mark D Plumbley. 2019. Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems. arXiv preprint arXiv:1904.03476 (2019).

[12]

Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation. 24--26.

Digital Library

[13]

Hongsong Li, Kenny Q Zhu, and Haixun Wang. 2013. Data-driven metaphor recognition and explanation. Transactions of the Association for Computational Linguistics, Vol. 1 (2013), 379--390.

[14]

George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM, Vol. 38, 11 (1995), 39--41.

Digital Library

[15]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European semantic web conference. Springer, 593--607.

[16]

Harsh Shrivaslava, Yfang Yin, Rajiv Ratn Shah, and Roger Zimmermann. 2020. MT-GCN for multi-label audio tagging with noisy labels. In ICASSP 2020--2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 136--140.

[17]

Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. arXiv preprint arXiv:2004.05483 (2020).

[18]

Yiwei Sun and Shabnam Ghaffarzadegan. 2020. An ontology-aware framework for audio event classification. In ICASSP 2020--2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 321--325.

[19]

Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. YFCC100M: The new data in multimedia research. Commun. ACM, Vol. 59, 2 (2016), 64--73.

Digital Library

[20]

Helin Wang, Yuexian Zou, Dading Chong, and Wenwu Wang. 2020. Modeling Label Dependencies for Audio Tagging With Graph Convolutional Network. IEEE Signal Processing Letters, Vol. 27 (2020), 1560--1564.

[21]

Hongming Zhang, Xin Liu, Haojie Pan, Yangqiu Song, and Cane Wing-Ki Leung. 2020. ASER: A large-scale eventuality knowledge graph. In Proceedings of The Web Conference 2020. 201--211.

Digital Library

[22]

Zhixian Zhang, Kenny Q Zhu, Haixun Wang, and Hongsong Li. 2013. Automatic extraction of top-k lists from the web. In 2013 IEEE 29th International Conference on Data Engineering (ICDE). IEEE, 1057--1068.

Digital Library

Cited By

Pittaras NGiannakopoulos GStamatopoulos PKarkaletsis V(2023)Content-based and Knowledge-enriched Representations for Classification Across Modalities: A SurveyACM Computing Surveys10.1145/358368255:14s(1-40)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3583682
Xu XZhang ZZhou ZZhang PXie ZWu MZhu KEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic DataProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613820(2756-2764)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613820
Zhou ZZhang ZXu XXie ZWu MZhu K(2022)Can Audio Captions Be Evaluated With Image Caption Metrics?ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9746427(981-985)Online publication date: 23-May-2022
https://doi.org/10.1109/ICASSP43922.2022.9746427

Index Terms

Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning

Recommendations

Meta-path reasoning of knowledge graph for commonsense question answering
Abstract
Commonsense question answering (CQA) requires understanding and reasoning over QA context and related commonsense knowledge, such as a structured Knowledge Graph (KG). Existing studies combine language models and graph neural networks to model ...
Knowledge Completion Method Based on Relational Embedding with GNN
Advanced Intelligent Computing Technology and Applications
Abstract
Knowledge graph, as an effective representation of structured knowledge, plays an increasingly important role in the field of artificial intelligence. However, the incompleteness and sparsity of knowledge graphs have become a bottleneck that ...
Sweet Apple, company? or food? Adjective-centric commonsense knowledge acquisition with taxonomy-guided induction
Abstract
How to enable intelligent machines to possess human commonsense knowledge is one of the central concerns of artificial intelligence. Consequently, a series of commonsense knowledge bases have been designed and constructed by scholars. However, as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

October 2021

4966 pages

ISBN:9781450384469

DOI:10.1145/3459637

General Chairs:
Gianluca Demartini
The University of Queensland, Australia
,
Guido Zuccon
The University of Queensland, Australia
,
Program Chairs:
J. Shane Culpepper
RMIT University, Australia
,
Zi Huang
The University of Queensland, Australia
,
Hanghang Tong
University of Illinois at Urbana-Champaign, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Natural Science Foundation of China

Conference

CIKM '21

Sponsor:

CIKM '21: The 30th ACM International Conference on Information and Knowledge Management

November 1 - 5, 2021

Queensland, Virtual Event, Australia

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
110
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pittaras NGiannakopoulos GStamatopoulos PKarkaletsis V(2023)Content-based and Knowledge-enriched Representations for Classification Across Modalities: A SurveyACM Computing Surveys10.1145/358368255:14s(1-40)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3583682
Xu XZhang ZZhou ZZhang PXie ZWu MZhu KEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic DataProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613820(2756-2764)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613820
Zhou ZZhang ZXu XXie ZWu MZhu K(2022)Can Audio Captions Be Evaluated With Image Caption Metrics?ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9746427(981-985)Online publication date: 23-May-2022
https://doi.org/10.1109/ICASSP43922.2022.9746427

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents