Report on the SIGDial 2021 special session on summarization of dialogues and multi-party meetings (SummDial)
Article No.: 12, Pages 1 - 17
Abstract
The SummDial special session on summarization of dialogues and multi-party meetings was held virtually within the SIGDial 2021 conference on July 29, 2021. SummDial @ SIGDial 2021 aimed to bring together the speech, dialogue, and summarization communities to foster cross-pollination of ideas and fuel the discussions/collaborations to attempt this crucial and timely problem. When the pandemic has restricted most of our in-person interactions, the current scenario has forced people to go virtual, resulting in an information overload from frequent dialogues and meetings in the virtual environment. Summarization could help reduce the cognitive burden on the participants; however, multi-party speech summarization comes with its own set of challenges. The SummDial special session aimed to leverage the community intelligence to find effective solutions while also brainstorming the future of AI interventions in meetings and dialogues. We report the findings of the special session in this article. We organized the SummDial special session under the aegis of the EU-funded H2020 European Live Translator (ELITR) project.1
Date: 29 July, 2021.
Website: https://elitr.github.io/automatic-minuting/summdial.html.
References
[1]
Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, Pengfei Liu, and Graham Neubig. Re-evaluating evaluation in text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9347--9359, Online, November 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020.emnlp-main.751.
[2]
Ondřej Bojar, Dominik Macháček, Sangeet Sagar, Otakar Smrž, Jonáš Kratochvíl, Ebrahim Ansari, Dario Franceschini, Chiara Canton, Ivan Simonini, Thai-Son Nguyen, Felix Schneider, Sebastian Stücker, Alex Waibel, Barry Haddow, Rico Sennrich, and Philip Williams. ELITR: European live translator. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 463--464, Lisboa, Portugal, November 2020. European Association for Machine Translation. URL https://aclanthology.org/2020.eamt-1.53.
[3]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6--12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html.
[4]
Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. DialogSum: A real-life scenario dialogue summarization dataset. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062--5074, Online, August 2021a. Association for Computational Linguistics. URL https://aclanthology.org/2021.findings-acl.449.
[5]
Yulong Chen, Yang Liu, and Yue Zhang. Dialogsum challenge: Summarizing real-life scenario dialogues. In Anya Belz, Angela Fan, Ehud Reiter, and Yaji Sripada, editors, Proceedings of the 14th International Conference on Natural Language Generation, INLG 2021, Aberdeen, Scotland, UK, 20--24 September, 2021, pages 308--313. Association for Computational Linguistics, 2021b. URL https://aclanthology.org/2021.inlg-1.33.
[6]
Ann Clifton, Sravana Reddy, Yongze Yu, Aasish Pappu, Rezvaneh Rezapour, Hamed Bonab, Maria Eskevich, Gareth Jones, Jussi Karlgren, Ben Carterette, and Rosie Jones. 100,000 pod-casts: A spoken English document corpus. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5903--5917, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. URL https://aclanthology.org/2020.coling-main.519.
[7]
Daniel Deutsch and Dan Roth. SacreROUGE: An open-source library for using and developing summarization evaluation metrics. In Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS), pages 120--125, Online, November 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020.nlposs-1.17.
[8]
Daniel Deutsch and Dan Roth. Understanding the extent to which content quality metrics measure the information quality of summaries. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 300--309, Online, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.conll-1.24.
[9]
Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. SummEval: Re-evaluating Summarization Evaluation. Transactions of the Association for Computational Linguistics, 9:391--409, 04 2021. ISSN 2307-387X. URL https://doi.org/10.1162/tacl_a_00373.
[10]
Tirthankar Ghosal, Muskaan Singh, Anja Nedoluzhko, and Ondřej Bojar. Overview of the First Shared Task on Automatic Minuting (AutoMin) at Interspeech 2021. In Proceedings of the First Shared Task on Automatic Minuting at Interspeech 2021, 2021.
[11]
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70--79, Hong Kong, China, November 2019. Association for Computational Linguistics. URL https://aclanthology.org/D19-5409.
[12]
Adam Janin, Don Baron, Jane Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, and Chuck Wooters. The ICSI meeting corpus. In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '03, Hong Kong, April 6--10, 2003, pages 364--367. IEEE, 2003. 2003.1198793. URL https://doi.org/10.1109/ICASSP.2003.1198793.
[13]
Mladen Karan, Prashant Khare, Patrick Healey, and Matthew Purver. Mitigating topic bias when detecting decisions in dialogue. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 542--547, Singapore and Online, July 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.sigdial-1.56.
[14]
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871--7880, Online, July 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020.acl-main.703.
[15]
Chin-Yew Lin and Eduard H. Hovy. Automatic evaluation of summaries using n-gram cooccurrence statistics. In Marti A. Hearst and Mari Ostendorf, editors, Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, HLT-NAACL 2003, Edmonton, Canada, May 27 - June 1, 2003. The Association for Computational Linguistics, 2003. URL https://aclanthology.org/N03-1020/.
[16]
Zhengyuan Liu and Nancy F. Chen. Dynamic sliding window for meeting summarization. CoRR, abs/2108.13629, 2021. URL https://arxiv.org/abs/2108.13629.
[17]
Zhengyuan Liu, Ke Shi, and Nancy Chen. Coreference-aware dialogue summarization. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 509--519, Singapore and Online, July 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.sigdial-1.53.
[18]
Annie Louis and Ani Nenkova. Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2):267--300, June 2013. URL https://aclanthology.org/J13-2002.
[19]
Ramesh Manuvinakurike, Saurav Sahay, Wenda Chen, and Lama Nachman. Incremental temporal summarization in multi-party meetings. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 530--541, Singapore and Online, July 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.sigdial-1.55.
[20]
Iain McCowan, Jean Carletta, Wessel Kraaij, Simone Ashby, S Bourban, M Flynn, M Guillemot, Thomas Hain, J Kadlec, Vasilis Karaiskos, et al. The ami meeting corpus. In Proceedings of the 5th international conference on methods and techniques in behavioral research, volume 88, page 100. Citeseer, 2005. URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.95.6326.
[21]
Anna Nedoluzhko and Ondrej Bojar. Towards automatic minuting of the meetings. In ITAT, 2019. URL http://ceur-ws.org/Vol-2473/paper3.pdf.
[22]
Martin Popel, Marketa Tomkova, Jakub Tomek, Lukasz Kaiser, Jakob Uszkoreit, Ondřej Bojar, and Zdeněk Žabokrtskỳ. Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals. Nature communications, 11(1): 1--15, 2020. URL https://www.nature.com/articles/s41467-020-18073-9.
[23]
Muskaan Singh, Tirthankar Ghosal, and Ondřej Bojar. An empirical analysis of text summarization approaches for automatic minuting. In Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation, Shanghai, China, November 2021. Association for Computational Linguistics.
[24]
Gökhan Tür, Andreas Stolcke, L. Lynn Voss, Stanley Peters, Dilek Hakkani-Tür, John Dowding, Benoît Favre, Raquel Fernández, Matthew Frampton, Michael W. Frandsen, Clint Frederickson, Martin Graciarena, Donald Kintzing, Kyle Leveque, Shane Mason, John Niekrasz, Matthew Purver, Korbinian Riedhammer, Elizabeth Shriberg, Jing Tien, Dimitra Vergyri, and Fan Yang. The CALO meeting assistant system. IEEE Trans. Speech Audio Process., 18(6): 1601--1611,2010. URL https://doi.org/10.1109/TASL.2009.2038810.
[25]
Klaus Zechner. Automatic generation of concise summaries of spoken dialogues in unrestricted domains. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, September 9--13, 2001, New Orleans, Louisiana, USA, pages 199--207. ACM, 2001a. URL https://doi.org/10.1145/383952.383989.
[26]
Klaus Zechner. Automatic summarization of spoken dialogues in unrestricted domains. 2001b. URL https://isl.anthropomatik.kit.edu/downloads/Zechner_Klaus_thesis.pdf.
[27]
Klaus Zechner. Automatic summarization of open-domain multiparty dialogues in diverse genres. Comput. Linguistics, 28(4):447--485, 2002a. URL https://doi.org/10.1162/089120102762671945.
[28]
Klaus Zechner. Summarization of spoken language-challenges, methods, and prospects. Speech technology expert eZine, 6, 2002b. URL http://www.cs.cmu.edu/~./zechner/ezine.ps.
[29]
Klaus Zechner and Alex Waibel. DIASUMM: flexible summarization of spontaneous dialogues in unrestricted domains. In COLING 2000, 18th International Conference on Computational Linguistics, Proceedings of the Conference, 2 Volumes, July 31 - August 4, 2000, Universität des Saarlandes, Saarbrücken, Germany, pages 968--974. Morgan Kaufmann, 2000. URL https://aclanthology.org/C00-2140/.
[30]
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=SkeHuCVFDr.
[31]
Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. QMSum: A new benchmark for query-based multi-domain meeting summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5905--5921, Online, June 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.naacl-main.472.
[32]
Chenguang Zhu, Yang Liu, Jie Mei, and Michael Zeng. MediaSum: A large-scale media interview dataset for dialogue summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5927--5934, Online, June 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.naacl-main.474.
[33]
Yingying Zhuang, Yichao Lu, and Simi Wang. Weakly supervised extractive summarization with attention. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 520--529, Singapore and Online, July 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.sigdial-1.54.
- Report on the SIGDial 2021 special session on summarization of dialogues and multi-party meetings (SummDial)
Recommendations
Report on supporting and understanding of conversational dialogues workshop (SUD 2021) at WSDM 2021
This report describes the workshop on Supporting and Understanding of (multi-party) conversational Dialogues (SUD) organized as a part of the Web Search and Data Mining conference (WSDM) 2021. The aim of SUD workshop was to encourage researchers to ...
Exploring the characteristics of multi-party dialogues
ACL '98/COLING '98: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 1This paper describes novel results on the characteristics of three-party dialogues by quantitatively comparing them with those of two-party. In previous dialogue research, two-party dialogues are mainly focussed because data collection of multi-party ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2022 Copyright is held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 March 2022
Published in SIGIR Volume 55, Issue 2
Check for updates
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 114Total Downloads
- Downloads (Last 12 months)13
- Downloads (Last 6 weeks)2
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in