More Web Proxy on the site http://driver.im/

short-paper

On data driven parametric backchannel synthesis for expressing attentiveness in conversational agents

Authors:

Catharine Oertel,

Joakim Gustafson,

Alan W. BlackAuthors Info & Claims

MA3HMI '16: Proceedings of the Workshop on Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction

Pages 43 - 47

https://doi.org/10.1145/3011263.3011272

Published: 12 November 2016 Publication History

Abstract

In this study, we are using a multi-party recording as a template for building a parametric speech synthesiser which is able to express different levels of attentiveness in backchannel tokens. This allowed us to investigate i) whether it is possible to express the same perceived level of attentiveness in synthesised than in natural backchannels; ii) whether it is possible to increase and decrease the perceived level of attentiveness of backchannels beyond the range observed in the original corpus.

References

[1]

A. Black and P. Muthukumar. Random forests for statistical speech synthesis. In Interspeech 2015, Dresden, Germany, 2015.

[2]

A. W. Black. Clustergen: a statistical parametric synthesizer using trajectory modeling. In Interspeech 2006, 2006.

[3]

N. Campbell. Towards conversational speech synthesis; lessons learned from the expressive speech processing project. In SSW 2207, pages 22--27, Bonn, Germany, 2007.

[4]

K. Ehlich. Interjektionen. Max Niemeyer Verlag, 1986.

[5]

A. Hunt and A. Black. Unit selection in a concatenative speech synthesis system using a large speech database. In ICASSP-96, volume 1, pages 373--376, Atlanta, Georgia, 1996.

Digital Library

[6]

B. Inden, Z. Malisz, P. Wagner, and I. Wachsmuth. Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent. Proceedings of the 15th ACM on International conference on multimodal interaction, pages 181--188, 2013.

Digital Library

[7]

L.-P. Morency, I. de Kok, and J. Gratch. Predicting listener backchannels: A probabilistic multimodal approach. In Intelligent Virtual Agents, pages 176--190. Springer, 2008.

Digital Library

[8]

C. Oertel, K. A. Funes Mora, J. Gustafson, and J.-M. Odobez. Deciphering the silent participant: On the use of audio-visual cues for the classification of listener categories in group discussions. In International Conference on Multimodal Interaction. ACM, 2015.

Digital Library

[9]

C. Oertel, K. A. Funes Mora, S. Sheikhi, J.-M. Odobez, and J. Gustafson. Who will get the grant?: A multimodal corpus for the analysis of conversational behaviours in group interviews. In Proceedings of the 2014 Workshop on Understanding and Modeling Multiparty, Multimodal Interactions, UM3I '14, pages 27--32, 2014.

Digital Library

[10]

C. Oertel, J. Gustafson, and A. W. Black. Towards building an attentive artificial listener: On the perception of attentiveness in feedback utterances. In Proc. of Interspeech, pages 2915--2919, 2016.

[11]

S. C. Pammi, M. Schröder, M. Charfuelan, O. Türk, and I. Steiner. Synthesis of listener vocalisations with imposed intonation contours. In Seventh ISCA Tutorial and Research Workshop on Speech Synthesis. ISCA, ISCA, 2010.

[12]

T. Stocksmeier, S. Kopp, and D. Gibbon. Synthesis of prosodic attitudinal variants in german backchannel ja. In Interspeech 2007, pages 1290--1293, Antwerp, Belgium, 2007.

[13]

A. Syrdal, A. Conkie, Y. Kim, and M. Beutnagel. Speech acts and dialog tts. In SSW 7, Keihanna, Japan, 2010.

[14]

N. G. Ward. Possible lexical cues for backchannel responses. In Feedback Behaviors in Dialog, 2012.

[15]

N. G. Ward and R. Escalante-Ruiz. Using responsive prosodic variation to acknowledge the user's current state. In Interspeech 2009, Brighton, UK, 2009.

[16]

H. Zen, K. Tokuda, and A. Black. Statistical parametric speech synthesis. Speech Communication, 51(11):1059--1064, 2009.

Digital Library

Cited By

Bachan J(2022)Phonetic Convergence in a Prototype Dialogue SystemCurrent Issues in Descriptive Linguistics and Digital Humanities10.1007/978-981-19-2932-8_48(705-719)Online publication date: 1-Dec-2022
https://doi.org/10.1007/978-981-19-2932-8_48
Bachan JOwsianny MDemenko G(2020)The Harmonia Corpus – A Dialogue Corpus for Automatic Analysis of Phonetic ConvergenceHuman Language Technology. Challenges for Computer Science and Linguistics10.1007/978-3-030-66527-2_11(149-163)Online publication date: 31-Dec-2020
https://doi.org/10.1007/978-3-030-66527-2_11
Oertel CJonell PEl Haddad KSzekely EGustafson JChaminade TNgyuen NOchs MLefèvre F(2017)Using crowd-sourcing for the design of listening agents: challenges and opportunitiesProceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents10.1145/3139491.3139499(37-38)Online publication date: 13-Nov-2017
https://dl.acm.org/doi/10.1145/3139491.3139499
Show More Cited By

Index Terms

On data driven parametric backchannel synthesis for expressing attentiveness in conversational agents
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing
  2. Human computer interaction (HCI)
    1. Interaction techniques
      1. Auditory feedback

Recommendations

Towards building an attentive artificial listener: on the perception of attentiveness in audio-visual feedback tokens
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Current dialogue systems typically lack a variation of audio-visual feedback tokens. Either they do not encompass feedback tokens at all, or only support a limited set of stereotypical functions. However, this does not mirror the subtleties of ...
Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

We report on an analysis of feedback behavior in an Active Listening Corpus as produced verbally, visually (head movement) and bimodally. The behavior is modeled in an embodied conversational agent and displayed in a conversation with a real human to ...
Influence of personality traits on backchannel selection
IVA'10: Proceedings of the 10th international conference on Intelligent virtual agents

Our aim is to build a real-time Embodied Conversational Agent able to act as an interlocutor in interaction, generating automatically verbal and non verbal signals. These signals, called backchannels, provide information about the listener's mental ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MA3HMI '16: Proceedings of the Workshop on Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction

November 2016

64 pages

ISBN:9781450345620

DOI:10.1145/3011263

Conference Chairs:
Ronald Böck
University of Magdeburg, Germany
,
Francesca Bonin
IBM Research, Ireland
,
Nick Campbell
Trinity College Dublin, Ireland
,
Ronald Poppe
Utrecht University, Netherlands

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '16

ICMI '16: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 12 - 16, 2016

Tokyo, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
134
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bachan J(2022)Phonetic Convergence in a Prototype Dialogue SystemCurrent Issues in Descriptive Linguistics and Digital Humanities10.1007/978-981-19-2932-8_48(705-719)Online publication date: 1-Dec-2022
https://doi.org/10.1007/978-981-19-2932-8_48
Bachan JOwsianny MDemenko G(2020)The Harmonia Corpus – A Dialogue Corpus for Automatic Analysis of Phonetic ConvergenceHuman Language Technology. Challenges for Computer Science and Linguistics10.1007/978-3-030-66527-2_11(149-163)Online publication date: 31-Dec-2020
https://doi.org/10.1007/978-3-030-66527-2_11
Oertel CJonell PEl Haddad KSzekely EGustafson JChaminade TNgyuen NOchs MLefèvre F(2017)Using crowd-sourcing for the design of listening agents: challenges and opportunitiesProceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents10.1145/3139491.3139499(37-38)Online publication date: 13-Nov-2017
https://dl.acm.org/doi/10.1145/3139491.3139499
Bechade LEl Haddad KBourquin JDupont SDevillers LChaminade TNgyuen NOchs MLefèvre F(2017)A corpus for experimental study of affect bursts in human-robot interactionProceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents10.1145/3139491.3139496(20-21)Online publication date: 13-Nov-2017
https://dl.acm.org/doi/10.1145/3139491.3139496
Haddad K(2017)Nonverbal conversation expressions processing for human-agent interactions2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII)10.1109/ACII.2017.8273663(601-605)Online publication date: Oct-2017
https://doi.org/10.1109/ACII.2017.8273663

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents