research-article

Addressee identification for human-human-agent multiparty conversations in different proxemics

Authors:

Naoya Baba,

Hung-Hsuan Huang,

Yukiko I. NakanoAuthors Info & Claims

Gaze-In '12: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction

Article No.: 6, Pages 1 - 6

https://doi.org/10.1145/2401836.2401842

Published: 26 October 2012 Publication History

Get Access

Abstract

This paper proposes a method for identifying the addressee based on speech and gaze information, and shows that the proposed method can be applicable to human-human-agent multiparty conversations in different proxemics. First, we collected human-human-agent interaction in different proxemics, and by analyzing the data, we found that people spoke with a higher tone of voice and more loudly and slowly when they talked to the agent. We also confirmed that this speech style was consistent regardless of the proxemics. Then, by employing SVM, we proposed a general addressee estimation model that can be used in different proxemics, and the model achieved over 80% accuracy in 10-fold cross-validation.

References

[1]

Akker, R.o.d. and D. Traum. A comparison of addressee detection methods for multiparty conversations. in 13th Workshop on the Semantics and Pragmatics of Dialogue. 2009.

Google Scholar

[2]

Clark, H. H., Using Language. 1996, Cambridge: Cambridge University Press.

Google Scholar

[3]

Duncan, S., Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology, 1972. 23(2): p. 283--292.

Crossref

Google Scholar

[4]

Frampton, M., et al. Who is "You"? Combining Linguistic and Gaze Features to Resolve Second-Person References in Dialogue. in the 12th Conference of the European Chapter of the ACL. 2009.

Digital Library

Google Scholar

[5]

Huang, H.-H., N. Baba, and Y. Nakano. Making a Virtual Conversational Agent be Aware of the Addressee of Users' Utterances in Multi-user Conversation from Nonverbal Information. in the 13th International Conference on Multimodal Interaction (ICMI2011). 2011.

Digital Library

Google Scholar

[6]

Katzenmaier, M., R. Stiefelhagen, and T. Schultz. Identifying the Addressee in HumanHumanRobot Interactions based on Head Pose and Speech. in international Conference on Multimodal interfaces (ICMI04). 2004.

Digital Library

Google Scholar

[7]

Lunsford, R. and S. Oviatt. Human perception of intended addressee during computer-assisted meetings. in the 8th international Conference on Multimodal interfaces (ICMI06). 2006.

Digital Library

Google Scholar

[8]

Lunsford, R., S. Oviatt, and A. M. Arthur. Toward Open-Microphone Engagement for Multiparty Interactions. in the 8th international Conference on Multimodal interfaces (ICMI06). 2006.

Digital Library

Google Scholar

[9]

Takemae, Y., K. Otsuka, and N. Mukawa. Video cut editing rule based on participants' gaze in multiparty conversation. in the 11th ACM International Conference on Multimedia. 2003.

Digital Library

Google Scholar

[10]

Terken, J., I. Joris, and L.d. Valk. Multimodal Cues for Addressee-hood in Triadic Communication with a Human Information Retrieval Agent. in International Conference on Multimodal interfaces (ICMI2007). 2007.

Digital Library

Google Scholar

[11]

Vertegaal, R., et al. Eye gaze patterns in conversations: there is more the conversational agents than meets the eyes. in CHI 2001. 2001.

Digital Library

Google Scholar

Cited By

View all

Rashik MJasim MKucher KSarvghad AMahyar N(2024)Beyond Text and Speech in Conversational Agents: Mapping the Design Space of AvatarsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661563(1875-1894)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661563
Repiso EGarrell ASanfeliu A(2023)Real-Life Experiment Metrics for Evaluating Human-Robot Collaborative Navigation Tasks2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309529(660-667)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309529
Tesema FGu JSong WWu HZhu SLin ZHuang MWang WKumar R(2023)Addressee Detection Using Facial and Audio Features in Mixed Human–Human and Human–Robot Settings: A Deep Learning FrameworkIEEE Systems, Man, and Cybernetics Magazine10.1109/MSMC.2022.32248439:2(25-38)Online publication date: Apr-2023
https://doi.org/10.1109/MSMC.2022.3224843
Show More Cited By

Index Terms

Addressee identification for human-human-agent multiparty conversations in different proxemics
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

The effects of virtual agent humor and gaze behavior on human-virtual agent proxemics
IVA'11: Proceedings of the 10th international conference on Intelligent virtual agents

We study whether a virtual agent that delivers humor through verbal behavior can affect an individual's proxemic behavior towards the agent. Participants interacted with a virtual agent through natural language and, in a separate task, performed an ...
Human-robot collaborative tutoring using multiparty multimodal spoken dialogue
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

In this paper, we describe a project that explores a novel experimental setup towards building a spoken, multi-modally rich, and human-like multiparty tutoring robot. A human-robot interaction setup is designed, and a human-human dialogue corpus is ...
Identifying utterances addressed to an agent in multiparty human-agent conversations
IVA'11: Proceedings of the 10th international conference on Intelligent virtual agents

In multiparty human-agent interaction, the agent should be able to properly respond to a user by determining whether the utterance is addressed to the agent or to another person. This study proposes a model for predicting the addressee by using the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Gaze-In '12: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction

October 2012

88 pages

ISBN:9781450315166

DOI:10.1145/2401836

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Japan Society for the Promotion of Science

Conference

ICMI '12

Sponsor:

SIGCHI

ICMI '12: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 26, 2012

California, Santa Monica

Acceptance Rates

Overall Acceptance Rate 19 of 21 submissions, 90%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
188
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Rashik MJasim MKucher KSarvghad AMahyar N(2024)Beyond Text and Speech in Conversational Agents: Mapping the Design Space of AvatarsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661563(1875-1894)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661563
Repiso EGarrell ASanfeliu A(2023)Real-Life Experiment Metrics for Evaluating Human-Robot Collaborative Navigation Tasks2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309529(660-667)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309529
Tesema FGu JSong WWu HZhu SLin ZHuang MWang WKumar R(2023)Addressee Detection Using Facial and Audio Features in Mixed Human–Human and Human–Robot Settings: A Deep Learning FrameworkIEEE Systems, Man, and Cybernetics Magazine10.1109/MSMC.2022.32248439:2(25-38)Online publication date: Apr-2023
https://doi.org/10.1109/MSMC.2022.3224843
Siegert IWeißkirchen NKrüger JAkhtiamov OWendemuth A(2022)Admitting the addressee detection faultiness of voice assistants to improve the activation performance using a continuous learning frameworkCognitive Systems Research10.1016/j.cogsys.2021.07.00570:C(65-79)Online publication date: 22-Apr-2022
https://dl.acm.org/doi/10.1016/j.cogsys.2021.07.005
Akhtiamov OSiegert IKarpov AMinker W(2020)Using Complexity-Identical Human- and Machine-Directed Utterances to Investigate Addressee Detection for Spoken Dialogue SystemsSensors10.3390/s2009274020:9(2740)Online publication date: 11-May-2020
https://doi.org/10.3390/s20092740
Huang HKimura SKuwabara KNishida T(2020)Generation of Head Movements of a Robot Using Multimodal Features of Peer Participants in Group Discussion ConversationMultimodal Technologies and Interaction10.3390/mti40200154:2(15)Online publication date: 29-Apr-2020
https://doi.org/10.3390/mti4020015
Divekar RSu HKephart JDeBayser MGuerra MMou XPeveler MChen LBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)HUMAINE: Human Multi-Agent Immersive Negotiation CompetitionExtended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3334480.3383001(1-10)Online publication date: 25-Apr-2020
https://dl.acm.org/doi/10.1145/3334480.3383001
Siegert IKrüger J(2020)“Speech Melody and Speech Content Didn’t Fit Together”—Differences in Speech Behavior for Device Directed and Human Directed InteractionsAdvances in Data Science: Methodologies and Applications10.1007/978-3-030-51870-7_4(65-95)Online publication date: 27-Aug-2020
https://doi.org/10.1007/978-3-030-51870-7_4
Divekar RKephart JMou XChen LSu H(2019)You Talkin’ to Me? A Practical Attention-Aware Embodied AgentHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29387-1_44(760-780)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29387-1_44
Tsai TStolcke ASlaney M(2015)A Study of Multimodal Addressee Detection in Human-Human-Computer InteractionIEEE Transactions on Multimedia10.1109/TMM.2015.245433217:9(1550-1561)Online publication date: Sep-2015
https://doi.org/10.1109/TMM.2015.2454332
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

The effects of virtual agent humor and gaze behavior on human-virtual agent proxemics

Human-robot collaborative tutoring using multiparty multimodal spoken dialogue

Identifying utterances addressed to an agent in multiparty human-agent conversations