extended-abstract

Am I Asking It Properly?: Designing and Evaluating Interview Chatbots to Improve Elicitation in an Ethical Way

Author:

Xu HanAuthors Info & Claims

IUI '20 Companion: Companion Proceedings of the 25th International Conference on Intelligent User Interfaces

Pages 33 - 34

https://doi.org/10.1145/3379336.3381509

Published: 17 March 2020 Publication History

Get Access

Abstract

The increasingly prevalent use of chatbots provides an innovative way to conduct conversation interviews. However, the lack of comprehensive design guidance and robust evaluation methodologies challenge designers in developing interview chatbots with good elicitation and ethics. This proposal presents our progress on investigating interview chatbots' performances in eliciting high-quality response without breaking any ethical rules, based on which an automatic evaluation framework will be developed. We also present our plan to propose design implications and prototypes for more robust interview chatbot development cycle.

References

[1]

Peter Allmark, Jonathan Boote, Eleni Chambers, Amanda Clarke, Ann McDonnell, Andrew Thompson, and Angela Mary Tod. 2009. Ethical Issues in the Use of In-Depth Interviews: Literature Review and Discussion. Research Ethics 5, 2 (June 2009), 48--54.

Crossref

Google Scholar

[2]

Frederick G Conrad, Mick P Couper, Roger Tourangeau, Mirta Galesic, and T Yan. 2005. Interactive feedback can improve the quality of responses in web surveys. In Proc. Surv. Res. Meth. Sect. Am. Statist. Ass. 3835--3840.

Google Scholar

[3]

H Paul Grice. 1975. Logic and conversation, syntax and semantics. Speech Acts 3 (1975), 41--58.

Crossref

Google Scholar

[4]

Jingyi Li, Michelle X Zhou, Huahai Yang, and Gloria Mark. 2018. Confiding in and Listening to Virtual Agents: The Effect of Personality. (Nov. 2018). arXiv:cs.AI/1811.00746

Google Scholar

[5]

Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv.1603.08023 (2016).

Google Scholar

[6]

Gale M Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37 (2014), 94--100.

Digital Library

Google Scholar

[7]

Marije Oudejans and Leah Melani Christian. 2011. Using interactive features to motivate and probe responses to open-ended questions. Social and behavioral research and the internet: Advances in applied methods and research strategies (2011), 215--44.

Google Scholar

[8]

Marilyn A Walker, Diane J Litman, Candace A Kamm, and Alicia Abella. 1997. PARADISE: A framework for evaluating spoken dialogue agents. arXiv preprint cmp-lg/9704004 (1997).

Google Scholar

[9]

Ziang Xiao, Michelle X Zhou, and Wat-Tat Fu. 2019. Who Should Be My Teammates: Using a Conversational Agent to Understand Individuals and Help Teaming. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI '19). ACM, New York, NY, USA, 437--447.

Digital Library

Google Scholar

[10]

Ziang Xiao, Michelle X Zhou, Q Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. 2019. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys. (May 2019). arXiv:cs.HC/1905.10700

Google Scholar

Cited By

View all

Coghlan SLeins KSheldrick SCheong MGooding PD'Alfonso S(2023)To chat or bot to chat: Ethical issues with using chatbots in mental healthDIGITAL HEALTH10.1177/205520762311835429Online publication date: 22-Jun-2023
https://doi.org/10.1177/20552076231183542
Doyle PRough DEdwards JCowan BClark LPorcheron MSchlögl STorres MMunteanu CMurad CSin JLee MAylett MCandello HVerbert KParra DHammond TKnijnenburg BO'Donovan JTaele P(2021)CUI@IUI: Theoretical and Methodological Challenges in Intelligent Conversational User Interface InteractionsCompanion Proceedings of the 26th International Conference on Intelligent User Interfaces10.1145/3397482.3450706(12-14)Online publication date: 14-Apr-2021
https://dl.acm.org/doi/10.1145/3397482.3450706
Suakanto SSiswanto JFebrianti Kusumasari TReza Prasetyo IHardiyanti M(2021)Interview Bot for Improving Human Resource Management2021 International Conference on ICT for Smart Society (ICISS)10.1109/ICISS53185.2021.9533248(1-5)Online publication date: 2-Aug-2021
https://doi.org/10.1109/ICISS53185.2021.9533248

Index Terms

Am I Asking It Properly?: Designing and Evaluating Interview Chatbots to Improve Elicitation in an Ethical Way
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Ethical Challenges in Data-Driven Dialogue Systems
AIES '18: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society

The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm. A growing number of dialogue systems use conversation strategies that are learned from large datasets. There are well documented instances where ...
Designing the Conversational Agent: Asking Follow-up Questions for Information Elicitation
CSCW

Conversational Agents (CAs) can facilitate information elicitation in various scenarios, such as semi-structured interviews. Current CAs can ask predetermined questions but lack skills for asking follow-up questions. Thus, we designed three approaches ...
Embodied conversational agents on a common ground
From brows to trust

One would like to rely on design guidelines for embodied conversational agents (ECAs), grounded on evaluation studies. How to define the physical and mental characteristics of an ECA, optimal for an envisioned application? What will be the added value ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IUI '20 Companion: Companion Proceedings of the 25th International Conference on Intelligent User Interfaces

March 2020

153 pages

ISBN:9781450375139

DOI:10.1145/3379336

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2020

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

IUI '20

Sponsor:

SIGAI
SIGCHI

IUI '20: 25th International Conference on Intelligent User Interfaces

March 17 - 20, 2020

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
308
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Coghlan SLeins KSheldrick SCheong MGooding PD'Alfonso S(2023)To chat or bot to chat: Ethical issues with using chatbots in mental healthDIGITAL HEALTH10.1177/205520762311835429Online publication date: 22-Jun-2023
https://doi.org/10.1177/20552076231183542
Doyle PRough DEdwards JCowan BClark LPorcheron MSchlögl STorres MMunteanu CMurad CSin JLee MAylett MCandello HVerbert KParra DHammond TKnijnenburg BO'Donovan JTaele P(2021)CUI@IUI: Theoretical and Methodological Challenges in Intelligent Conversational User Interface InteractionsCompanion Proceedings of the 26th International Conference on Intelligent User Interfaces10.1145/3397482.3450706(12-14)Online publication date: 14-Apr-2021
https://dl.acm.org/doi/10.1145/3397482.3450706
Suakanto SSiswanto JFebrianti Kusumasari TReza Prasetyo IHardiyanti M(2021)Interview Bot for Improving Human Resource Management2021 International Conference on ICT for Smart Society (ICISS)10.1109/ICISS53185.2021.9533248(1-5)Online publication date: 2-Aug-2021
https://doi.org/10.1109/ICISS53185.2021.9533248

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Ethical Challenges in Data-Driven Dialogue Systems

Designing the Conversational Agent: Asking Follow-up Questions for Information Elicitation

Embodied conversational agents on a common ground