[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3343413.3377941acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
demonstration
Public Access

Wizard of Oz Interface to Study System Initiative for Conversational Search

Published: 14 March 2020 Publication History

Abstract

We describe a Wizard of Oz (WoZ) system, a WebApp, which we use to study how a conversational search system should take the initiative when engaging with users during collaborative search. This system integrates directly into Slack, a chat-messaging platform where users will collaborate. Through our system, the Wizard plays the role of a conversational search system that can search for information, send relevant web results, and message users. In our research, we study three Wizard conditions: bot\_info, bot\_dialog, and bot\_task, which differ in terms of how the Wizard can intervene in a conversation. The intervention modes follow the mixed-initiative framework by Chu-Carroll and Brown~\citechu1997tracking and provide us a foundation to study system initiative for conversational search. In this paper, we describe our design decisions and technical details on how we implemented the system.

References

[1]
Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 475--484.
[2]
Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, and Kaheer Suleman. 2017. Frames: A corpus for adding memory to goal-oriented dialogue systems. arXiv preprint arXiv:1704.00057 (2017).
[3]
Sandeep Avula, Jaime Arguello, Robert Capra, Jordan Dodson, Yuhui Huang, and Filip Radlinski. 2019. Embedding Search into a Conversational Platform to Support Collaborative Search. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval. ACM, 15--23.
[4]
Sandeep Avula, Gordon Chadwick, Jaime Arguello, and Robert Capra. 2018. SearchBots: User Engagement with ChatBots During Collaborative Search. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval. ACM, 52--61.
[5]
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gavs ić. 2018. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278 (2018).
[6]
Robert Capra, Annie T Chen, Katie Hawthorne, Jaime Arguello, Lee Shaw, and Gary Marchionini. 2012. Design and evaluation of a system to support collaborative search. Proceedings of the American Society for Information Science and Technology, Vol. 49, 1 (2012), 1--10.
[7]
Robert Capra, Gary Marchionini, Javier Velasco-Martin, and Katrina Muller. 2010. Tools-at-hand and learning in multi-session, collaborative search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 951--960.
[8]
Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 815--824.
[9]
Jennifer Chu-Carroll and Michael K Brown. 1997. Tracking initiative in collaborative dialogue interactions. In Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 262--270.
[10]
Laura Dietz, Ben Gamari, Jeff Dalton, and Nick Craswell. 2018. TREC Complex Answer Retrieval Overview. In TREC .
[11]
Mateusz Dubiel, Martin Halvey, Leif Azzopardi, and Sylvain Daronnat. 2018. Investigating how conversational search agents affect user's behaviour, performance and search experience. In The Second International Workshop on Conversational Approaches to Information Retrieval .
[12]
Mihail Eric and Christopher D Manning. 2017. Key-value retrieval networks for task-oriented dialogue. arXiv preprint arXiv:1705.05414 (2017).
[13]
Gene Golovchinsky, Abdigani Diriye, and Jeremy Pickens. 2011. Designing for collaboration in information seeking. Proc. HCIR (2011).
[14]
Helia Hashemi, Mohammad Aliannejadi, Hamed Zamani, and W Bruce Croft. 2019. ANTIQUE: A Non-Factoid Question Answering Benchmark. arXiv preprint arXiv:1905.08957 (2019).
[15]
Marti A Hearst. 2014. What's missing from collaborative search? Computer, Vol. 47, 3 (2014), 58--61.
[16]
Diane Kelly et almbox. 2009. Methods for evaluating interactive information retrieval systems with users. Foundations and Trends® in Information Retrieval, Vol. 3, 1--2 (2009), 1--224.
[17]
Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In Advances in Neural Information Processing Systems. 9725--9735.
[18]
Q Vera Liao, Mas-ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaeni, Marco Patricio Crasso, Dakuo Wang, Michael Muller, N Sadat Shami, Werner Geyer, et almbox. 2018. All Work and No Play?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 3.
[19]
Meredith Ringel Morris. 2013. Collaborative search revisited. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 1181--1192.
[20]
Meredith Ringel Morris and Eric Horvitz. 2007. SearchTogether: an interface for collaborative web search. In Proceedings of the 20th annual ACM symposium on User interface software and technology. ACM, 3--12.
[21]
Sharoda A Paul and Meredith Ringel Morris. 2009. CoSense: enhancing sensemaking for collaborative web search. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1771--1780.
[22]
Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proceedings of the 2017 conference on conference human information interaction and retrieval. ACM, 117--126.
[23]
Chirag Shah. 2010. Coagmento-a collaborative information seeking, synthesis and sense-making framework. Integrated demo at CSCW, Vol. 2010 (2010).
[24]
Ameneh Shamekhi, Q Vera Liao, Dakuo Wang, Rachel KE Bellamy, and Thomas Erickson. 2018. Face Value? Exploring the effects of embodiment for a group facilitation agent. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 391.
[25]
Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 235--244.
[26]
Johanne R Trippas, Damiano Spina, Lawrence Cavedon, Hideo Joho, and Mark Sanderson. 2018. Informing the design of spoken conversational search: perspective paper. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval. ACM, 32--41.
[27]
Johanne R Trippas, Damiano Spina, Lawrence Cavedon, and Mark Sanderson. 2017. How do people interact in conversational speech-only search tasks: A preliminary analysis. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. ACM, 325--328.
[28]
Alexandra Vtyurina, Denis Savenkov, Eugene Agichtein, and Charles LA Clarke. 2017. Exploring conversational search with humans, assistants, and wizards. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2187--2193.
[29]
Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2016. A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562 (2016).
[30]
Zhen Yue, Shuguang Han, and Daqing He. 2012. An investigation of search processes in collaborative exploratory web search. Proceedings of the American Society for Information Science and Technology, Vol. 49, 1 (2012), 1--4.
[31]
Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management . ACM, 177--186.

Cited By

View all
  • (2024)CoSearchAgent: A Lightweight Collaborative Search Agent with Large Language ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657672(2729-2733)Online publication date: 10-Jul-2024
  • (2024)Interactions with Generative Information Retrieval SystemsInformation Access in the Era of Generative AI10.1007/978-3-031-73147-1_3(47-71)Online publication date: 12-Sep-2024
  • (2023)Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple WizardsProceedings of the ACM on Human-Computer Interaction10.1145/35795917:CSCW1(1-34)Online publication date: 16-Apr-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CHIIR '20: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval
March 2020
596 pages
ISBN:9781450368926
DOI:10.1145/3343413
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2020

Check for updates

Author Tags

  1. collaborative search
  2. conversational search
  3. mixed-initiative interactions
  4. wizard of oz

Qualifiers

  • Demonstration

Funding Sources

Conference

CHIIR '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)130
  • Downloads (Last 6 weeks)11
Reflects downloads up to 11 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)CoSearchAgent: A Lightweight Collaborative Search Agent with Large Language ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657672(2729-2733)Online publication date: 10-Jul-2024
  • (2024)Interactions with Generative Information Retrieval SystemsInformation Access in the Era of Generative AI10.1007/978-3-031-73147-1_3(47-71)Online publication date: 12-Sep-2024
  • (2023)Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple WizardsProceedings of the ACM on Human-Computer Interaction10.1145/35795917:CSCW1(1-34)Online publication date: 16-Apr-2023
  • (2021)Examining Usability on Atreya Bot: A Chatbot Designed for Chemical Scientists2021 International Conference on Computational Performance Evaluation (ComPE)10.1109/ComPE53109.2021.9752288(729-733)Online publication date: 1-Dec-2021

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media