More Web Proxy on the site http://driver.im/

research-article

Investigating Proactive Search Support in Conversations

Authors:

Salvatore Andolina,

Hendrik Schneider,

Khalil Klouche,

Tuukka Ruotsalo,

Luciano Gamberini,

Giulio JacucciAuthors Info & Claims

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

Pages 1295 - 1307

https://doi.org/10.1145/3196709.3196734

Published: 08 June 2018 Publication History

Abstract

Conversations among people involve solving disputes, building common ground, and reinforce mutual beliefs and assumptions. Conversations often require external information that can support these human activities. In this paper, we study how a spoken conversation can be supported by a proactive search agent that listens to the conversation, detects entities mentioned in the conversation, and proactively retrieves and presents information related to the conversation. A total of 24 participants (12 pairs) were involved in informal conversations, using either the proactive search agent or a control condition that did not support conversational analysis or proactive information retrieval. Data comprising transcripts, interaction logs, questionnaires, and interviews indicated that the proactive search agent effectively augmented the conversations, affected the conversations' topical structure, and reduced the need for explicit search activity. The findings also revealed key challenges in the design of proactive search systems that assist people in natural conversations.

Supplementary Material

suppl.mov (disfp255.mp4)

Supplemental video

Download
20.51 MB

References

[1]

Jesper Aagaard. 2016. Mobile devices, interaction, and distraction: a qualitative exploration of absent presence. AI & SOCIETY 31, 2 (01 May 2016), 223--231.

Digital Library

[2]

Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Trans. on Knowl. and Data Eng. 17, 6 (June 2005), 734--749.

Digital Library

[3]

Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving Web Search Ranking by Incorporating User Behavior Information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '06). ACM, New York, NY, USA, 19--26.

Digital Library

[4]

Salvatore Andolina, Khalil Klouche, Diogo Cabral, Tuukka Ruotsalo, and Giulio Jacucci. 2015a. InspirationWall: Supporting Idea Generation Through Automatic Information Exploration. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (C&C '15). ACM, New York, NY, USA, 103--106.

Digital Library

[5]

Salvatore Andolina, Khalil Klouche, Jaakko Peltonen, Mohammad Hoque, Tuukka Ruotsalo, Diogo Cabral, Arto Klami, Dorota Głowacka, Patrik Flor´ een, and Giulio Jacucci. 2015b. IntentStreams: Smart Parallel Search Streams for Branching Exploratory Search. In Proceedings of the 20th International Conference on Intelligent User Interfaces. ACM, 300--305.

Digital Library

[6]

Salvatore Andolina, Hendrik Schneider, Joel Chan, Khalil Klouche, Giulio Jacucci, and Steven Dow. 2017. Crowdboard: Augmenting In-Person Idea Generation with Real-Time Crowds. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C&C '17). ACM, New York, NY, USA, 106--118.

Digital Library

[7]

Tony Bergstrom and Karrie Karahalios. 2009. Conversation Clusters: Grouping Conversation Topics Through Human-computer Dialog. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 2349--2352.

Digital Library

[8]

Dan Bohus and Eric Horvitz. 2009. Dialog in the Open World: Platform and Applications. In Proceedings of the 2009 International Conference on Multimodal Interfaces (ICMI-MLMI '09). ACM, New York, NY, USA, 31--38.

Digital Library

[9]

Barry Brown, Moira McGregor, and Donald McMillan. 2015. Searchable Objects: Search in Everyday Conversation. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 508--517.

Digital Library

[10]

Huanhuan Cao, Derek Hao Hu, Dou Shen, Daxin Jiang, Jian-Tao Sun, Enhong Chen, and Qiang Yang. 2009. Context-aware Query Classification. In Proceedings of the 32Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '09). ACM, New York, NY, USA, 3--10.

Digital Library

[11]

David Carmel, Naama Zwerdling, Ido Guy, Shila Ofek-Koifman, Nadav Har'el, Inbal Ronen, Erel Uziel, Sivan Yogev, and Sergey Chernov. 2009. Personalized Social Search Based on the User's Social Network. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM '09). ACM, New York, NY, USA, 1227--1236.

Digital Library

[12]

Zhicong Cheng, Bin Gao, and Tie-Yan Liu. 2010. Actively predicting diverse search intent from user browsing behaviors. In Proceedings of the 19th international conference on World wide web. ACM, 221--230.

Digital Library

[13]

Anind K. Dey, Gerd Kortuem, David R. Morse, and Albrecht Schmidt. 2001. Situated Interaction and Context-Aware Computing. Personal Ubiquitous Comput. 5, 1 (Jan. 2001), 1--3.

Digital Library

[14]

Joan Morris DiMicco and Walter Bender. 2004. Second messenger: increasing the visibility of minority viewpoints with a face-to-face collaboration tool. In Proceedings of the 9th international conference on Intelligent user interfaces. ACM, 232--234.

Digital Library

[15]

Joan Morris DiMicco, Anna Pandolfo, and Walter Bender. 2004. Influencing group participation with a shared display. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. ACM, 614--623.

Digital Library

[16]

Desmond Elliott and Joemon M. Jose. 2009. A Proactive Personalised Retrieval System. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM '09). ACM, New York, NY, USA, 1935--1938.

Digital Library

[17]

Greg Guest. 2014. Public health research methods. Sage Publications.

[18]

Ramanathan Guha, Vineet Gupta, Vivek Raghunathan, and Ramakrishnan Srikant. 2015. User Modeling for a Personal Assistant. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM '15). ACM, New York, NY, USA, 275--284.

Digital Library

[19]

Qi Guo and Eugene Agichtein. 2010. Ready to Buy or Just Browsing?: Detecting Web Searcher Goals from Interaction Data. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '10). ACM, New York, NY, USA, 130--137.

Digital Library

[20]

Monika Henzinger, Bay-Wei Chang, Brian Milch, and Sergey Brin. 2005. Query-Free News Search. World Wide Web 8, 2 (2005), 101--126.

Digital Library

[21]

Jonathan Kilgour, Jean Carletta, and Steve Renals. 2010. The Ambient Spotlight: Queryless Desktop Search from Meeting Speech. In Proceedings of the 2010 International Workshop on Searching Spontaneous Conversational Speech (SSCS '10). ACM, New York, NY, USA, 49--52.

Digital Library

[22]

David Kirsh. 2014. The importance of chance and interactivity in creativity. Pragmatics & Cognition 22, 1 (2014), 5--26.

[23]

Weize Kong, Rui Li, Jie Luo, Aston Zhang, Yi Chang, and James Allan. 2015. Predicting Search Intent Based on Pre-Search Context. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '15). ACM, New York, NY, USA, 503--512.

Digital Library

[24]

Henry Lieberman. 1995. Letizia: An agent that assists web browsing. In Proc. IJCAI (1). 924--929.

Digital Library

[25]

Daniel J. Liebling, Paul N. Bennett, and Ryen W. White. 2012. Anticipatory Search: Using Context to Initiate Search. In Proc. SIGIR. ACM, 1035--1036.

Digital Library

[26]

Moira McGregor and John C. Tang. 2017. More to Meetings: Challenges in Using Speech-Based Technology to Support Meetings. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 2208--2220.

Digital Library

[27]

Donald McMillan, Antoine Loriette, and Barry Brown. 2015. Repurposing Conversation: Experiments with the Continuous Speech Stream. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3953--3962.

Digital Library

[28]

Michael F McTear. 2002. Spoken dialogue technology: enabling the conversational user interface. ACM Computing Surveys (CSUR) 34, 1 (2002), 90--169.

Digital Library

[29]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. CoRR abs/1301.3781 (2013). http://arxiv.org/abs/1301.3781

[30]

Takashi Nakamura. 2015. The action of looking at a mobile phone display as nonverbal behavior/communication: A theoretical perspective. Computers in Human Behavior 43 (2015), 68 -- 75.

Digital Library

[31]

Matteo Negri, Marco Turchi, Jos´ e G. C. de Souza, and Daniele Falavigna. 2014. Quality Estimation for Automatic Speech Recognition. In COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, August 23--29, 2014, Dublin, Ireland. 1813--1823. http://aclweb.org/anthology/C/C14/C14--1171.pdf

[32]

Martin Porcheron, Joel E. Fischer, and Sarah Sharples. 2016. Using Mobile Phones in Pub Talk. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 1649--1661.

Digital Library

[33]

Pearl Pu, Li Chen, and Rong Hu. 2012. Evaluating recommender systems from the user's perspective: survey of the state of the art. User Modeling and User-Adapted Interaction 22, 4 (01 Oct 2012), 317--355.

Digital Library

[34]

Paul Resnick and Hal R. Varian. 1997. Recommender Systems. Commun. ACM 40, 3 (March 1997), 56--58.

Digital Library

[35]

Bradley Rhodes and Thad Starner. 1996. Remembrance Agent: A continuously running automated information retrieval system. In The Proceedings of The First International Conference on The Practical Application Of Intelligent Agents and Multi Agent Technology. 487--495.

[36]

B. J. Rhodes and P. Maes. 2000. Just-in-time Information Retrieval Agents. IBM Syst. J. 39, 3--4 (July 2000), 685--704.

Digital Library

[37]

Tuukka Ruotsalo, Giulio Jacucci, Petri Myllym¨ aki, and Samuel Kaski. 2014. Interactive Intent Modeling: Information Discovery Beyond Search. Commun. ACM 58, 1 (Dec. 2014), 86--92.

Digital Library

[38]

G. Salton, A. Wong, and C. S. Yang. 1975. A Vector Space Model for Automatic Indexing. Commun. ACM 18, 11 (Nov. 1975), 613--620.

Digital Library

[39]

Gianluca Schiavo, Alessandro Cappelletti, Eleonora Mencarini, Oliviero Stock, and Massimo Zancanaro. 2014. Overt or subtle? Supporting group conversations with automatically targeted directives. In Proceedings of the 19th international conference on Intelligent User Interfaces. ACM, 225--234.

Digital Library

[40]

Yang Shi, Yang Wang, Ye Qi, John Chen, Xiaoyao Xu, and Kwan-Liu Ma. 2017. IdeaWall: Improving Creative Collaboration Through Combinatorial Visual Stimuli. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 594--603.

Digital Library

[41]

Sosuke Shiga, Hideo Joho, Roi Blanco, Johanne R. Trippas, and Mark Sanderson. 2017. Modelling Information Needs in Collaborative Search Conversations. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). ACM, New York, NY, USA, 715--724.

Digital Library

[42]

Yang Song and Qi Guo. 2016. Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines. In Proceedings of the 25th International Conference on World Wide Web (WWW '16). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 543--553.

Digital Library

[43]

Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2005. Personalizing Search via Automated Analysis of Interests and Activities. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '05). ACM, New York, NY, USA, 449--456.

Digital Library

[44]

Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2010. Potential for Personalization. ACM Trans. Comput.-Hum. Interact. 17, 1, Article 4 (April 2010), 31 pages.

Digital Library

[45]

Jaime Teevan, Amy Karlson, Shahriyar Amini, A. J. Bernheim Brush, and John Krumm. 2011. Understanding the Importance of Location, Time, and People in Mobile Local Search Behavior. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 77--80.

Digital Library

[46]

Tung Vuong, Giulio Jacucci, and Tuukka Ruotsalo. 2017a. Proactive Information Retrieval via Screen Surveillance. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). ACM, New York, NY, USA, 1313--1316.

Digital Library

[47]

Tung Vuong, Giulio Jacucci, and Tuukka Ruotsalo. 2017b. Watching Inside the Screen: Digital Activity Monitoring for Task Recognition and Proactive Information Retrieval. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 109 (Sept. 2017), 23 pages.

Digital Library

[48]

Ryen W. White and Resa A. Roth. 2009. Exploratory Search: Beyond the Query-Response Paradigm. Morgan & Claypool Publishers, {San Rafael, Calif.}.

[49]

Biao Xiang, Daxin Jiang, Jian Pei, Xiaohui Sun, Enhong Chen, and Hang Li. 2010. Context-aware ranking in web search. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, 451--458.

Digital Library

Cited By

Samarinas CZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)ProCIS: A Benchmark for Proactive Retrieval in ConversationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657869(830-840)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657869
Zulfikar WChan SMaes P(2024)Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory AugmentationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642450(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642450
Tabalba RKirshenbaum NLeigh JBhattacharya AGrosso VDi Eugenio BJohnson AZellner M(2023)An Investigation into an Always Listening Interface to Support Data ExplorationProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584079(128-141)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584079
Show More Cited By

Index Terms

Investigating Proactive Search Support in Conversations
1. Human-centered computing

Recommendations

SearchBot: Supporting Voice Conversations with Proactive Search
CSCW '18 Companion: Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing

Searching during conversations and social interactions is becoming increasingly common. Although searching could be helpful for solving arguments, building common ground, and reinforcing mutual assumptions, it can also cause interactional problems. ...
Investigating the Intervention in Parallel Conversations
HAI '23: Proceedings of the 11th International Conference on Human-Agent Interaction

In recent years, a framework of parallel conversations has been proposed to facilitate efficient conversations through cooperation between humans and dialogue systems. This approach aims to enable simultaneous conversations with multiple users by ...
Spoken Conversational Context Improves Query Auto-completion in Web Search
Web searches often originate from conversations in which people engage before they perform a search. Therefore, conversations can be a valuable source of context with which to support the search process. We investigate whether spoken input from ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

June 2018

1418 pages

ISBN:9781450351980

DOI:10.1145/3196709

General Chairs:
Ilpo Koskinen
University of Twente
,
Youn-kyung Lim
KAIST
,
Program Chairs:
Teresa Cerratto-Pargman
Stockholm University
,
Kenny Chow
The Hong Kong Polytechnic University
,
William Odom
Simon Fraser University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Academy of Finland

Conference

DIS '18

Sponsor:

SIGCHI

DIS '18: Designing Interactive Systems Conference 2018

June 9 - 13, 2018

Hong Kong, China

Acceptance Rates

DIS '18 Paper Acceptance Rate 107 of 487 submissions, 22%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
523
Total Downloads

Downloads (Last 12 months)70
Downloads (Last 6 weeks)10

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Samarinas CZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)ProCIS: A Benchmark for Proactive Retrieval in ConversationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657869(830-840)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657869
Zulfikar WChan SMaes P(2024)Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory AugmentationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642450(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642450
Tabalba RKirshenbaum NLeigh JBhattacharya AGrosso VDi Eugenio BJohnson AZellner M(2023)An Investigation into an Always Listening Interface to Support Data ExplorationProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584079(128-141)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584079
Ros KJin MLevine JZhai CYoshioka MKiseleva JAliannejadi M(2023)Retrieving Webpages Using Online DiscussionsProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605139(159-168)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605139
Meck ASardone MCullmann J(2023)Will the Assistant Become the Driver, and the Driver Become the Assistant?Proceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603753(1-5)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603753
Liu XKirilyuk VYuan XOlwal AChi PChen XDu R(2023)Visual Captions: Augmenting Verbal Communication with On-the-fly VisualsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581566(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581566
Meck ADraxler CVogt T(2023)How May I Interrupt? Linguistic-Driven Design Guidelines for Proactive in-Car Voice AssistantsInternational Journal of Human–Computer Interaction10.1080/10447318.2023.226625140:22(7517-7531)Online publication date: 15-Oct-2023
https://doi.org/10.1080/10447318.2023.2266251
Getchell KCarradini SCardon PFleischmann CMa HAritz JStapp J(2022)Artificial Intelligence in Business Communication: The Changing Landscape of Research and TeachingBusiness and Professional Communication Quarterly10.1177/2329490622107431185:1(7-33)Online publication date: 3-Feb-2022
https://doi.org/10.1177/23294906221074311
Malkin NWagner DEgelman S(2022)Can Humans Detect Malicious Always-Listening Assistants? A Framework for Crowdsourcing Test DrivesProceedings of the ACM on Human-Computer Interaction10.1145/35556136:CSCW2(1-28)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555613
Dubiel MDaronnat SLeiva L(2022)Conversational Agents Trust CalibrationProceedings of the 4th Conference on Conversational User Interfaces10.1145/3543829.3544518(1-6)Online publication date: 26-Jul-2022
https://dl.acm.org/doi/10.1145/3543829.3544518
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents