More Web Proxy on the site http://driver.im/

research-article

Towards Predicting a Realisation of an Information Need based on Brain Signals

Authors:

Yashar Moshfeghi,

Peter Triantafillou,

Frank PollickAuthors Info & Claims

WWW '19: The World Wide Web Conference

Pages 1300 - 1309

https://doi.org/10.1145/3308558.3313671

Published: 13 May 2019 Publication History

Abstract

The goal of Information Retrieval (IR) systems is to satisfy searchers' Information Need (IN). Our research focuses on next-generation IR engines, which can proactively detect, identify, and serve INs without receiving explicit queries. It is essential, therefore, to be able to detect when INs occur. Previous research has established that a realisation of INs physically manifests itself with specific brain activity. With this work we take the next step, showing that monitoring brain activity can lead to accurate predictions of a realisation of IN occurrence. We have conducted experiments whereby twenty-four participants performed a Q/A Task, while their brain activity was being monitored using functional Magnetic Resonance Imaging (fMRI) technology. The questions were selected and developed from the TREC-8 and TREC 2001 Q/A Tracks. We present two methods for predicting the realisation of an IN, i.e. Generalised method (GM) and Personalised method (PM). GM is based on the collective brain activity of all twenty-four participants in a predetermined set of brain regions known to be involved in representing a realisation of INs. PM is unique to each individual and employs a 'Searchlight' analysis to locate brain regions informative for distinguishing when a “specific” user realises an information need. The results of our study show that both methods were able to predict a realisation of an IN (statistically) significantly better than chance. Our results also show that PM (statistically) significantly outperformed GM in terms of prediction accuracy. These encouraging findings make the first fundamental step towards proactive IR engines based on brain signals.

References

[1]

Marco Allegretti, Yashar Moshfeghi, Maria Hadjigeorgieva, Frank E. Pollick, Joemon M. Jose, and Gabriella Pasi. 2015. When Relevance Judgement is Happening?: An EEG-based Study. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '15). ACM, New York, NY, USA, 719-722.

Digital Library

[2]

Ioannis Arapakis, Konstantinos Athanasakos, and Joemon M. Jose. 2010. A Comparison of General vs Personalised Affective Models for the Prediction of Topical Relevance. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '10). ACM, New York, NY, USA, 371-378.

Digital Library

[3]

Ioannis Arapakis, Ioannis Konstas, and Joemon M. Jose. 2009. Using Facial Expressions and Peripheral Physiological Signals As Implicit Indicators of Topical Relevance. In Proceedings of the 17th ACM International Conference on Multimedia(MM '09). ACM, New York, NY, USA, 461-470.

Digital Library

[4]

Ioannis Arapakis, Yashar Moshfeghi, Hideo Joho, Reede Ren, David Hannah, and Joemon M. Jose. 2009. Enriching User Profiling with Affective Features for the Improvement of a Multimodal Recommender System. In Proceedings of the ACM International Conference on Image and Video Retrieval(CIVR '09). ACM, New York, NY, USA, Article 29, 29:1-29:8 pages.

Digital Library

[5]

I. Arapakis, Y. Moshfeghi, H. Joho, R. Ren, D. Hannah, and J. M. Jose. 2009. Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on. 1440-1443.

Digital Library

[6]

Marcia J. Bates. 1998. Indexing and access for digital libraries and the internet: Human, database, and domain factors. J. Am. Soc. Inf. Sci. 49, 13 (12 Dec. 1998), 1185-1205.

Digital Library

[7]

N. J. Belkin, R. N. Oddy, and H. M. Brooks. 1982. ASK Fir Information Retrieval: Part II. Results of a Design Study. Journal of Documentation 38, 3 (1982), 145-164.

[8]

N. J. Belkin, R. N. Oddy, and H. M. Brooks. 1997. Ask for Information Retrieval: Part I.: Background and Theory. (1997), 299-304. http://dl.acm.org/citation.cfm?id=275537.275703

Digital Library

[9]

Jan R. Benetka, Krisztian Balog, and Kjetil Nørvåg. 2017. Anticipating Information Needs Based on Check-in Activity. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining(WSDM '17).

Digital Library

[10]

Sumit Bhatia, Debapriyo Majumdar, and Nitish Aggarwal. 2016. Proactive Information Retrieval: Anticipating Users' Information Need. Springer International Publishing, Cham, 874-877.

[11]

Christine L. Borgman. 2003. From Gutenberg to the Global Information Infrastructure: Access to Information in the Networked World. MIT Press, Cambridge, MA, USA.

Digital Library

[12]

Charles Cole. 2011. A theory of information need for information retrieval that connects information to knowledge.JASIST 62, 7 (2011), 1216-1231.

Digital Library

[13]

Charles Cole, Charles-Antoine Julien, and John E Leide. 2010. An associative index model for hypertext Internet search based on Vannevar Bush's Memex machine: An exploratory case study. Information Research 15, 3 (2010), 15-3.

[14]

Marc N. Coutanche. 2013. Distinguishing multi-voxel patterns and mean activation: Why, how, and what does it tell us?Cognitive, Affective, & Behavioral Neuroscience 13, 3(2013), 667-673.

[15]

Angelika Dimoka. 2012. How to Conduct a Functional Magnetic Resonance (fMRI) Study in Social Science Research. MIS Q. 36, 3 (Sept. 2012), 811-840. http://dl.acm.org/citation.cfm?id=2481655.2481664

Digital Library

[16]

Manuel JA Eugster, Tuukka Ruotsalo, Michiel M Spape´, Oswald Barral, Niklas Ravaja, Giulio Jacucci, and Samuel Kaski. 2016. Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals. Scientific Reports 6(2016).

[17]

Manuel J.A. Eugster, Tuukka Ruotsalo, Michiel M. Spape´, Ilkka Kosunen, Oswald Barral, Niklas Ravaja, Giulio Jacucci, and Samuel Kaski. 2014. Predicting Term-relevance from Brain Signals. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '14). ACM, New York, NY, USA, 425-434.

Digital Library

[18]

Karl Friston. 2010. The free-energy principle: a unified brain theory?Nature reviews. Neuroscience 11, 2 (13 Feb. 2010), 127-138.

[19]

K.J Friston, C Buechel, G.R Fink, J Morris, E Rolls, and R.J Dolan. 1997. Psychophysiological and Modulatory Interactions in Neuroimaging. NeuroImage 6, 3 (1997), 218 - 229.

[20]

Rainer Goebel. {n. d.}. BrainVoyager QX, Vers.2.1, Brain Innovation B.V.Maastricht, Netherlands.

[21]

J. V. Haxby, M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, and P. Pietrini. 2001. Distributed and overlapping representations of faces and objects in ventral temporal cortex.Science (New York, N.Y.) 293, 5539 (28 Sept. 2001), 2425-2430.

[22]

Birger Hjørland. 2010. The foundation of the concept of relevance. J. Am. Soc. Inf. Sci. 61, 2 (1 Feb. 2010), 217-237.

Digital Library

[23]

Peter Ingwersen. 1996. Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory. Journal of Documentation 52, 1 (1996), 3-50.

[24]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately Interpreting Clickthrough Data As Implicit Feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '05). ACM, New York, NY, USA, 154-161.

Digital Library

[25]

Jukka-Pekka Kauppi, Melih Kandemir, Veli-Matti Saarinen, Lotta Hirvenkari, Lauri Parkkonen, Arto Klami, Riitta Hari, and Samuel Kaski. 2015. Towards brain-activity-controlled information retrieval: Decoding image relevance from MEG signals. NeuroImage 112(2015), 288-298.

[26]

S. Sathiya Keerthi and Chih-Jen Lin. 2003. Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel. Neural Comput. 15, 7 (July 2003), 1667-1689.

Digital Library

[27]

Diane Kelly and Nicholas J Belkin. 2002. A user modeling system for personalized interaction and tailored retrieval in interactive IR. Proceedings of the American Society for Information Science and Technology 39, 1 (2002), 316-325.

[28]

Diane Kelly and Nicholas J. Belkin. 2004. Display Time As Implicit Feedback: Understanding Task Effects. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '04). ACM, New York, NY, USA, 377-384.

Digital Library

[29]

Jürgen Koenemann and Nicholas J. Belkin. 1996. A Case for Interaction: A Study of Interactive Information Retrieval Behavior and Effectiveness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI '96). ACM, New York, NY, USA, 205-212.

Digital Library

[30]

N. Kriegeskorte and P. A. Bandettini. 2007. Analyzing for information, not activation, to exploit high-resolution fMRI. NeuroImage 38(2007), 649-662.

[31]

Nikolaus Kriegeskorte, Rainer Goebel, and Peter Bandettini. 2006. Information-based functional brain mapping.Proceedings of the National Academy of Sciences of the United States of America 103, 10 (7 March 2006), 3863-3868.

[32]

Carol C. Kuhlthau. 1993. A Principle of Uncertainty for Information Seeking. Journal of Documentation 49, 4 (1993), 339-355.

[33]

KS Lashley. 1952. Functional interpretation of anatomic patterns.Research publications-Association for Research in Nervous and Mental Disease 30(1952), 529.

[34]

Robert Leech, Salwa Kamourieh, Christian F. Beckmann, and David J. Sharp. 2011. Fractionating the Default Mode Network: Distinct Contributions of the Ventral and Dorsal Posterior Cingulate Cortex to Cognitive Control. Journal of Neuroscience 31, 9 (2011), 3217-3224.

[35]

Robert Leech and David J. Sharp. 2014. The role of the posterior cingulate cortex in cognition and disease. Brain : a journal of neurology 137, Pt 1 (01 Jan. 2014), 12-32.

[36]

Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. J. Am. Soc. Inf. Sci. 59, 7 (1 May 2008), 1041-1052.

Digital Library

[37]

Robert McGill, John W. Tukey, and Wayne A. Larsen. 1978. Variations of Box Plots. The American Statistician 32, 1 (1978), 12-16.

[38]

Yashar Moshfeghi and Joemon M. Jose. 2013. An Effective Implicit Relevance Feedback Technique Using Affective, Physiological and Behavioural Features. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '13). ACM, New York, NY, USA, 133-142.

Digital Library

[39]

Yashar Moshfeghi and Joemon M. Jose. 2013. On Cognition, Emotion, and Interaction Aspects of Search Tasks with Different Search Intentions. In Proceedings of the 22Nd International Conference on World Wide Web(WWW '13). ACM, New York, NY, USA, 931-942.

Digital Library

[40]

Yashar Moshfeghi, Luisa R. Pinto, Frank E. Pollick, and Joemon M. Jose. 2013. Understanding Relevance: An fMRI Study. In Advances in Information Retrieval: 35th European Conference on IR Research, ECIR 2013, Moscow, Russia, March 24-27, 2013. Proceedings, Pavel Serdyukov, Pavel Braslavski, Sergei O. Kuznetsov, Jaap Kamps, Stefan Rüger, Eugene Agichtein, Ilya Segalovich, and Emine Yilmaz (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 14-25.

Digital Library

[41]

Yashar Moshfeghi and Frank E. Pollick. 2018. Search Process As Transitions Between Neural States. In Proceedings of the 2018 World Wide Web Conference(WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1683-1692.

Digital Library

[42]

Yashar Moshfeghi, Peter Triantafillou, and Frank E. Pollick. 2016. Understanding Information Need: An fMRI Study. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR '16). ACM, New York, NY, USA, 335-344.

Digital Library

[43]

Javed Mostafa, Vincent Carrasco, Chris Foster, and Kelly Giovenallo. 2015. Identifying Neurological Patterns Associated with Information Seeking: A Pilot fMRI Study. Springer International Publishing, Cham, 167-173.

[44]

Javed Mostafa and Jacek Gwizdka. 2016. Deepening the Role of the User: Neuro-Physiological Evidence As a Basis for Studying and Improving Search. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval(CHIIR '16). ACM, New York, NY, USA, 63-70.

Digital Library

[45]

R. C. Oldfield. 1971. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9, 1 (March 1971), 97-113.

[46]

Joseph M Orr, Harry R Smolker, and Marie T Banich. 2015. Organization of the human frontal pole revealed by large-scale DTI-based connectivity: implications for control of behavior. PloS one 10, 5 (2015), e0124797.

[47]

Michael Petrides and Deepak N. Pandya. 2007. Efferent Association Pathways from the Rostral Prefrontal Cortex in the Macaque Monkey. Journal of Neuroscience 27, 43 (2007), 11573-11586.

[48]

Jesse Rissman and Anthony D Wagner. 2012. Distributed representations in memory: insights from functional brain imaging. Annual review of psychology 63 (2012), 101-128.

[49]

Yang Song and Qi Guo. 2016. Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines. In Proceedings of the 25th International Conference on World Wide Web(WWW '16). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 543-553.

Digital Library

[50]

J. Talairach and P. Tournoux. 1988. Co-planar Stereotaxic Atlas of the Human Brain: 3-dimensional Proportional System : an Approach to Cerebral Imaging. Thieme. https://books.google.co.uk/books?id=pYFiQgAACAAJ

[51]

Robert S. Taylor. 1968. Question-Negotiation and Information Seeking in Libraries. College and Research Libraries29 (1968), 178-194.

[52]

Michael T. Todd, Leigh E. Nystrom, and Jonathan D. Cohen. 2013. Confounds in multivariate pattern analysis: Theory and rule representation case study. NeuroImage 77 (15 Aug. 2013), 157-165.

[53]

Peter Ingwersen und Kalervo Järvelin. 2006. The Turn: Integration of Information Seeking and Retrieval in Context. Springer, 2005. xiv, 448 S. ISBN 1-4020-3850-X.

Digital Library

[54]

Chih wei Hsu, Chih chung Chang, and Chih jen Lin. 2010. A practical guide to support vector classification.

[55]

Ryen White, Joemon M. Jose, and Ian Ruthven. 2003. A task-oriented study on the influencing effects of query-biased summarisation in web searching. Inf. Process. Manage. 39, 5 (2003), 707-733.

Digital Library

[56]

Ryen W. White. 2004. Implicit Feedback for Interactive Information Retrieval. Ph.D. Dissertation. University of Glasgow.

[57]

Ryen W. White and Diane Kelly. 2006. A Study on the Effects of Personalization and Task Information on Implicit Feedback Performance. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management(CIKM '06). ACM, New York, NY, USA, 297-306.

Digital Library

[58]

T. D. Wilson. 1981. On User Studies and Information Needs.Journal of Documentation 37, 1 (0 March 1981), 3-15. http://www.eric.ed.gov/ERICWebPortal/detail?accno=EJ248909

[59]

Qingbai Zhao, Zhijin Zhou, Haibo Xu, Shi Chen, Fang Xu, Wenliang Fan, and Lei Han. 2013. Dynamic neural network of insight: a functional magnetic resonance imaging study on solving Chinese 'chengyu' riddles. PloS one 8, 3 (2013), e59351.

Cited By

McGuire NMoshfeghi Y(2024)What Song Am I Thinking Of?Machine Learning, Optimization, and Data Science10.1007/978-3-031-53966-4_31(418-432)Online publication date: 15-Feb-2024
https://doi.org/10.1007/978-3-031-53966-4_31
Michalkova DRodriguez MMoshfeghi Y(2023)Understanding Feeling-of-Knowing in Information Search: An EEG StudyACM Transactions on Information Systems10.1145/361138442:3(1-30)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3611384
Torres-García AMartínez-Santiago FMontejo-Ráez AUreña-López L(2023)Toward an Educative EEG-Based neuroIIR System for Adapting ContentsInternational Journal of Human–Computer Interaction10.1080/10447318.2023.227508840:23(7955-7969)Online publication date: 3-Nov-2023
https://doi.org/10.1080/10447318.2023.2275088
Show More Cited By

Recommendations

Understanding Information Need: An fMRI Study
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

The raison d'etre of IR is to satisfy human information need. But, do we really understand information need? Despite advances in the past few decades in both the IR and relevant scientific communities, this question is largely unanswered. We do not ...
Information Need Awareness: An EEG Study
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

A fundamental goal of Information Retrieval (IR) is to satisfy searchers' information need (IN). Advances in neuroimaging technologies have allowed for interdisciplinary research to investigate the brain activity associated with the realisation of IN. ...
Towards Understanding Neuroscience of Realisation of Information Need in Light of Relevance and Satisfaction Judgement
Machine Learning, Optimization, and Data Science
Abstract
Understanding how to satisfy searchers’ information need (IN) is the main goal of Information Retrieval (IR) systems. In this study, we investigate the relationships between information need and the two key concepts of relevance and satisfaction ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: The World Wide Web Conference

May 2019

3620 pages

ISBN:9781450366748

DOI:10.1145/3308558

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
199
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

McGuire NMoshfeghi Y(2024)What Song Am I Thinking Of?Machine Learning, Optimization, and Data Science10.1007/978-3-031-53966-4_31(418-432)Online publication date: 15-Feb-2024
https://doi.org/10.1007/978-3-031-53966-4_31
Michalkova DRodriguez MMoshfeghi Y(2023)Understanding Feeling-of-Knowing in Information Search: An EEG StudyACM Transactions on Information Systems10.1145/361138442:3(1-30)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3611384
Torres-García AMartínez-Santiago FMontejo-Ráez AUreña-López L(2023)Toward an Educative EEG-Based neuroIIR System for Adapting ContentsInternational Journal of Human–Computer Interaction10.1080/10447318.2023.227508840:23(7955-7969)Online publication date: 3-Nov-2023
https://doi.org/10.1080/10447318.2023.2275088
Michalkova DParra Rodriguez MMoshfeghi YCrestani FPasi GGaussier E(2022)Drivers of Information Needs: A Behavioural Study – Exploring Searcher's Feeling-of-KnowingProceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3539813.3545125(171-181)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3539813.3545125
Ye ZXie XLiu YWang ZChen XZhang MMa SMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Brain Topography Adaptive Network for Satisfaction Modeling in Interactive Information Access SystemProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548258(90-100)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548258
Ye ZXie XLiu YWang ZChen XZhang MMa S(2022)Towards a Better Understanding of Human Reading Comprehension with Brain SignalsProceedings of the ACM Web Conference 202210.1145/3485447.3511966(380-391)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3511966
Ye ZXie XLiu YWang ZLi XLi JChen XZhang MMa SAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Why Don't You ClickProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3532082(633-645)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3532082
Michalkova DParra-Rodriguez MMoshfeghi YAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Information Need AwarenessProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531999(610-621)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531999
Pinkosova ZMcGeown WMoshfeghi Y(2022)Revisiting Neurological Aspects of Relevance: An EEG StudyMachine Learning, Optimization, and Data Science10.1007/978-3-031-25891-6_41(549-563)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-25891-6_41
Paisalnan SPollick FMoshfeghi Y(2022)Neural Correlates of Satisfaction of an Information NeedMachine Learning, Optimization, and Data Science10.1007/978-3-031-25891-6_34(443-457)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-25891-6_34
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents