More Web Proxy on the site http://driver.im/

research-article

A Passage-Level Reading Behavior Model for Mobile Search

Authors:

Heyan HuangAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 3236 - 3246

https://doi.org/10.1145/3543507.3583343

Published: 30 April 2023 Publication History

Abstract

Reading is a vital and complex cognitive activity during users’ information-seeking process. Several studies have focused on understanding users’ reading behavior in desktop search. Their findings greatly contribute to the design of information retrieval models. However, little is known about how users read a result in mobile search, although search currently happens more frequently in mobile scenarios. In this paper, we conduct a lab-based user study to investigate users’ fine-grained reading behavior patterns in mobile search. We find that users’ reading attention allocation is strongly affected by several behavior biases, such as position and selection biases. Inspired by these findings, we propose a probabilistic generative model, the Passage-level Reading behavior Model (PRM), to model users’ reading behavior in mobile search. The PRM utilizes observable passage-level exposure and viewport duration events to infer users’ unobserved skimming event, reading event, and satisfaction perception during the reading process. Besides fitting the passage-level reading behavior, we utilize the fitted parameters of PRM to estimate the passage-level and document-level relevance. Experimental results show that PRM outperforms existing unsupervised relevance estimation models. PRM has strong interpretability and provides valuable insights into the understanding of how users seek and perceive useful information in mobile search.

References

[1]

Klinton Bicknell and Roger Levy. 2010. A rational model of eye movement control in reading. In Proceedings of the 48th annual meeting of the Association for Computational Linguistics. 1168–1178.

[2]

Andrei Broder. 2002. A taxonomy of web search. In ACM Sigir forum, Vol. 36. ACM New York, NY, USA, 3–10.

[3]

George Buchanan and Fernando Loizides. 2007. Investigating document triage on paper and electronic media. In International Conference on Theory and Practice of Digital Libraries. Springer, 416–427.

[4]

Orkut Buyukkokten. 2002. Wireless Web access on handheld devices. Stanford University.

[5]

Andrew Dillon. 1992. Reading from paper versus screens: A critical review of the empirical literature. Ergonomics 35, 10 (1992), 1297–1326.

[6]

Robert L Duchnicky and Paul A Kolers. 1983. Readability of text scrolled on visual display terminals as a function of window size. Human factors 25, 6 (1983), 683–692.

[7]

Ralf Engbert, André Longtin, and Reinhold Kliegl. 2002. A dynamical model of saccade generation in reading based on spatially distributed lexical processing. Vision research 42, 5 (2002), 621–636.

[8]

Ralf Engbert, Antje Nuthmann, Eike M Richter, and Reinhold Kliegl. 2005. SWIFT: a dynamical model of saccade generation during reading.Psychological review 112, 4 (2005), 777.

[9]

Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychological bulletin 76, 5 (1971), 378.

[10]

Max Grusky, Jeiran Jahani, Josh Schwartz, Dan Valente, Yoav Artzi, and Mor Naaman. 2017. Modeling sub-document attention using viewport time. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6475–6480.

Digital Library

[11]

Qi Guo and Eugene Agichtein. 2012. Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior. In Proceedings of the 21st international conference on World Wide Web. 569–578.

Digital Library

[12]

Qi Guo, Haojian Jin, Dmitry Lagun, Shuai Yuan, and Eugene Agichtein. 2013. Mining touch interaction data on mobile devices to predict web search result relevance. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 153–162.

Digital Library

[13]

Michael Hahn and Frank Keller. 2016. Modeling human reading with neural attention. arXiv preprint arXiv:1608.05604 (2016).

[14]

Jeff Huang, Ryen W White, Georg Buscher, and Kuansan Wang. 2012. Improving searcher models using mouse cursor activity. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 195–204.

Digital Library

[15]

Kalervo Järvelin and Jaana Kekäläinen. 2000. IR Evaluation Methods for Retrieving Highly Relevant Documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Athens, Greece) (SIGIR ’00). ACM, New York, NY, USA, 41–48. https://doi.org/10.1145/345508.345545

Digital Library

[16]

Matt Jones, George Buchanan, and Harold Thimbleby. 2003. Improving web search on small screen devices. Interacting with computers 15, 4 (2003), 479–495.

[17]

Diane Kelly. 2005. Implicit feedback: Using behavior to infer relevance. In New directions in cognitive information retrieval. Springer, 169–186.

[18]

Diane Kelly and Nicholas J Belkin. 2001. Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval. 408–409.

Digital Library

[19]

Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, and Hwan-Jin Yoon. 2015. Eye-tracking analysis of user behavior and performance in web search on large and small screens. Journal of the Association for Information Science and Technology 66, 3 (2015), 526–544.

Digital Library

[20]

Dmitry Lagun and Eugene Agichtein. 2011. Viewser: Enabling large-scale remote user studies of web search examination and interaction. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. 365–374.

Digital Library

[21]

Dmitry Lagun, Chih-Hung Hsieh, Dale Webster, and Vidhya Navalpakkam. 2014. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 113–122.

Digital Library

[22]

J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.

[23]

Xiangsheng Li, Yiqun Liu, Jiaxin Mao, Zexue He, Min Zhang, and Shaoping Ma. 2018. Understanding reading attention distribution during relevance judgement. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 733–742.

Digital Library

[24]

Xiangsheng Li, Jiaxin Mao, Chao Wang, Yiqun Liu, Min Zhang, and Shaoping Ma. 2019. Teach machine how to read: reading behavior inspired relevance estimation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 795–804.

Digital Library

[25]

Yiqun Liu, Chao Wang, Ke Zhou, Jianyun Nie, Min Zhang, and Shaoping Ma. 2014. From skimming to reading: A two-stage examination model for web search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. 849–858.

Digital Library

[26]

Jiaxin Mao, Yiqun Liu, Noriko Kando, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. Investigating result usefulness in mobile search. In European Conference on Information Retrieval. Springer, 223–236.

[27]

Jiaxin Mao, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. Constructing click models for mobile search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 775–784.

Digital Library

[28]

Catherine C Marshall and Frank M Shipman III. 1997. Spatial hypertext and the practice of information triage. In Proceedings of the eighth ACM conference on Hypertext. 124–133.

Digital Library

[29]

Kate Moran. 2016. Reading Content on Mobile Devices. Nielsen Norman Group, https://www.nngroup.com/articles/mobile-content/ (2016).

[30]

Jakob Nielsen. 2011. Mobile content is twice as difficult. Nielsen Norman Group, https://www.nngroup.com/articles/mobile-content-is-twice-as-difficult-2011/ (2011).

[31]

Keith Rayner, Alexander Pollatsek, Jane Ashby, and Charles Clifton Jr. 2012. Psychology of reading. (2012).

[32]

Erik D Reichle, Keith Rayner, and Alexander Pollatsek. 1999. Eye movement control in reading: Accounting for initial fixation locations and refixations within the EZ Reader model. Vision research 39, 26 (1999), 4403–4411.

[33]

Erik D Reichle, Keith Rayner, and Alexander Pollatsek. 2003. The EZ Reader model of eye-movement control in reading: Comparisons to other models. Behavioral and brain sciences 26, 4 (2003), 445–476.

[34]

Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of the 17th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR’94). Springer, 232–241.

[35]

Milad Shokouhi, Rosie Jones, Umut Ozertem, Karthik Raghunathan, and Fernando Diaz. 2014. Mobile query reformulations. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 1011–1014.

Digital Library

[36]

Maximilian Speicher, Andreas Both, and Martin Gaedke. 2013. TellMyRelevance! predicting the relevance of web search results from cursor interactions. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 1281–1290.

[37]

Jan Theeuwes. 1993. Visual selective attention: A theoretical analysis. Acta psychologica 83, 2 (1993), 93–154.

[38]

Adam Trischler, Zheng Ye, Xingdi Yuan, and Kaheer Suleman. 2016. Natural language comprehension with the epireader. arXiv preprint arXiv:1606.02270 (2016).

[39]

Manisha Verma and Emine Yilmaz. 2016. Characterizing relevance on mobile and desktop. In European Conference on Information Retrieval. Springer, 212–223.

[40]

Yizhong Wang, Kai Liu, Jing Liu, Wei He, Yajuan Lyu, Hua Wu, Sujian Li, and Haifeng Wang. 2018. Multi-passage machine reading comprehension with cross-passage answer verification. arXiv preprint arXiv:1805.02220 (2018).

[41]

Ryen W White and Diane Kelly. 2006. A study on the effects of personalization and task information on implicit feedback performance. In Proceedings of the 15th ACM international conference on Information and knowledge management. 297–306.

Digital Library

[42]

Zhijing Wu, Jiaxin Mao, Yiqun Liu, Jingtao Zhan, Yukun Zheng, Min Zhang, and Shaoping Ma. 2020. Leveraging passage-level cumulative gain for document ranking. In Proceedings of The Web Conference 2020. 2421–2431.

Digital Library

[43]

Zhijing Wu, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. Investigating Reading Behavior in Fine-grained Relevance Judgment. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1889–1892.

Digital Library

[44]

Zhi-Jing Wu, Yi-Qun Liu, Jia-Xin Mao, Min Zhang, and Shao-Ping Ma. 2022. Leveraging Document-Level and Query-Level Passage Cumulative Gain for Document Ranking. Journal of Computer Science and Technology 37, 4 (2022), 814–838.

Digital Library

[45]

Chengxiang Zhai and John Lafferty. 2017. A study of smoothing methods for language models applied to ad hoc information retrieval. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 268–276.

[46]

Zhuosheng Zhang, Junjie Yang, and Hai Zhao. 2020. Retrospective reader for machine reading comprehension. arXiv preprint arXiv:2001.09694 (2020).

[47]

Yukun Zheng, Jiaxin Mao, Yiqun Liu, Cheng Luo, Min Zhang, and Shaoping Ma. 2019. Constructing click model for mobile search with viewport time. ACM Transactions on Information Systems (TOIS) 37, 4 (2019), 1–34.

Digital Library

[48]

Yukun Zheng, Jiaxin Mao, Yiqun Liu, Mark Sanderson, Min Zhang, and Shaoping Ma. 2020. Investigating examination behavior in mobile search. In Proceedings of the 13th International Conference on Web Search and Data Mining. 771–779.

Digital Library

[49]

Yukun Zheng, Jiaxin Mao, Yiqun Liu, Zixin Ye, Min Zhang, and Shaoping Ma. 2019. Human behavior inspired machine reading comprehension. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 425–434.

Digital Library

Cited By

Zhu RCheng CSong YCai Z(2024)Modeling Attentive Interaction Behavior for Web Content Identification in Exploratory Information SeekingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997508:4(1-28)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699750
Gienapp LScells HDeckers NBevendorff JWang SKiesel JSyed SFröbe MZuccon GStein BHagen MPotthast MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Evaluating Generative Ad Hoc Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657849(1916-1929)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657849

Index Terms

A Passage-Level Reading Behavior Model for Mobile Search
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
      1. Probabilistic retrieval models
  2. World Wide Web
    1. Web searching and information discovery
      1. Content ranking

Recommendations

Investigating Examination Behavior in Mobile Search
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data Mining

Examination is one of the most important user interactions in Web search. A number of works studied examination behavior in Web search and helped researchers better understand how users allocate their attention on search engine result pages (SERPs). ...
Mobile search made easier: An ability-based mobile search prototype for people with dyslexia
CHIIR '24: Proceedings of the 2024 Conference on Human Information Interaction and Retrieval

Although 1 person in 14 has dyslexia, most search interfaces are designed based on a ‘one-size-fits-all’ approach, creating inequity for neurodiverse searchers. This is also the case for mobile search, which accounts for most Google searches. While ...
How Does Mobile Context Affect People's Web Search Behavior?: A Diary Study of Mobile Information Needs and Search Behaviors
AINA '12: Proceedings of the 2012 IEEE 26th International Conference on Advanced Information Networking and Applications

Mobile devices have become popular to access the information on-the-move. However, the physical restrictions of mobile devices such as display size and input capabilities and the changes of users' contexts may affect the users' information needs and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

the China Postdoctoral Science Foundation

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
222
Total Downloads

Downloads (Last 12 months)91
Downloads (Last 6 weeks)24

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu RCheng CSong YCai Z(2024)Modeling Attentive Interaction Behavior for Web Content Identification in Exploratory Information SeekingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997508:4(1-28)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699750
Gienapp LScells HDeckers NBevendorff JWang SKiesel JSyed SFröbe MZuccon GStein BHagen MPotthast MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Evaluating Generative Ad Hoc Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657849(1916-1929)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657849

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents