More Web Proxy on the site http://driver.im/

research-article

Open access

AutoDesc: Facilitating Convenient Perusal of Web Data Items for Blind Users

Authors:

Sampath Jayarathna,

Vikas AshokAuthors Info & Claims

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

Pages 32 - 45

https://doi.org/10.1145/3581641.3584049

Published: 27 March 2023 Publication History

All formats PDF

Abstract

Web data items such as shopping products, classifieds, and job listings are indispensable components of most e-commerce websites. The information on the data items are typically distributed over two or more webpages, e.g., a ‘Query-Results’ page showing the summaries of the items, and ‘Details’ pages containing full information about the items. While this organization of data mitigates information overload and visual cluttering for sighted users, it however increases the interaction overhead and effort for blind users, as back-and-forth navigation between webpages using screen reader assistive technology is tedious and cumbersome. Existing usability-enhancing solutions are unable to provide adequate support in this regard as they predominantly focus on enabling efficient content access within a single webpage, and as such are not tailored for content distributed across multiple webpages. As an initial step towards addressing this issue, we developed AutoDesc, a browser extension that leverages a custom extraction model to automatically detect and pull out additional item descriptions from the ‘details’ pages, and then proactively inject the extracted information into the ‘Query-Results’ page, thereby reducing the amount of back-and-forth screen reader navigation between the two webpages. In a study with 16 blind users, we observed that within the same time duration, the participants were able to peruse significantly more data items on average with AutoDesc, compared to that with their preferred screen readers as well as with a state-of-the-art solution.

References

[1]

Waleed Abdulla. 2017. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. https://github.com/matterport/Mask_RCNN.

[2]

Julian Alarte, David Insa, and Josep Silva. 2017. Webpage menu detection based on DOM. In International Conference on Current Trends in Theory and Practice of Informatics. Springer, 411–422.

[3]

Manuel Álvarez, Alberto Pan, Juan Raposo, Fernando Bellas, and Fidel Cacheda. 2010. Finding and extracting data records from web pages. Journal of Signal Processing Systems 59, 1 (2010), 123–137.

Digital Library

[4]

Apple Inc.2022. Accessibility - Vision - Apple. https://www.apple.com/accessibility/vision/.

[5]

Chieko Asakawa and Hironobu Takagi. 2000. Annotation-based transcoding for nonvisual web access. In Proceedings of the fourth international ACM conference on Assistive technologies. 172–179.

Digital Library

[6]

Vikas Ashok, Yevgen Borodin, Yury Puzis, and IV Ramakrishnan. 2015. Capti-speak: a speech-enabled web screen reader. In Proceedings of the 12th International Web for All Conference. 1–10.

Digital Library

[7]

Vikas Ashok, Yury Puzis, Yevgen Borodin, and IV Ramakrishnan. 2017. Web screen reading automation assistance using semantic abstraction. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. 407–418.

Digital Library

[8]

Ali Selman Aydin, Shirin Feiz, Vikas Ashok, and IV Ramakrishnan. 2020. Sail: Saliency-driven injection of aria landmarks. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 111–115.

Digital Library

[9]

Rakesh Babu. 2013. Understanding challenges in non-visual interaction with travel sites: An exploratory field study with blind users. First Monday (2013).

[10]

Sean Bechhofer, Simon Harper, and Darren Lunn. 2006. Sadie: Semantic annotation for accessibility. In International Semantic Web Conference. Springer, 101–115.

Digital Library

[11]

Jeffrey P Bigham, Anna C Cavender, Jeremy T Brudvik, Jacob O Wobbrock, and Richard E Ladner. 2007. WebinSitu: a comparative analysis of blind and sighted browsing behavior. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility. 51–58.

Digital Library

[12]

Jeffrey P Bigham and Richard E Ladner. 2007. Accessmonkey: a collaborative scripting framework for web users and developers. In Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A). 25–34.

Digital Library

[13]

Michael Bolin, Matthew Webber, Philip Rha, Tom Wilson, and Robert C Miller. 2005. Automation and customization of rendered web pages. In Proceedings of the 18th annual ACM symposium on User interface software and technology. 163–172.

Digital Library

[14]

Yevgen Borodin, Jeffrey P. Bigham, Glenn Dausch, and I. V. Ramakrishnan. 2010. More than Meets the Eye: A Survey of Screen-Reader Browsing Strategies. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)(Raleigh, North Carolina) (W4A ’10). Association for Computing Machinery, New York, NY, USA, Article 13, 10 pages. https://doi.org/10.1145/1805986.1806005

Digital Library

[15]

Roopa Bose and Helmut Jürgensen. 2014. Accessibility of E-Commerce Websites for Vision-Impaired Persons. In ICCHP.

[16]

James V. Bradley. 1958. Complete Counterbalancing of Immediate Sequential Effects in a Latin Square Design. J. Amer. Statist. Assoc. 53, 282 (1958), 525–528. https://doi.org/10.1080/01621459.1958.10501456 arXiv:https://amstat.tandfonline.com/doi/pdf/10.1080/01621459.1958.10501456

[17]

John Brooke. 1996. Sus: a “quick and dirty’usability. Usability evaluation in industry 189, 3 (1996).

[18]

Andy Brown and Simon Harper. 2013. Dynamic injection of WAI-ARIA into web content. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility. 1–4.

Digital Library

[19]

Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. Vips: a vision-based page segmentation algorithm. (2003).

[20]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).

[21]

Yixiang Fang, Xiaoqin Xie, Xiaofeng Zhang, Reynold Cheng, and Zhiqiang Zhang. 2018. STEM: a suffix tree-based method for web data records extraction. Knowledge and Information Systems 55, 2 (2018), 305–331.

Digital Library

[22]

Javedul Ferdous, Hae-Na Lee, Sampath Jayarathna, and Vikas Ashok. 2022. InSupport: Proxy Interface for Enabling Efficient Non-Visual Interaction with Web Data Records. In 27th International Conference on Intelligent User Interfaces. 49–62.

[23]

Freedom Scientific. 2022. JAWS ® – Freedom Scientific. https://www.freedomscientific.com/products/software/jaws/.

[24]

Mahak Gambhir and Vishal Gupta. 2017. Recent automatic text summarization techniques: a survey. Artificial Intelligence Review 47, 1 (2017), 1–66.

Digital Library

[25]

Boni García, Mario Munoz-Organero, Carlos Alario-Hoyos, and Carlos Delgado Kloos. 2021. Automated driver management for selenium WebDriver. Empirical Software Engineering 26, 5 (2021), 1–51.

Digital Library

[26]

Cole Gleason, Amy Pavel, Emma McCamey, Christina Low, Patrick Carrington, Kris M Kitani, and Jeffrey P Bigham. 2020. Twitter A11y: A browser extension to make Twitter images accessible. In Proceedings of the 2020 chi conference on human factors in computing systems. 1–12.

Digital Library

[27]

Wael H Gomaa, Aly A Fahmy, 2013. A survey of text similarity approaches. international journal of Computer Applications 68, 13(2013), 13–18.

[28]

Simon Harper and Yeliz Yesilada. 2007. Web authoring for accessibility (WAfA). Journal of Web Semantics 5, 3 (2007), 175–179.

Digital Library

[29]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.

[30]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969.

[31]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[32]

Megan L Hilts. 2008. Internet dependency, motivations for internet use and their effect on work productivity: The 21 st century addiction. Rochester Institute of Technology.

[33]

Ravi Ilango. 2019. Using NLP (BERT) to improve OCR accuracy. https://medium.com/doma/using-nlp-bert-to-improve-ocr-accuracy-385c98ae174c

[34]

Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810(2017).

[35]

Eduardo Sany Laber, Críston Pereira de Souza, Iam Vita Jabour, Evelin Carvalho Freire de Amorim, Eduardo Teixeira Cardoso, Raúl Pierre Rentería, Lúcio Cunha Tinoco, and Caio Dias Valentim. 2009. A fast and simple method for extracting relevant content from news webpages. In Proceedings of the 18th ACM conference on Information and knowledge management. 1685–1688.

Digital Library

[36]

Jonathan Lazar, Aaron Allen, Jason Kleinman, and Chris Malarkey. 2007. What frustrates screen reader users on the web: A study of 100 blind users. International Journal of human-computer interaction 22, 3(2007), 247–269.

[37]

Hae-Na Lee and Vikas Ashok. 2022. Customizable Tabular Access to Web Data Records for Convenient Low-Vision Screen Magnifier Interaction. ACM Transactions on Accessible Computing (TACCESS) (2022).

[38]

Hae-Na Lee, Sami Uddin, and Vikas Ashok. 2020. iTOC: Enabling Efficient Non-Visual Interaction with Long Web Documents. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 3799–3806.

Digital Library

[39]

Hae-Na Lee, Sami Uddin, and Vikas Ashok. 2020. TableView: Enabling Efficient Access to Web Data Records for Screen-Magnifier Users. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1–12.

Digital Library

[40]

Barbara Leporini and Fabio Paternò. 2004. Increasing usability when interacting through screen readers. Universal access in the information society 3, 1 (2004), 57–70.

[41]

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2117–2125.

[42]

Letresa McLawhorn. 2001. Leveling the accessibility playing field: Section 508 of the Rehabilitation Act.

[43]

Valentyn Melnyk, Vikas Ashok, Yury Puzis, Andrii Soviak, Yevgen Borodin, and IV Ramakrishnan. 2014. Widget classification with applications to web accessibility. In International Conference on Web Engineering. Springer, 341–358.

[44]

Thi Tuyet Hai Nguyen, Adam Jatowt, Nhu-Van Nguyen, Mickael Coustaty, and Antoine Doucet. 2020. Neural machine translation with BERT for post-OCR error detection and correction. In Proceedings of the ACM/IEEE joint conference on digital libraries in 2020. 333–336.

Digital Library

[45]

Oliver Nina, Bryan Morse, and William Barrett. 2011. A recursive Otsu thresholding method for scanned document binarization. In 2011 IEEE Workshop on Applications of Computer Vision (WACV). IEEE, 307–314.

Digital Library

[46]

NV Access. 2022. NV Access. https://www.nvaccess.org/.

[47]

Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.

[48]

Jyotika Prasad and Andreas Paepcke. 2008. Coreex: content extraction from online news articles. In Proceedings of the 17th ACM conference on Information and knowledge management. 1391–1392.

Digital Library

[49]

Elisa Prati, Simone Pozzi, Fabio Grandi, and Margherita Peruzzini. 2021. E-commerce Usability Guidelines for Visually Impaired Users. In International Conference on Human-Computer Interaction. Springer, 280–293.

[50]

Yury Puzis, Yevgen Borodin, Andrii Soviak, Valentyn Melnyk, and IV Ramakrishnan. 2015. Affordable web accessibility: A case for cheaper ARIA. In Proceedings of the 12th International Web for All Conference. 1–4.

Digital Library

[51]

IV Ramakrishnan, Vikas Ashok, and Syed Masum Billah. 2017. Non-visual web browsing: Beyond web accessibility. In International Conference on Universal Access in Human-Computer Interaction. Springer, 322–334.

[52]

Kunal Relan. 2019. Building REST APIs with Flask. Building REST APIs with Flask(2019).

[53]

Subendhu Rongali, Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2020. Improved pretraining for domain-specific contextual embedding models. (2020).

[54]

Brijendra Singh and Hemant Kumar Singh. 2010. Web data mining research: a survey. In 2010 IEEE International Conference on Computational Intelligence and Computing Research. IEEE, 1–10.

[55]

Ray Smith. 2007. An overview of the Tesseract OCR engine. In Ninth international conference on document analysis and recognition (ICDAR 2007), Vol. 2. IEEE, 629–633.

Digital Library

[56]

Tony Stockman and Oussama Metatla. 2008. The influence of screen-readers on web cognition. In Proceeding of Accessible design in the digital world conference (ADDW 2008), York, UK.

[57]

The GIMP Development Team. 1998. GNU Image Manipulation Program. https://www.gimp.org

[58]

Esko Ukkonen. 1995. On-line construction of suffix trees. Algorithmica 14, 3 (1995), 249–260.

Digital Library

[59]

W3. 1997. WAI. W3C Web Accessibility Initiative. https://www.w3.org/WAI/.

[60]

Ruolin Wang, Zixuan Chen, Mingrui Ray Zhang, Zhaoheng Li, Zhixiu Liu, Zihan Dang, Chun Yu, and Xiang’Anthony’ Chen. 2021. Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[61]

WebAIM. 2018. Survey of Users with Low Vision #2 Results. https://webaim.org/projects/lowvisionsurvey2/

[62]

WebAIM. 2019. WebAIM: Screen Reader User Survey #8 Results. https://webaim.org/projects/screenreadersurvey8/

[63]

Shaomei Wu, Jeffrey Wieland, Omid Farivar, and Julie Schiller. 2017. Automatic alt-text: Computer-generated image descriptions for blind users on a social network service. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 1180–1192.

Digital Library

[64]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144(2016).

[65]

Canhui Xu, Cao Shi, Hengyue Bi, Chuanqi Liu, Yongfeng Yuan, Haoyan Guo, and Yinong Chen. 2021. A Page Object Detection Method Based on Mask R-CNN. IEEE Access 9(2021), 143448–143457.

[66]

Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020. Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1192–1200.

Digital Library

[67]

Yeliz Yesilada, Simon Harper, Carole Goble, and Robert Stevens. 2004. Screen readers cannot see. In International Conference on Web Engineering. Springer, 445–458.

[68]

Yeliz Yesilada, Robert Stevens, Simon Harper, and Carole Goble. 2007. Evaluating DANTE: Semantic transcoding for visually disabled users. ACM Transactions on Computer-Human Interaction (TOCHI) 14, 3(2007), 14–es.

Digital Library

[69]

Yanhong Zhai and Bing Liu. 2005. Web data extraction based on partial tree alignment. In Proceedings of the 14th international conference on World Wide Web. 76–85.

Digital Library

[70]

Zhuoyao Zhong, Lei Sun, and Qiang Huo. 2019. An anchor-free region proposal network for Faster R-CNN-based text detection approaches. International Journal on Document Analysis and Recognition (IJDAR) 22, 3(2019), 315–327.

Digital Library

[71]

Jun Zhu, Zaiqing Nie, Ji-Rong Wen, Bo Zhang, and Wei-Ying Ma. 2006. Simultaneous record detection and attribute labeling in web data extraction. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 494–503.

Digital Library

Cited By

Prakash YNayak ASunkara MJayarathna SLee HAshok V(2024)All in One Place: Ensuring Usable Access to Online Shopping Items for Blind UsersProceedings of the ACM on Human-Computer Interaction10.1145/36646398:EICS(1-25)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3664639
Singh TSingh KVarshney NGupta PKumar G(2024)Enhancing Web Browser Extensions: Preventing JavaScript Code Injection and VulnerabilitiesInnovative Computing and Communications10.1007/978-981-97-3588-4_44(547-557)Online publication date: 24-Jul-2024
https://doi.org/10.1007/978-981-97-3588-4_44
Sunkara MNayak AKalari SKodandaram SJayarathna SLee HAshok V(2024)Assessing the Accessibility and Usability of Web Archives for Blind UsersLinking Theory and Practice of Digital Libraries10.1007/978-3-031-72437-4_12(203-221)Online publication date: 26-Sep-2024
https://doi.org/10.1007/978-3-031-72437-4_12

Index Terms

AutoDesc: Facilitating Convenient Perusal of Web Data Items for Blind Users
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
    2. Empirical studies in accessibility

Recommendations

Enabling Efficient Web Data-Record Interaction for People with Visual Impairments via Proxy Interfaces
Web data records are usually accompanied by auxiliary webpage segments, such as filters, sort options, search form, and multi-page links, to enhance interaction efficiency and convenience for end users. However, blind and visually impaired (BVI) persons ...
Instant Adaptation Enrichment Technique to Improve Web Accessibility for Blind Users
ICISDM '19: Proceedings of the 2019 3rd International Conference on Information System and Data Mining

The Internet in general and the web are considered the main medium of remote communication. They are important sources of information and services to all the users. Assistive technologies and tools facilitate web accessibility to help users with special ...
Universal and ubiquitous web access with Capti
W4A '12: Proceedings of the International Cross-Disciplinary Conference on Web Accessibility

In this paper we present Capti -- a universally and ubiquitously accessible web browsing application enabling intuitive and usable web access for people with and w/o vision impairments. Capti provides a usable screen-reader interface for web browsing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

March 2023

972 pages

ISBN:9798400701061

DOI:10.1145/3581641

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IUI '23

Sponsor:

IUI '23: 28th International Conference on Intelligent User Interfaces

March 27 - 31, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
690
Total Downloads

Downloads (Last 12 months)381
Downloads (Last 6 weeks)40

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Prakash YNayak ASunkara MJayarathna SLee HAshok V(2024)All in One Place: Ensuring Usable Access to Online Shopping Items for Blind UsersProceedings of the ACM on Human-Computer Interaction10.1145/36646398:EICS(1-25)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3664639
Singh TSingh KVarshney NGupta PKumar G(2024)Enhancing Web Browser Extensions: Preventing JavaScript Code Injection and VulnerabilitiesInnovative Computing and Communications10.1007/978-981-97-3588-4_44(547-557)Online publication date: 24-Jul-2024
https://doi.org/10.1007/978-981-97-3588-4_44
Sunkara MNayak AKalari SKodandaram SJayarathna SLee HAshok V(2024)Assessing the Accessibility and Usability of Web Archives for Blind UsersLinking Theory and Practice of Digital Libraries10.1007/978-3-031-72437-4_12(203-221)Online publication date: 26-Sep-2024
https://doi.org/10.1007/978-3-031-72437-4_12

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents