More Web Proxy on the site http://driver.im/

research-article

Moving Deep Learning into Web Browser: How Far Can We Go?

Authors:

Xuanzhe LiuAuthors Info & Claims

WWW '19: The World Wide Web Conference

Pages 1234 - 1244

https://doi.org/10.1145/3308558.3313639

Published: 13 May 2019 Publication History

Abstract

Recently, several JavaScript-based deep learning frameworks have emerged, making it possible to perform deep learning tasks directly in browsers. However, little is known on what and how well we can do with these frameworks for deep learning in browsers. To bridge the knowledge gap, in this paper, we conduct the first empirical study of deep learning in browsers. We survey 7 most popular JavaScript-based deep learning frameworks, investigating to what extent deep learning tasks have been supported in browsers so far. Then we measure the performance of different frameworks when running different deep learning tasks. Finally, we dig out the performance gap between deep learning in browsers and on native platforms by comparing the performance of TensorFlow.js and TensorFlow in Python. Our findings could help application developers, deep-learning framework vendors and browser vendors to improve the efficiency of deep learning in browsers.

References

[1]

2018. brain.js. https://github.com/BrainJS.

[2]

2018. Caffe. http://caffe.berkeleyvision.org/.

[3]

2018. CNTK. https://www.microsoft.com/en-us/cognitive-toolkit/.

[4]

2018. ConvNetJS. https://cs.stanford.edu/people/karpathy/convnetjs/.

[5]

2018. Keras.js. https://github.com/transcranial/keras-js.

[6]

2018. MIL WebDNN Benchmark. https://mil-tokyo.github.io/webdnn/#benchmar.

[7]

2018. Mind. https://github.com/stevenmiller888/mind.

[8]

2018. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/.

[9]

2018. MorphCast. https://www.morphcast.com/.

[10]

2018. Sklearn DecisionTreeRegressor. https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html.

[11]

2018. synaptic.js. https://github.com/cazala/synaptic.

[12]

2018. Teachable Machine. https://teachablemachine.withgoogle.com/.

[13]

2018. TensorFlow Playgournd. http://playground.tensorflow.org.

[14]

2018. TensorFlow.js. https://js.tensorflow.org/.

[15]

2018. WebDNN. https://github.com/mil-tokyo/webdnn.

[16]

2018. WebGL. https://www.khronos.org/webgl/.

[17]

2018. WebGPU. https://www.w3.org/community/gpu/.

[18]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, (OSDI). 265-283.

Digital Library

[19]

Michael Auer. 2012. Real-time Web GIS Analysis Using WebGL. International Journal of 3-D Information Modeling (IJ3DIM) 1, 3(2012), 49-61.

[20]

Bijin Chen and Zhiqi Xu. 2011. A Framework for Browser-Based Multiplayer Online Games Using WebGL and WebSocket. In Proceedings of 2011 International Conference on Multimedia Technology (ICMT). 471-474.

[21]

Mauricio Guignard, Marcelo Schild, Carlos S. Bederián, Nicolás Wolovick, and Augusto J. Vega. 2018. Performance Characterization of State-Of-The-Art Deep Learning Workloads on an IBM ”Minsky” Platform. In Proceedings of the 51st Hawaii International Conference on System Sciences, HICSS 2018.

[22]

Song Han, Huizi Mao, and William J. Dally. 2015. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. CoRR abs/1510.00149(2015). http://arxiv.org/abs/1510.00149

[23]

Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. 2006. A Fast Learning Algorithm for Deep Belief Nets. Neural computation 18, 7 (2006), 1527-1554.

Digital Library

[24]

Ayae Ichinose, Atsuko Takefusa, Hidemoto Nakada, and Masato Oguchi. 2018. Performance Evaluation of Pipeline-Based Processing for the Caffe Deep Learning Framework. IEICE Transactions 101-D (2018), 1042-1052.

[25]

Andrey Ignatov, Radu Timofte, William Chou, Ke Wang, Max Wu, Tim Hartley, and Luc Van Gool. 2018. AI Benchmark: Running Deep Neural Networks on Android Smartphones. In Computer Vision - ECCV 2018 Workshops. 288-314.

[26]

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor N. Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems, (ASPLOS). 615-629.

Digital Library

[27]

Nicholas D Lane and Petko Georgiev. 2015. Can Deep Learning Revolutionize Mobile Sensing?. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (HotMobile). 117-122.

Digital Library

[28]

Yann LeCun 1989. Generalization and Network Design Strategies. Connectionism in perspective(1989), 143-155.

[29]

Weiqing Liu, Jiannong Cao, Lei Yang, Lin Xu, Xuanjia Qiu, and Jing Li. 2017. AppBooster: Boosting the Performance of Interactive Mobile Applications with Computation Offloading and Parameter Tuning. IEEE Transactions on Parallel and Distributed Systems 28, 6 (2017), 1593-1606.

Digital Library

[30]

Y. Liu, P. Sun, M. R. Highsmith, N. M. Wergeles, J. Sartwell, A. Raedeke, M. Mitchell, H. Hagy, A. D. Gilbert, B. Lubinski, and Y. Shang. 2018. Performance Comparison of Deep Learning Techniques for Recognizing Birds in Aerial Images. In 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). 317-324.

[31]

Bernd Malle, Nicola Giuliani, Peter Kieseberg, and Andreas Holzinger. 2018. The Need for Speed of AI Applications: Performance Comparison of Native vs. Browser-based Algorithm Implementations. CoRR abs/1802.03707(2018). http://arxiv.org/abs/1802.03707

[32]

Chris Marrin. 2011. WebGL specification. Khronos WebGL Working Group(2011).

[33]

Edward Meeds, Remco Hendriks, Said Al Faraby, Magiel Bruntink, and Max Welling. 2015. MLitB: machine learning in the browser. PeerJ Computer Science 1(2015), e11.

[34]

Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L. Lewis, and Satinder P. Singh. 2015. Action-Conditional Video Prediction using Deep Networks in Atari Games. In NIPS.

Digital Library

[35]

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533.

[36]

Shaohuai Shi, Qiang Wang, and Xiaowen Chu. 2018. Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs. In Proceedings of IEEE DASC/PiCom/DataCom/CyberSciTech 2018. 949-957.

[37]

Surat Teerapittayanon, Bradley McDanel, and HT Kung. 2017. Distributed Deep Neural Networks over the Cloud, the Edge and End Devices. In Proceedings of IEEE 37th International Conference on Distributed Computing Systems (ICDCS). 328-339.

[38]

Aaron Tucker, Adam Gleave, and Stuart Russell. 2018. Inverse Reinforcement Learning for Video Games. arXiv preprint arXiv:1810.10593(2018).

[39]

Ji Wang, Jianguo Zhang, Weidong Bao, Xiaomin Zhu, Bokai Cao, and Philip S. Yu. 2018. Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, (KDD). 2407-2416.

Digital Library

[40]

Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. In Proceedings of the 26th International Conference on World Wide Web. 351-360.

Digital Library

Cited By

Zou YZhai JFang CLiu JZheng TChen ZFilkov VRay BZhou M(2024)Mutation-Based Deep Learning Framework Testing Method in JavaScript EnvironmentProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695478(970-981)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695478
Righetto LKhademizadeh MGiachetti APonchio FGigilashvili DBettio FGobbetti E(2024)Efficient and User-Friendly Visualization of Neural Relightable Images for Cultural Heritage ApplicationsJournal on Computing and Cultural Heritage 10.1145/369039017:4(1-24)Online publication date: 7-Dec-2024
https://dl.acm.org/doi/10.1145/3690390
Jia FJiang SCao TCui WXia TCao XLi YWang QZhang DRen JLiu YQiu LYang MOkoshi TKo JLiKamWa R(2024)Empowering In-Browser Deep Learning Inference on Edge Through Just-In-Time Kernel OptimizationProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661892(438-450)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661892
Show More Cited By

Recommendations

The visible Web browser

As an aid to the study of the World-Wide Web, we have developed a software application that allows a user to observe the messages passed between a Web browser and a Web server. The application is based on the Mozilla Web Browser, and displays the HTTP ...
Demystifying Mobile Extended Reality in Web Browsers: How Far Can We Go?
WWW '23: Proceedings of the ACM Web Conference 2023

Mobile extended reality (XR) has developed rapidly in recent years. Compared with the app-based XR, XR in web browsers has the advantages of being lightweight and cross-platform, providing users with a pervasive experience. Therefore, many frameworks ...
The visible Web browser
ITiCSE '99: Proceedings of the 4th annual SIGCSE/SIGCUE ITiCSE conference on Innovation and technology in computer science education

As an aid to the study of the World-Wide Web, we have developed a software application that allows a user to observe the messages passed between a Web browser and a Web server. The application is based on the Mozilla Web Browser, and displays the HTTP ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: The World Wide Web Conference

May 2019

3620 pages

ISBN:9781450366748

DOI:10.1145/3308558

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
805
Total Downloads

Downloads (Last 12 months)98
Downloads (Last 6 weeks)10

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zou YZhai JFang CLiu JZheng TChen ZFilkov VRay BZhou M(2024)Mutation-Based Deep Learning Framework Testing Method in JavaScript EnvironmentProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695478(970-981)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695478
Righetto LKhademizadeh MGiachetti APonchio FGigilashvili DBettio FGobbetti E(2024)Efficient and User-Friendly Visualization of Neural Relightable Images for Cultural Heritage ApplicationsJournal on Computing and Cultural Heritage 10.1145/369039017:4(1-24)Online publication date: 7-Dec-2024
https://dl.acm.org/doi/10.1145/3690390
Jia FJiang SCao TCui WXia TCao XLi YWang QZhang DRen JLiu YQiu LYang MOkoshi TKo JLiKamWa R(2024)Empowering In-Browser Deep Learning Inference on Edge Through Just-In-Time Kernel OptimizationProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661892(438-450)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661892
Liu JZhai YZhao GXu HFang JZeng ZZhu YChua TNgo CKa-Wei Lee RKumar RLauw H(2024)InArt: In-Network Aggregation with Route Selection for Accelerating Distributed TrainingProceedings of the ACM Web Conference 202410.1145/3589334.3645394(2879-2889)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645394
Peng HBao SLi L(2024)A Survey of Security Protection Methods for Deep Learning ModelIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33143985:4(1533-1553)Online publication date: Apr-2024
https://doi.org/10.1109/TAI.2023.3314398
Tian DMa YHan YYang QYang HHuang G(2024)WPIA: accelerating DNN warm-up in Web browsers by precompiling WebGL programsFrontiers of Computer Science10.1007/s11704-024-40066-w18:6Online publication date: 25-Jun-2024
https://doi.org/10.1007/s11704-024-40066-w
Wu RHan JRamos JLipani A(2024)Building and Evaluating a WebApp for Effortless Deep Learning Model DeploymentAdvances in Information Retrieval10.1007/978-3-031-56069-9_26(246-250)Online publication date: 23-Mar-2024
https://doi.org/10.1007/978-3-031-56069-9_26
Pan DLi ZLing LFei G(2023)FLOW: Filtering and LSTM-based Optimization for Web Browser InteractionsProceedings of the Eleventh International Symposium of Chinese CHI10.1145/3629606.3629611(37-43)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3629606.3629611
Huang KChen BWu SCao JMa LPeng XChandra SBlincoe KTonella P(2023)Demystifying Dependency Bugs in Deep Learning StackProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616325(450-462)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616325
Yi RCao TZhou AMa XWang SXu MHui PAmiri Sani ANurmi PLiu Y(2023)Boosting DNN Cold Inference on Edge DevicesProceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services10.1145/3581791.3596842(516-529)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3581791.3596842
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents