More Web Proxy on the site http://driver.im/

research-article

Real-time remote rendering of 3D video for mobile devices

Authors:

Klara Nahrstedt,

Roy H. CampbellAuthors Info & Claims

MM '09: Proceedings of the 17th ACM international conference on Multimedia

Pages 391 - 400

https://doi.org/10.1145/1631272.1631326

Published: 19 October 2009 Publication History

Abstract

At the convergence of computer vision, graphics, and multimedia, the emerging 3D video technology promises immersive experiences in a truly seamless environment. However, the requirements of huge network bandwidth and computing resources make it still a big challenge to render 3D video on mobile devices at real-time. In this paper, we present how remote rendering framework can be used to solve the problem. The differences between dynamic 3D video and static graphic models are analyzed. A general proxy-based framework is presented to render 3D video streams on the proxy and transmit the rendered scene to mobile devices over a wireless network. An image-based approach is proposed to enhance 3D interactivity and reduce the interaction delay. Experiments prove that the remote rendering framework can be effectively used for quality 3D video rendering on mobile devices in real time.

References

[1]

Bumblebee 2 camera. http://www.ptgrey.com/products/bumblebee2/index.asp.

[2]

Chromium. http://chromium.sourceforge.net/.

[3]

Equalizer: Parallel rendering. http://www.equalizergraphics.com/.

[4]

Glintercept. http://glintercept.nutty.org/.

[5]

Gltrace. http://www.hawksoft.com/gltrace/.

[6]

P. Bao and D. Gourlay. Remote walkthrough over mobile networks using 3-D image warping and streaming. Vision, Image and Signal Processing, IEE Proceedings, 151(4):329--336, Aug. 2004.

[7]

A. Boukerche and R. Pazzi. Remote rendering and streaming of progressive panoramas for mobile devices. In Multimedia '06, pages 691--694. ACM, 2006.

Digital Library

[8]

C.-F. Chang and S.-H. Ger. Enhancing 3D graphics on mobile devices by image-based rendering. In PCM '02, pages 1105--1111, London, UK, 2002.

Digital Library

[9]

H. Kim and et al. An immersive free-viewpoint video system using multiple outer/inner cameras. In 3DPVT, pages 782--789, 2006.

Digital Library

[10]

S.-U. Kum and K. Mayer-Patel. Real-time multidepth stream compression. TOMCCAP, 1(2):128--150, 2005.

Digital Library

[11]

F. Lamberti and A. Sanna. A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Vis. Comput. Graph., 13(2):247--260, 2007.

Digital Library

[12]

J.-M. Lien and et al. Skeleton-based data compression for multi-camera tele-immersion system. In ISVC'07, pages 714--723, November 2007.

Digital Library

[13]

W. R. Mark and et al. Post-rendering 3D warping. In SI3D '97: Proceedings of the 1997 symposium on Interactive 3D graphics, 1997.

Digital Library

[14]

W. Matusik and H. Pfister. 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In SIGGRAPH '04, pages 814--824, 2004.

Digital Library

[15]

L. McMillan. An image-based approach to three-dimensional computer graphics. In Ph.D. Dissertation. University of North Carolina at Chapel Hill, Department of Computer Science, 1997.

Digital Library

[16]

L. McMillan and G. Bishop. Plenoptic modeling: an image-based rendering system. In SIGGRAPH '95, pages 39--46, 1995.

Digital Library

[17]

S. K. Penta and P. J. Narayanan. Compression of multiple depth maps for IBR. The Visual Computer, 21(8-10):611--618, 2005.

[18]

A. Redert and et al. ATTEST: Advanced three-dimensional television system technologies. In 3DPVT '02, pages 313--319, June 2002.

[19]

J. Shade and et al. Layered depth images. In SIGGRAPH '98, pages 231--242, 1998.

Digital Library

[20]

S. Shi and et al. View-dependent real-time 3D video compression for mobile devices. In Multimedia '08, pages 781--784. ACM, 2008.

Digital Library

[21]

H.-Y. Shum, S. B. Kang, and S.-C. Chan. Survey of image-based representations and compression techniques. IEEE Trans. Circuits Syst. Video Techn., 13(11):1020--1037, 2003.

Digital Library

[22]

A. Smolic and et al. 3D video and free viewpoint video - technologies, applications and mpeg standards. In ICME, pages 2161--2164. IEEE, 2006.

[23]

S. Wurmlin and et al. 3D video recorder: a system for recording and playing free-viewpoint video. Comput. Graph. Forum, 22(2):181--194, 2003.

[24]

Z. Yang and et al. TEEVE: The next generation architecture for tele-immersive environment. In ISM '05, pages 112--119, December 2005.

Digital Library

[25]

Z. Yang and et al. Real-time 3D video compression for tele-immersive environments. In MMCN '06, 2006.

[26]

Z. Yang and et al. Viewcast: view dissemination and management for multi-party 3D tele-immersive environments. In Multimedia '07. ACM, 2007.

Digital Library

Cited By

Zhong JZhang HJia QWu JWang PWang HLiu LZhang XGuo Z(2024)Low-bitrate Volumetric Video Streaming with Depth ImageProceedings of the 2024 SIGCOMM Workshop on Emerging Multimedia Systems10.1145/3672196.3673397(39-44)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672196.3673397
Wu JGuan YMao QCui YGuo ZZhang XSchulzrinne HKohler EMaltz DMisra V(2023)ZGaming: Zero-Latency 3D Cloud Gaming by Image PredictionProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604819(710-723)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604819
Lavrič PBohak CMarolt M(2018)Vulkan Abstraction Layer for Large Data Remote Rendering SystemAugmented Reality, Virtual Reality, and Computer Graphics10.1007/978-3-319-95270-3_40(480-488)Online publication date: 14-Jul-2018
https://doi.org/10.1007/978-3-319-95270-3_40
Show More Cited By

Index Terms

Real-time remote rendering of 3D video for mobile devices
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

A high-quality low-delay remote rendering system for 3D video
MM '10: Proceedings of the 18th ACM international conference on Multimedia

As an emerging technology, 3D video has shown a great potential to become the next generation media for tele-immersion. However, streaming and rendering this dynamic 3D data in real-time requires tremendous network bandwidth and computing resources. In ...
A real-time remote rendering system for interactive mobile graphics
Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimedia

Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications ...
Free-viewpoint video rendering for mobile devices
MIRAGE '13: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications

Free-viewpoint video renderers (FVVR) allow a user to view captured video footage from any position and direction. Despite the obvious appeal of such systems, they have yet to make a major impact on digital entertainment. Current FVVR implementations ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '09: Proceedings of the 17th ACM international conference on Multimedia

October 2009

1202 pages

ISBN:9781605586083

DOI:10.1145/1631272

General Chairs:
Wen Gao
Peking University, China
,
Yong Rui
Microsoft, China
,
Alan Hanjalic
Delft University of Technology, The Netherlands
,
Program Chairs:
Changsheng Xu
Institute of Automation, Chinese Academy of Sciences, China
,
Eckehard Steinbach
Technical University of Munich, Germany
,
Abdulmotaleb El Saddik
University of Ottawa, Canada
,
Michelle Zhou
IBM T. J. Watson Research Center, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM09

Sponsor:

SIGMM

MM09: ACM Multimedia Conference

October 19 - 24, 2009

Beijing, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
1,198
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)3

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhong JZhang HJia QWu JWang PWang HLiu LZhang XGuo Z(2024)Low-bitrate Volumetric Video Streaming with Depth ImageProceedings of the 2024 SIGCOMM Workshop on Emerging Multimedia Systems10.1145/3672196.3673397(39-44)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672196.3673397
Wu JGuan YMao QCui YGuo ZZhang XSchulzrinne HKohler EMaltz DMisra V(2023)ZGaming: Zero-Latency 3D Cloud Gaming by Image PredictionProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604819(710-723)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604819
Lavrič PBohak CMarolt M(2018)Vulkan Abstraction Layer for Large Data Remote Rendering SystemAugmented Reality, Virtual Reality, and Computer Graphics10.1007/978-3-319-95270-3_40(480-488)Online publication date: 14-Jul-2018
https://doi.org/10.1007/978-3-319-95270-3_40
Zhu MMorin GCharvillat VOoi W(2017)Sprite treeThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-016-1286-033:11(1385-1402)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s00371-016-1286-0
Zarb TDebono C(2016)Broadcasting Free-Viewpoint Television Over Long-Term Evolution NetworksIEEE Systems Journal10.1109/JSYST.2015.245348310:2(773-784)Online publication date: Jun-2016
https://doi.org/10.1109/JSYST.2015.2453483
Lee JChoi KKim YHan HKang S(2016)Exploiting remote GPGPU in mobile devicesCluster Computing10.1007/s10586-016-0614-519:3(1571-1583)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1007/s10586-016-0614-5
Zhao SOoi W(2016)Modeling 3D synthetic view dissimilarityThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-015-1069-z32:4(429-443)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1007/s00371-015-1069-z
Shi SHsu C(2015)A Survey of Interactive Remote Rendering SystemsACM Computing Surveys10.1145/271992147:4(1-29)Online publication date: 26-May-2015
https://dl.acm.org/doi/10.1145/2719921
Xiaochuan WXiaohui L(2015)Viewpoint-Predicting-Based Remote Rendering on Mobile Devices Using Multiple Depth Images2015 International Conference on Virtual Reality and Visualization (ICVRV)10.1109/ICVRV.2015.23(216-223)Online publication date: Oct-2015
https://doi.org/10.1109/ICVRV.2015.23
Walia EVerma V(2014)A Framework for Interactive 3D Rendering on Mobile DevicesInternational Journal of Computer Vision and Image Processing10.4018/ijcvip.20140401024:2(18-31)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.4018/ijcvip.2014040102
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents