More Web Proxy on the site http://driver.im/

research-article

Joint view expansion and filtering for automultiscopic 3D displays

Authors:

Pitchaya Sitthi-Amorn,

William Freeman,

Wojciech MatusikAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 6

Article No.: 221, Pages 1 - 8

https://doi.org/10.1145/2508363.2508376

Published: 01 November 2013 Publication History

Abstract

Multi-view autostereoscopic displays provide an immersive, glasses-free 3D viewing experience, but they require correctly filtered content from multiple viewpoints. This, however, cannot be easily obtained with current stereoscopic production pipelines. We provide a practical solution that takes a stereoscopic video as an input and converts it to multi-view and filtered video streams that can be used to drive multi-view autostereoscopic displays. The method combines a phase-based video magnification and an interperspective antialiasing into a single filtering process. The whole algorithm is simple and can be efficiently implemented on current GPUs to yield a near real-time performance. Furthermore, the ability to retarget disparity is naturally supported. Our method is robust and works well for challenging video scenes with defocus blur, motion blur, transparent materials, and specularities. We show that our results are superior when compared to the state-of-the-art depth-based rendering methods. Finally, we showcase the method in the context of a real-time 3D videoconferencing system that requires only two cameras.

Supplementary Material

ZIP File (a221-didyk.zip)

Supplemental material.

Download
89.14 MB

References

[1]

Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. 23, 3, 804--813.

Digital Library

[2]

Balogh, T. 2006. The holovizio system. In Electronic Imaging 2006, 60550U-60550U.

[3]

Brown, M. Z., Burschka, D., and Hager, G. D. 2003. Advances in computational stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 8, 993--1008.

Digital Library

[4]

Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 307--318.

Digital Library

[5]

Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., and Seidel, H.-P. 2010. Adaptive image-based stereo view synthesis. In Proc. VMV.

[6]

Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.-P., and Matusik, W. 2012. A luminance-contrast-aware disparity model and applications. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 31, 6, 184:1--184:10.

Digital Library

[7]

Farre, M., Wang, O., Lang, M., Stefanoski, N., Hornung, A., and Smolic, A. 2011. Automatic content creation for multiview autostereoscopic displays using image domain warping. In IEEE International Conference on Multimedia and Expo.

Digital Library

[8]

Holliman, N. S., Dodgson, N. A., Favalora, G. E., and Pockett, L. 2011. Three-dimensional displays: a review and applications analysis. IEEE Transactions on Broadcasting 57, 2, 362--371.

[9]

Hosni, A., Rhemann, C., Bleyer, M., Rother, C., and Gelautz, M. 2013. Fast cost-volume filtering for visual correspondence and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2, 504--511.

Digital Library

[10]

Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., 297--306.

Digital Library

[11]

Lambooij, M., IJsselsteijn, W., Fortuin, M., and Heynderickx, I. 2009. Visual discomfort and visual fatigue of stereoscopic displays: a review. J Imaging Science and Technology 53, 030201--14.

[12]

Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graph. 29, 4, 75:1--75:10.

Digital Library

[13]

Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, 31--42.

Digital Library

[14]

Lipton, L., and Feldman, M. H. 2002. New autostereoscopic display technology: the synthagram. In Electronic Imaging 2002, International Society for Optics and Photonics, 229--235.

[15]

Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24, 3, 519--526.

Digital Library

[16]

Matusik, W., and Pfister, H. 2004. 3d tv: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. 23, 3, 814--824.

Digital Library

[17]

Richardt, C., Stoll, C., Dodgson, N. A., Seidel, H.-P., and Theobalt, C. 2012. Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos. Computer Graphics Forum (Proc. Eurographics) 31, 2, 247--256.

Digital Library

[18]

Schmidt, A., and Grasnick, A. 2002. Multiviewpoint autostereoscopic dispays from 4d-vision gmbh. In Electronic Imaging 2002, 212--221.

[19]

Simoncelli, E. P., and Freeman, W. T. 1995. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In IEEE International Conference on Image Processing, vol. 3, 444--447.

Digital Library

[20]

Simoncelli, E. P., Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1992. Shiftable multiscale transforms. IEEE Transactions on Information Theory 38, 2, 587--607.

Digital Library

[21]

Smolic, A., Muller, K., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Intermediate view interpolation based on multiview video plus depth for advanced 3d video systems. In IEEE International Conference on Image Processing, 2448--2451.

[22]

Stewart, J., Yu, J., Gortler, S. J., and McMillan, L. 2003. A new reconstruction filter for undersampled light fields. In Proceedings of the 14th Eurographics workshop on Rendering, Eurographics Association, 150--156.

Digital Library

[23]

Takaki, Y., and Nago, N. 2010. Multi-projection of lenticular displays to construct a 256-view super multi-view display. Optics Express 18, 9, 8824--8835.

[24]

Wadhwa, N., Rubinstein, M., Guttag, J., Durand, F., and Freeman, W. T. 2013. Phase-based video motion processing. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 4, 80:1--80:10.

Digital Library

[25]

Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612.

Digital Library

[26]

Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 1--11.

Digital Library

[27]

Wilburn, B. S., Smulski, M., Lee, H.-H. K., and Horowitz, M. A. 2001. Light field video camera. In Electronic Imaging 2002, International Society for Optics and Photonics, 29--36.

[28]

Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. T. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 65:1--65:8.

Digital Library

[29]

Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3d displays. In Proceedings of the 17th Eurographics conference on Rendering Techniques, Eurographics Association, 73--82.

Digital Library

Cited By

Varma AVishwakarma H(2024)Performance analysis of flow-based Video Interpolation Techniques2024 Asia Pacific Conference on Innovation in Technology (APCIT)10.1109/APCIT62007.2024.10673434(1-6)Online publication date: 26-Jul-2024
https://doi.org/10.1109/APCIT62007.2024.10673434
Liu JLiu F(2022)Modified 2D-Ghost-Free Stereoscopic Display with Depth-of-Field EffectsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/353496419:1s(1-16)Online publication date: 12-May-2022
https://dl.acm.org/doi/10.1145/3534964
Xiao LNouri SHegland JGarcia ALanman D(2022)NeuralPassthrough: Learned Real-Time View Synthesis for VRACM SIGGRAPH 2022 Conference Proceedings10.1145/3528233.3530701(1-9)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1145/3528233.3530701
Show More Cited By

Index Terms

Joint view expansion and filtering for automultiscopic 3D displays
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

3DTV at home: eulerian-lagrangian stereo-to-multiview conversion

Stereoscopic 3D (S3D) movies have become widely popular in the movie theaters, but the adoption of S3D at home is low even though most TV sets support S3D. It is widely believed that S3D with glasses is not the right approach for the home. A much more ...
Antialiasing for automultiscopic 3D displays
EGSR '06: Proceedings of the 17th Eurographics conference on Rendering Techniques

Automultiscopic displays show stereoscopic images that can be viewed from any viewpoint without special glasses. They hold great promise for the future of television and digital entertainment. However, the image quality on these 3D displays is currently ...
Special Section on Advanced Displays: Display adaptive 3D content remapping

Glasses-free automultiscopic displays are on the verge of becoming a standard technology in consumer products. These displays are capable of producing the illusion of 3D content without the need of any additional eyewear. However, due to limitations in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 6

November 2013

671 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2508363

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013

Published in TOG Volume 32, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Division of Information and Intelligent Systems
Quanta Computer

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
708
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)6

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Varma AVishwakarma H(2024)Performance analysis of flow-based Video Interpolation Techniques2024 Asia Pacific Conference on Innovation in Technology (APCIT)10.1109/APCIT62007.2024.10673434(1-6)Online publication date: 26-Jul-2024
https://doi.org/10.1109/APCIT62007.2024.10673434
Liu JLiu F(2022)Modified 2D-Ghost-Free Stereoscopic Display with Depth-of-Field EffectsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/353496419:1s(1-16)Online publication date: 12-May-2022
https://dl.acm.org/doi/10.1145/3534964
Xiao LNouri SHegland JGarcia ALanman D(2022)NeuralPassthrough: Learned Real-Time View Synthesis for VRACM SIGGRAPH 2022 Conference Proceedings10.1145/3528233.3530701(1-9)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1145/3528233.3530701
Su LYe ZSui YYuan YLiu AZhu C(2022)Epipolar plane images based light-field angular super-resolution networkSeventh Asia Pacific Conference on Optics Manufacture (APCOM 2021)10.1117/12.2615593(146)Online publication date: 15-Feb-2022
https://doi.org/10.1117/12.2615593
Fukiage TNishida S(2022)HiddenGazeStereo: Hiding Gaze-Contingent Disparity Remapping for 2D-Compatible Natural 3D ViewingIEEE Access10.1109/ACCESS.2022.320487410(94778-94796)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3204874
Chen HWang WLi JMo HChen J(2022)Phase-based side information generation in distributed video codingMultimedia Tools and Applications10.1007/s11042-022-12589-681:15(21295-21312)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s11042-022-12589-6
Han TChen SShi J(2021)Frame Rate Up-Conversion Algorithm Based on Infrared ImageProceedings of the 13th International Conference on Computer Modeling and Simulation10.1145/3474963.3475843(225-230)Online publication date: 25-Jun-2021
https://dl.acm.org/doi/10.1145/3474963.3475843
Gao WZhou LTao L(2021)A Fast View Synthesis Implementation Method for Light Field ApplicationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/345909817:4(1-20)Online publication date: 30-Nov-2021
https://dl.acm.org/doi/10.1145/3459098
Shen BWang RLaw AKamath RChoo HKong Z(2021)Super Resolution for Multi-Sources Image Stream Data using Smooth and Sparse Tensor Completion and its Applications in Data Acquisition of Additive ManufacturingTechnometrics10.1080/00401706.2021.1905074(1-41)Online publication date: 19-Mar-2021
https://doi.org/10.1080/00401706.2021.1905074
Parihar AVarshney DPandya KAggarwal A(2021)A comprehensive survey on video frame interpolation techniquesThe Visual Computer10.1007/s00371-020-02016-yOnline publication date: 4-Jan-2021
https://doi.org/10.1007/s00371-020-02016-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents