[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Scalable 3D video of dynamic scenes

  • original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper we present a scalable 3D video framework for capturing and rendering dynamic scenes. The acquisition system is based on multiple sparsely placed 3D video bricks, each comprising a projector, two grayscale cameras, and a color camera. Relying on structured light with complementary patterns, texture images and pattern-augmented views of the scene are acquired simultaneously by time-multiplexed projections and synchronized camera exposures. Using space–time stereo on the acquired pattern images, high-quality depth maps are extracted, whose corresponding surface samples are merged into a view-independent, point-based 3D data structure. This representation allows for effective photo-consistency enforcement and outlier removal, leading to a significant decrease of visual artifacts and a high resulting rendering quality using EWA volume splatting. Our framework and its view-independent representation allow for simple and straightforward editing of 3D video. In order to demonstrate its flexibility, we show compositing techniques and spatiotemporal effects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Alexa, M., Gross, M., Pauly, M., Pfister, H., Stamminger, M., Zwicker, M.: Point-Based Computer Graphics. SIGGRAPH ’04 Course Notes (2004)

  2. Bayakovski, Y., Levkovich-Maslyuk, L., Ignatenko, A., Konushin, A., Timasov, D., Zhirkov, A., Han, M., Park, I.K.: Depth image-based representations for static and animated 3D objects. In: ICIP ’02, 3, 25–28 (2002)

  3. Bouguet, J.Y.: Camera calibration toolbox for matlab, http://www.vision.caltech.edu/ bouguetj/calib_doc

  4. Broadhurst, A., Drummond, T., Cipolla, R.: A probabilistic framework for the space carving algorithm. In: ICCV ’01, pp. 388–393 (2001)

  5. Buehler, C., Bosse, M., McMillan, L., Gortler, S., Cohen, M.: Unstructured lumigraph rendering. In: SIGGRAPH ’01, pp. 425–432 (2001)

  6. Carranza, J., Theobalt, C., Magnor, M., Seidel, H.P.: Free-viewpoint video of human actors. In: SIGGRAPH ’03, pp. 569–577 (2003)

  7. Cockshott, W.P., Hoff, S., Nebel, J.C.: An experimental 3D digital TV studio. In: Vision, Image & Signal Processing ’03, pp. 28–33 (2003)

  8. Cotting, D., Naef, M., Gross, M., Fuchs, H.: Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In: ISMAR ’04, pp. 100–109 (2004)

  9. Goldlücke, B., Magnor, M., Wilburn, B.: Hardware-accelerated dynamic light field rendering. In: VMV ’02, pp. 455–462 (2002)

  10. Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: SIGGRAPH ’96, pp. 43–54 (1996)

  11. Gross, M., Würmlin, S., Näf, M., Lamboray, E., Spagno, C., Kunz, A., Moere, A.V., Strehlke, K., Lang, S., Svoboda, T., Koller-Meier, E., Gool, L.V., Staadt, O.: blue-c: A spatially immersive display and 3D video portal for telepresence. In: SIGGRAPH ’03, pp. 819–827 (2003)

  12. Hofsetz, C., Ng, K., Max, N., Chen, G., Liu, Y., McGuinness, P.: Image-based rendering of range data with estimated depth uncertainty. IEEE CG&A 24(4), 34–42 (2005)

    Google Scholar 

  13. Iddan, G.J., Yahav, G.: 3D imaging in the studio (and elsewhere...). In: SPIE ’01, 4298, 48–55 (2001)

  14. Kanade, T., Rander, P., Narayanan, P.J.: Virtualized reality: construction of virtual worlds from real scenes. IEEE Multimedia 4(1), 34–47 (1997)

    Google Scholar 

  15. Kang, S., Szeliski, R.: Boundary matting for view synthesis. In: CVPRW ’04 (2004)

  16. Kang, S., Webb, J., Zitnick, L., Kanade, T.: A multi-baseline stereo system with active illumination and real-time image acquisition. In: ICCV ’95, pp. 88–93 (1995)

  17. Levin, D.: Mesh-independent surface interpolation. In: Geometric Modeling for Scientific Visualization, pp. 37–49, ed. by Brunnett, Hamann, Mueller, Springer 2003

  18. Levoy, M., Hanrahan, P.: Light field rendering. In: SIGGRAPH ’96, pp. 31–42 (1996)

  19. Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: EGRW ’01, pp. 115–125 (2001)

  20. Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: SIGGRAPH ’00, pp. 369–374 (2000)

  21. Matusik, W., Pfister, H.: 3D TV: A scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In: SIGGRAPH ’04 (2004)

  22. Mulligan, J., Daniilidis, K.: View-independent scene acquisition for tele-presence. In: International Symposium on Augmented Reality, pp. 105–110 (2000)

  23. Pauly, M., Gross, M.: Spectral processing of point sampled geometry. In: SIGGRAPH ’01(2001)

  24. Pauly, M., Gross, M., Kobbelt, L.: Efficient simplification of point-sampled geometry. In: VIS ’02, pp. 163–170 (2002)

  25. Pfister, H., Zwicker, M., van Baar, J., Gross, M.: Surfels: Surface elements as rendering primitives. In: SIGGRAPH ’00, pp. 335–342 (2000)

  26. Redert, A., de Beeck, M.O., Fehn, C., Ijsselsteijn, W., Pollefeys, M., Gool, L.V., Ofek, E., Sexton, I., Surman, P.: ATTEST: Advanced three-dimensional television system technologies. In: 3DPVT ’02, pp. 313–319 (2002)

  27. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Google Scholar 

  28. Shade, J., Gortler, S., He, L.W., Szeliski, R.: Layered depth images. In: SIGGRAPH ’98, pp. 231–242 (1998)

  29. Smolic, A., Kimata, H.: Description of exploration experiments in 3DAV. In: JTC1/SC29/WG11 N6194. ISO/IEC (2003)

  30. Vedula, S., Baker, S., Kanade, T.: Spatio-temporal view interpolation. In: EGRW ’02, pp. 65–76 (2002)

  31. Weyrich, T., Pauly, M., Keiser, R., Heinzle, S., Scandella, S., Gross, M.: Post-processing of scanned 3D surface data. In: Eurographics Symposium on Point-Based Graphics ’04 (2004)

  32. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. In: SIGGRAPH ’05 pp. 765–776 (2005)

  33. Würmlin, S., Lamboray, E., Gross, M.: 3D video fragments: Dynamic point samples for real-time free-viewpoint video. In: Computers and Graphics ’04 28(1), 3–14 (2004)

  34. Würmlin, S., Lamboray, E., Staadt, O.G., Gross, M.H.: 3D video recorder. In: Proceedings of Pacific Graphics 2002, pp. 325–334. IEEE Press, New York (2002)

  35. Yang, J.C., Everett, M., Buehler, C., McMillan, L.: A real-time distributed light field camera. In: EGRW ’02, pp. 77–86 (2002)

  36. Zhang, L., Curless, B., Seitz, S.M.: Spacetime stereo: shape recovery for dynamic scenes. In: CVPR ’03, pp. 367–374 (2003)

  37. Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: high resolution capture for modeling and animation. In: SIGGRAPH ’04, pp. 548–558 (2004)

  38. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: SIGGRAPH ’04, pp. 600–608 (2004)

  39. Zwicker, M., Pfister, H., van Baar, J., Gross, M.: EWA splatting. IEEE Trans. Visual. Comput. Graph. 8(3), 223–238 (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Graphics Laboratory, Department of Computer Science, Swiss Federal Institute of Technology (ETH), Zurich, Switzerland

    Michael Waschbüsch, Stephan Würmlin, Daniel Cotting, Filip Sadlo & Markus Gross

Authors
  1. Michael Waschbüsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephan Würmlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Cotting
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Filip Sadlo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Waschbüsch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Waschbüsch, M., Würmlin, S., Cotting, D. et al. Scalable 3D video of dynamic scenes. Visual Comput 21, 629–638 (2005). https://doi.org/10.1007/s00371-005-0346-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-005-0346-7

Keywords

Search

Navigation