[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Log in

Structure and motion in urban environments using upright panoramas

  • SI: Mixed and Augmented Reality
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Image-based modeling of urban environments is a key component of enabling outdoor, vision-based augmented reality applications. The images used for modeling may come from off-line efforts, or online user contributions. Panoramas have been used extensively in mapping cities and can be captured quickly by an end-user with a mobile phone. In this paper, we describe and evaluate a reconstruction pipeline for upright panoramas taken in an urban environment. We first describe how panoramas can be aligned to a common vertical orientation using vertical vanishing point detection, which we show to be robust for a range of inputs. The orientation sensors in modern cameras can also be used to correct the vertical orientation. Secondly, we introduce a pose estimation algorithm, which uses knowledge of a common vertical orientation as a simplifying constraint. This procedure is shown to reduce pose estimation error in comparison with the state of the art. Finally, we evaluate our reconstruction pipeline with several real-world examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Panoramas available from: http://tracking.mat.ucsb.edu.

  2. From Occipital Inc.: http://www.occipital.com/360.

References

  • Antone M, Teller S (2002) Scalable extrinsic calibration of omni-directional image networks. Int J Comput Vis 49:143–174

    Article  MATH  Google Scholar 

  • Baatz G, Köser K, Chen D, Grzeszczuk R, Pollefeys M (2010) Handling urban location recognition as a 2d homothetic problem. In: Daniilidis K, Maragos P, Paragios N (eds) Computer vision—ECCV 2010, lecture notes in computer science, Springer, Berlin, pp 266–279

    Google Scholar 

  • Fraundorfer F, Tanskanen P, Pollefeys M (2010) A minimal case solution to the calibrated relative pose problem for the case of two known orientation angles. In: Proceedings of the 11th European conference on computer vision: part IV, ECCV’10, Springer, Berlin, pp 269–282

  • Gallagher AC (2005) Using vanishing points to correct camera rotation in images. In: Proceedings of the 2nd Canadian conference on computer and robot vision, CRV ’05, IEEE Computer Society, Washington, pp 460–467

  • Haralick RM, Lee CN, Ottenberg K, Nölle M (1994) Review and analysis of solutions of the three point perspective pose estimation problem. Int J Comput Vis 13:331–356

    Article  Google Scholar 

  • Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, ISBN: 0521540518

  • Horn BKP, Hilden H, Negahdaripour S (1988) Closed-form solution of absolute orientation using orthonormal matrices. J Opt Soc Am 5(7):1127–1135

    Article  MathSciNet  Google Scholar 

  • Kosecka J, Zhang W (2002) Video compass. In: Proceedings of the 7th European conference on computer vision-part IV, ECCV ’02, Springer, London, pp 476–490

  • Kukelova Z, Bujnak M, Pajdla T (2011) Closed-form solutions to minimal absolute pose problems with known vertical direction. In: Computer vision—ACCV 2010, lecture notes in computer science, vol 6493, pp 216–229

  • Lepetit V, Moreno-Noguer F, Fua P (2009) Epnp: An accurate o(n) solution to the pnp problem. Int J Comput Vis 81:155–166

    Article  Google Scholar 

  • Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  • Micusik B, Kosecka J (2009) Piecewise planar city 3d modeling from street view panoramic sequences. IEEE Conference on computer vision and pattern recognition (CVPR), Miami, USA, pp 2906–2912

  • Nistér D (2004) An efficient solution to the five-point relative pose problem. IEEE Trans Pattern Anal Mach Intell 26:756–777

    Article  Google Scholar 

  • Pollefeys M, Nistér D, Frahm JM, Akbarzadeh A, Mordohai P, Clipp B, Engels C, Gallup D, Kim SJ, Merrell P, Salmi C, Sinha S, Talton B, Wang L, Yang Q, Stewénius H, Yang R, Welch G, Towles H (2008) Detailed real-time urban 3d reconstruction from video. Int J Comput Vis 78:143–167

    Article  Google Scholar 

  • Robertson D, Cipolla R (2004) An image-based system for urban navigation. In: British machine vision conference, pp 819–828

  • Rother C (2002) A new approach to vanishing point detection in architectural environments. Image Vis Comput 20(9-10):647–655

    Article  Google Scholar 

  • Snavely N, Seitz SM, Szeliski R (2006) Photo tourism: exploring photo collections in 3d. In: ACM SIGGRAPH 2006 papers, SIGGRAPH ’06, ACM, New York, pp 835–846

  • Tardif JP, Pavlidis Y, Daniilidis K (2008) Monocular visual odometry in urban environments using an omnidirectional camera. In: IROS’08, pp 2531–2538

  • Torii A, Havlena M, Pajdla T (2009) From google street view to 3d city models. In: Computer vision workshops (ICCV Workshops), 2009 IEEE 12th international conference on 2009

  • Werner T, Pajdla T (2001) Cheirality in epipolar geometry. Computer vision, IEEE international conference on 1: 548

Download references

Acknowledgments

Thanks to Chris Coffin and Sehwan Kim for preparing the tripod panorama datasets, and to Google, Inc. for providing the Street View datasets. This work was partially supported by NSF CAREER grant IIS-0747520.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonathan Ventura.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ventura, J., Höllerer, T. Structure and motion in urban environments using upright panoramas. Virtual Reality 17, 147–156 (2013). https://doi.org/10.1007/s10055-012-0208-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-012-0208-3

Keywords

Navigation