Dense and scalable reconstruction from unstructured videos with occlusions
Pages 53 - 60
Abstract
Depth-map-based multi-view stereo algorithms typically recover textureless surfaces by assuming smoothness per view, so they require processing different views to solve occlusions. Moreover, the highly redundant viewpoints of videos make exhaustive calculation of depth maps unfeasible for large scenes. This paper achieves dense and scalable reconstruction from videos by adaptively selecting a minimum subset of views from the unstructured camera paths, that are most beneficial for incremental occlusion handling and coverage improvement. Furthermore, we simplify and optimize each set of locally consistent points as the points accumulated from a cluster of previously processed views. By combining content-aware view selection and clustering, as well as cluster-wise point merging, our approach can reduce both computational and memory costs while producing accurate, concise, and dense 3D points, even for homogeneous areas. The superior efficiency and point-level fashion of our operations facilitate 3D modeling at large scales.
References
[1]
{BBH08} Bradley D., Boubekeur T., Heidrich W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In CVPR (2008). 2
[2]
{BFL12} Bailer C., Finckh M., Lensch H. P. A.: Scale robust multi view stereo. In ECCV (2012). 2, 5
[3]
{CL96} Curless B., Levoy M.: A volumetric method for building complex models from range images. ACM Trans. Graph. 30 (1996), 303--312. 2
[4]
{CT11} Calakli F., Taubin G.: Ssd: Smooth signed distance surface reconstruction. Computer Graphics Forum 30, 7 (2011), 1993--2002. 2
[5]
{DF09} Dunn E., Frahm J. M.: Next best view planning for active model improvement. In BMVC (2009). 2
[6]
{ESC14} Engel J., Schops T., Cremers D.: Lsd-slam: Large-scale direct monocular slam. In ECCV (2014). 5
[7]
{FCSS10} Furukawa Y., Curless B., Seitz S. M., Szeliski R.: Towards internet-scale multi-view stereo. In CVPR (2010). 2, 5
[8]
{FG11} Fuhrmann S., Goesele M.: Fusion of depth maps with multiple scales. In Proc. SIGGRAPH Asia (2011). 2, 3
[9]
{FG14} Fuhrmann S., Goesele M.: Floating scale surface reconstruction. ACM Trans. Graph. 33, 4 (2014), 46. 2, 4, 5
[10]
{FTF*15} Fioraio N., Taylor J., Fitzgibbon A., Di Stefano L., Izadi S.: Large-scale and drift-free surface reconstruction using online subvolume registration. In CVPR (2015). 2
[11]
{GFMP08} Gallup D., Frahm J. M., Mordohai P., Pollefeys M.: Variable baseline/resolution stereo. In CVPR (2008). 2
[12]
{GFP10} Gallup D., Frahm J. M., Pollefeys M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In CVPR (2010). 1
[13]
{GSC*07} Goesele M., Snavely N., Curless B., Hoppe H., Seitz S. M.: Multi-view stereo for community photo collections. In ICCV (2007). 2
[14]
{HH12} Haner S., Heyden A.: Covariance propagation and next best view planning for 3d reconstruction. In ECCV (2012). 2
[15]
{HM12} Hu X., Mordohai P.: Least commitment, viewpointbased, multi-view stereo. In 3DIMPVT (2012). 2
[16]
{HZK08} Hornung A., Zeng B., Kobbelt L.: Image selection for improved multi-view stereo. In CVPR (2008). 2
[17]
{KH13} Kazhdan M., Hoppe H.: Screened poisson surface reconstruction. ACM Trans. Graph. 32, 3 (2013), 29. 2
[18]
{KHSM16} Kuhn A., Hirschmüller H., Scharstein D., Mayer H.: A tv prior for high-quality scalable multi-view stereo reconstruction. Int. J. Comput. Vision (2016). 2
[19]
{KM15a} Kang Z., Medioni G.: Progressive 3d model acquisition with a commodity hand-held camera. In WACV (2015). 2
[20]
{KM15b} Kuhn A., Mayer H.: Incremental division of very large point clouds for scalable 3d surface reconstruction. In ICCV Workshops (2015). 2
[21]
{KTSP14} Kolev K., Tanskanen P., Speciale P., Pollefeys M.: Turning mobile phones into 3d scanners. In CVPR (2014). 2
[22]
{KZP*13} Kim C., Zimmer H., Pritch Y., Sorkine-Hornung A., Gross M. H.: Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 4 (2013), 73:1--73:12. 5
[23]
{LIN09} Ladikos A., Ilic S., Navab N.: Spectral camera clustering. In ICCV Workshops (2009). 2
[24]
{LLCX10} Li J., Li E., Chen Y., Xu L.: Bundled depth-map merging for multi-view stereo. In CVPR (2010). 2
[25]
{LPVG16} Locher A., Perdoch M., Van Gool L.: Progressive prioritized multi-view stereo. In CVPR (2016). 2
[26]
{MAW*07} Merrell P., Akbarzadeh A., Wang L., Mordohai P., Frahm J. M., Yang R., Nister D., Pollefeys M.: Real-time visibility-based fusion of depth maps. In ICCV (2007). 2
[27]
{MHPB16} Mendez O., Hadfield S., Pugeault N., Bowden R.: Next-best stereo: Extending next-best view optimisation for collaborative sensors. In BMVC (2016). 2
[28]
{MKG11} Múcke P., Klowsky R., Goesele M.: Surface reconstruction from multi-resolution sample points. In VMV (2011). 2
[29]
{MRS*14a} Mauro M., Riemenschneider H., Signoroni A., Leonardi R., Van Gool L.: An integer linear programming model for view selection on overlapping camera clusters. In 3DV (2014). 2
[30]
{MRS*14b} Mauro M., Riemenschneider H., Signoroni A., Leonardi R., Van Gool L., Brescia I.: A unified framework for content-aware view selection and planning through view importance. In BMVC (2014). 2
[31]
{MRVG*13} Mauro M., Riemenschneider H., Van Gool L., Leonardi R., Brescia I.: Overlapping camera clustering through dominant sets for scalable 3d reconstruction. In BMVC (2013). 2
[32]
{ND10} Newcombe R. A., Davison A. J.: Live dense reconstruction with a single moving camera. In CVPR (2010). 2
[33]
{NZIS13} Niessner M., Zollhöfer M., Izadi S., Stamminger M.: Real-time 3d reconstruction at scale using voxel hashing. ACM Trans. Graph. 32, 6 (2013), 169. 2
[34]
{OKI15} Ondruska P., Kohli P., Izadi S.: Mobilefusion: Real-time volumetric surface reconstruction and dense tracking on mobile phones. IEEE Trans. Vis. Comput. Graph. 21, 11 (2015), 1--1. 2
[35]
{PFS14} Pizzoli M., Forster C., Scaramuzza D.: Remode: Probabilistic, monocular dense reconstruction in real time. In ICRA (2014). 2
[36]
{PKMR15} Prisacariu V. A., Kähler O., Murray D. W., Reid I. D.: Real-time 3d tracking and reconstruction on mobile phones. IEEE Trans. Vis. Comput. Graph. 21, 5 (2015), 557--570. 2
[37]
{PNF*08} Pollefeys M., Nistér D., Frahm J.-M., Akbarzadeh A., Mordohai P., Clipp B., Engels C., Gallup D., Kim S.-J., Merrell P., Salmi C., Sinha S., Talton B., Wang L., Yang Q., Stewénius H., Yang R., Welch G., Towles H.: Detailed real-time urban 3d reconstruction from video. Int. J. Comput. Vision 78, 2 (2008), 143--167. 2
[38]
{RLW*15} Resch B., Lensch H. P. A., Wang O., Pollefeys M., Solkine-Hornung A.: Scalable structure from motion for densely sampled videos. In CVPR (2015). 2
[39]
{SOS05} Shen C., O'Brien J. F., Shewchuk J. R.: Interpolating and approximating implicit surfaces from polygon soup. ACM Trans. Graph. (Proc. ACM SIGGRAPH) (2005), 896--904. 2
[40]
{SSHP17} Schöps T., Sattler T., Häne C., Pollefeys M.: Large-scale outdoor 3d reconstruction on a mobile device. Comput. Vis. Image Unders. 157 (2017), 151--166. 2
[41]
{STO13} Sugiura T., Torii A., Okutomi M.: 3d surface extraction using incremental tetrahedra carving. In ICCV Workshops (2013). 2
[42]
{SZV*12} Schroers C., Zimmer H., Valgaerts L., Bruhn A., Demetz O., Weickert J.: Anisotropic range image integration. In DAGM (2012). 2
[43]
{TS17} Thomas D., Sugimoto A.: Modeling large-scale indoor scenes with rigid fragments using rgb-d cameras. Comput. Vis. Image Unders. 157 (2017), 103--116. 2
[44]
{WRL14} Wei J., Resch B., Lensch H. P. A.: Multi-view depth map estimation with cross-view consistency. In BMVC (2014). 5
[45]
{WRL16} Wei J., Resch B., Lensch H. P. A.: Dense and occlusion-robust multi-view stereo for unstructured videos. In CRV (2016). 1, 2, 3, 5
[46]
{ZCI*08} Zaharescu A., Cagniart C., Ilic S., Boyer E., Horaud R.: Camera-clustering for multi-resolution 3-d surface reconstruction. In M2SFA2 (2008). 2
[47]
{ZDRF12} Zheng E., Dunn E., Raguram R., Frahm J. M.: Efficient and scalable depthmap fusion. In BMVC (2012). 2
- Dense and scalable reconstruction from unstructured videos with occlusions
Recommendations
Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera
Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In ...
Dense Rigid Reconstruction from Unstructured Discontinuous Video
ICCVW '15: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)Although 3D reconstruction from a monocular video has been an active area of research for a long time, and the resulting models offer great realism and accuracy, strong conditions must be typically met when capturing the video to make this possible. ...
Efficient Dense Reconstruction from Video
CVMP '11: Proceedings of the 2011 Conference for Visual Media ProductionWe present a framework for efficient reconstruction of dense scene structure from video. Sequential structure-from-motion recovers camera information from video, providing only sparse 3D points. We build a dense 3D point cloud by performing full-frame ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2017
175 pages
ISBN:9783038680499
- Conference Chairs:
- Matthias Hullin,
- Reinhard Klein,
- Thomas Schultz,
- Angela Yao
Publisher
Eurographics Association
Goslar, Germany
Publication History
Published: 25 September 2017
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Jan 2025