Hierarchical upsampling for fast image-based depth estimation

While many stereo vision algorithms can quickly and robustly estimate sparse geometry from sets of photos, dense reconstruction, where depth estimate is required at per-pixel or sub-pixel level, remains a time-consuming and memory-intensive process. In this work, we propose a fast hierarchical upsampling method for dense image-based depth estimation. The main idea is to start from sparse depth estimates that can be quickly computed using any existing multiview stereopsis tool, then iteratively upsample the depth values to obtain a dense reconstruction consisting of millions of points. Using a GPU-based implementation, the upsampling algorithm can perform up to 15 images per second. The results can be directly used for 3D modeling applications; in addition, they can be used to digitally manipulate the depth-of-field effects in the input images in order to simulate refocusing.