Abstract
Image stitching plays a crucial role for various computer vision applications, like panoramic photography, video production, medical imaging and satellite imagery. It makes it possible to align two images captured at different views onto a single image with a wider field of view. However, for 3D scenes with high depth complexity and images captured from two different positions, the resulting image pair may exhibit significant parallaxes. Stitching images with multiple or large apparent motion shifts remains a challenging task, and existing methods often fail in such cases. In this paper, a novel image stitching pipeline is introduced, addressing the aforementioned challenge: First, iterative dense feature matching is performed, which results in a multi-homography decomposition. Then, this output is used to compute a per-pixel multidimensional weight map of the estimated homographies for image alignment via weighted warping. Additionally, the homographic image space decomposition is exploited using combinatorial analysis to identify parallaxes, resulting in a parallax-aware overlapping region: Parallax-free overlapping areas only require weighted warping and blending. For parallax areas, these operations are omitted to avoid ghosting artifacts. Instead, histogram- and mask-based color mapping is performed to ensure visual color consistency. The presented experiments demonstrate that the proposed method provides superior results regarding precision and handling of parallaxes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Acharya, T., Ray, A.K.: Image Processing - Principles and Applications (2005)
Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)
Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_12
Chen, Y., Xue, W., Chen, S.: Large parallax image stitching via structure preservation and multi-matching. In: Zhang, H., et al. (eds.) NCAA 2022. CCIS, vol. 1638, pp. 177–191. Springer, Singapore (2022). https://doi.org/10.1007/978-981-19-6135-9_14
Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 49–56 (2011)
Gao, J., Li, Y., Chin, T.J., Brown, M.S.: Seam-driven image stitching. In: Eurographics (2013)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision (2003)
Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Jia, Q., Li, Z., Fan, X., Zhao, H., Teng, S., Ye, X., Latecki, L.J.: Leveraging line-point consistence to preserve structures for wide parallax image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 12181–12190 (2021)
Lee, K.Y., Sim, J.Y.: Stitching for multi-view videos with large parallax based on adaptive pixel warping. IEEE Access 6, 26904–26917 (2018)
Lee, K.Y., Sim, J.Y.: Warping residual based image stitching for large parallax. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8195–8203 (2020)
Li, J., Wang, Z., Lai, S., Zhai, Y., Zhang, M.: Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimed. 20, 1672–1687 (2018)
Li, N., Xu, Y., Wang, C.: Quasi-homography warps in image stitching. IEEE Trans. Multimed. 20, 1365–1375 (2017)
Liao, T., Li, N.: Single-perspective warps in natural image stitching. IEEE Trans. Image Process. 29, 724–735 (2020)
Lin, C.C., Pankanti, S.U., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1163 (2015)
Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_23
Liu, W.X., Chin, T.J.: Correspondence insertion for as-projective-as-possible image stitching. ArXiv (2016)
Lyu, W., Zhou, Z., Chen, L., Zhou, Y.: A survey on image and video stitching. Virtual Reality Intell. Hardw. 1, 55–83 (2019)
Megha, V., Rajkumar, K.K.: Automatic satellite image stitching based on speeded up robust feature. In: International Conference on Artificial Intelligence and Machine Vision, pp. 1–6 (2021)
Nie, L., Lin, C., Liao, K., Liu, M., Zhao, Y.: A view-free image stitching network based on global homography. J. Vis. Commun. Image Representation 73, 102950 (2020)
Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Unsupervised deep image stitching: reconstructing stitched features to images. IEEE Trans. Image Process. 30, 6184–6197 (2021)
Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2022–2038 (2013)
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Seibt, S., Von Rymon Lipinski, B., Latoschik, M.E.: Dense feature matching based on homographic decomposition. IEEE Access 10, 21236–21249 (2022)
Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends Comput. Graph. Vision 2, 1–104 (2006)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Win, K.P., Kitjaidure, Y., Hamamoto, K.: Automatic stitching of medical images using feature based approach. Adv. Sci. Technol. Eng. Syst. J. 4, 127–133 (2019)
Xue, W., Xie, W., Zhang, Y., Chen, S.: Stable linear structures and seam measurements for parallax image stitching. IEEE Trans. Circuits Syst. Video Technol. 32, 253–261 (2022)
Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2339–2346 (2013)
Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269 (2014)
Zhang, G., He, Y., Chen, W., Jia, J., Bao, H.: Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans. Image Process. 25, 3099–3111 (2016)
Zhao, Q., Ma, Y., Zhu, C., Yao, C., Feng, B., Dai, F.: Image stitching via deep homography estimation. Neurocomputing 450, 219–229 (2021)
Zhao, Q., Wan, L., Feng, W., Zhang, J., Wong, T.T.: Cube2video: navigate between cubic panoramas in real-time. IEEE Trans. Multimed. 15, 1745–1754 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Seibt, S., Arold, M., von Rymon Lipinski, B., Wienkopf, U., Latoschik, M.E. (2024). Parallax-Aware Image Stitching Based on Homographic Decomposition. In: Köthe, U., Rother, C. (eds) Pattern Recognition. DAGM GCPR 2023. Lecture Notes in Computer Science, vol 14264. Springer, Cham. https://doi.org/10.1007/978-3-031-54605-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-54605-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54604-4
Online ISBN: 978-3-031-54605-1
eBook Packages: Computer ScienceComputer Science (R0)