Abstract
This paper presents a multi-view acquisition system using multi-modal sensors, composed of time-of-flight (ToF) range sensors and color cameras. Our system captures the multiple pairs of color images and depth maps at multiple viewing directions. In order to ensure the acceptable accuracy of measurements, we compensate errors in sensor measurement and calibrate multi-modal devices. Upon manifold experiments and extensive analysis, we identify the major sources of systematic error in sensor measurement and construct an error model for compensation. As a result, we provide a practical solution for the real-time error compensation of depth measurement. Moreover, we implement the calibration scheme for multi-modal devices, unifying the spatial coordinate for multi-modal sensors.
The main contribution of this work is to present the thorough analysis of systematic error in sensor measurement and therefore provide a reliable methodology for robust error compensation. The proposed system offers a real-time multi-modal sensor calibration method and thereby is applicable for the 3D reconstruction of dynamic scenes.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Levoy, M., Hanrahan, P.: Light field rendering. In: Proc. ACM SIGGRAPH, pp. 31–42 (1996)
Scharstein, D.: Stereo vision for view synthesis. In: Proc. Computer Vision and Pattern Recognition, pp. 852–858 (1996)
Woodham, D.R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)
Zhang, R., Tsai, P.-S., Cryer, J.E., Shah, M.: Shape from shading: a survey. In: IEEE Trans. on Pattern Analysis and Machine Intelligence, pp. 690–706 (1999)
Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital Michelangelo project. In: Proc. ACM SIGGRAPH, pp. 131–144 (2000)
Batlle, J., Mouaddib, E.M., Salvi, J.: Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. Pattern Recogn. 1–63 (1998). doi:10.1109/ROBOT.1197.620027
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 1330–1334 (2000). doi:10.1109/34.888718
Kahlmann, T., Ingesand, H.: Calibration and improvements of the high-resolution range-imaging camera SwissRanger TM. In: Conference on Videometrics VIII, Part of the IS&T/SPIE Symposium on Electronic Imaging (2005)
Kahlmann, T., Ingensand, H.: High-precision investigations of the fast range imaging sensor swissranger. In: Optics East (2007)
Fuchs, S., May, S.: Calibration and registration for precise surface reconstruction. In: Proceedings of the DAGM Dyn3D Workshop (2007)
Fuchs, S., Hirzinger, G.: Extrinsic and depth calibration of ToF-cameras. In: Proc. Computer Vision and Pattern Recognition, pp. 1–6 (2008)
Kim, Y.M., Chan, D., Theobalt, C., Thrun, S.: Design and calibration of a multi-view ToF sensor fusion system. In: Proc. Computer Vision and Pattern Recognition (2008)
Guan, L., Pollefeys, M.: A unified approach to calibrate a network of camcorders and ToF cameras. In: Proc. European Conference on Computer Vision (2008)
Guan, L., Franco, J.-S., Pollefeys, M.: 3D object reconstruction with heterogeneous sensor data. In: 3DPVT (2008)
Bouguet, Y.J.: Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/
Gut, O.: Untersuchungen des 3D-sensors swissranger, Diploma thesis
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Fryer, J.G., Brown, D.C.: Lens distortion for close-range photogrammetry. Photogramm. Eng. Remote Sens. 52, 51–58 (1986)
Payne, A., Dorrington, A., Cree, M.: Optimization of 3D range-imaging sensors. SPIE Newsroom (2008)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shim, H., Adelsberger, R., Kim, J.D. et al. Time-of-flight sensor and color camera calibration for multi-view acquisition. Vis Comput 28, 1139–1151 (2012). https://doi.org/10.1007/s00371-011-0664-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-011-0664-x