Abstract
Image-guided surgery needs an effective and efficient registration between 2D video images of the surgical scene and a preoperative model of a patient from 3D MRI or CT scans. Such an alignment process is difficult due to the lack of robustly trackable features on the operative surface as well as tissue deformation and specularity. In this paper, we propose a novel approach to perform the registration using PTAM camera tracking and colour-consistency. PTAM provides a set of video images with the corresponding camera positions. Registration of the 3D model to the video images can then be achieved by maximization of colour-consistency between all 2D pixels corresponding to a given 3D surface point. An improved algorithm for calculation of visible surface points is provided. It is hoped that PTAM camera tracking using a reduced set of points can be combined with colour-consistency to provide a robust registration. A ground truth simulation test bed has been developed for validating the proposed algorithm and empirical studies have shown that the approach is feasible, with ground truth simulation data providing a capture range of ±9mm/° with a TRE less than 2mm. Our intended application is robot-assisted laparoscopic prostatectomy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bouguet, J.Y.: Complete camera calibration toolbox for Matlab, http://www.vision.caltech.edu/bouguetj/calib_doc/
Clarkson, M.J., Rueckert, D., Hill, D.L.G., Hawkes, D.J.: Using photo-consistency to register 2D optical images of the human face to a 3D surface model. IEEE Trans. Pattern Analysis and Machine Intelligence 23(11), 1266–1280 (2001)
Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press (2004)
Jannin, P., Fitzpatrick, J.M., Hawkes, D.J., Pennec, X., Shahidi, R., Vannier, M.W.: Validation of medical image processing in image-guided therapy. IEEE Transactions on Medical Imaging 21(12), 1445–1449 (2002)
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), pp. 225–234 (2007)
Lavallee, S., Sautot, P., Troccaz, J., Cinquin, P., Merloz, P.: Computer-assisted spine surgery: A technique for accurate transpedicular screw fixation using CT data and a 3-D optical localizer. Journal of Image Guided Surgery 1(1), 65–73 (1995)
Maes, F., Vandermeulen, D., Suetens, P.: Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information. Medical Image Analysis 3(4), 373–386 (1999)
Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis (2010)
Mountney, P., Lo, B., Thiemjarus, S., Stoyanov, D., Yang, G.-Z.: A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 34–41. Springer, Heidelberg (2007)
Mountney, P., Stoyanov, D., Davison, A.J., Yang, G.-Z.: Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006, Part I. LNCS, vol. 4190, pp. 347–354. Springer, Heidelberg (2006)
Mourgues, F., Devernay, F., Coste-Maniere, E.: 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: IEEE and ACM International Symposium on Augmented Reality (ISAR), p. 191 (2001)
Newcombe, R.A., Davison, A.J.: Live dense reconstruction with a single moving camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)
Powell, M.J.D.: The bobyqa algorithm for bound constrained optimization without derivatives. Technical Report: Department of Applied Mathematics and Theoretical Physics, Cambridge, England (2009)
Quartucci Forster, C.H., Tozzi, C.L.: Towards 3D reconstruction of endoscope images using shape from shading. In: Proceedings of Brazilian Symposium on Computer Graphics and Image Processing, p. 90 (2000)
Salzmann, M., Fua, P.: Deformable surface 3D reconstruction from monocular images. Synthesis Lectures on Computer Vision 2(1), 1–113 (2010)
Spoerk, J., Gendrin, C., Weber, C., Figl, M., Pawiro, S., Furtado, H., Fabri, D., Bloch, C., Bergmann, H., Gröller, E., Birkfellner, W.: High-performance gpu-based rendering for real-time, rigid 2d/3d-image registration and motion prediction in radiation oncology. Journal of Medical Physics p. epub ahead of print (2011)
Stoyanov, D., Darzi, A.W., Yang, G.-.Z.: Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 41–48. Springer, Heidelberg (2004)
Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)
Wang, H., Mirota, D., Ishii, M., Hager, G.: Robust motion estimation and structure recovery from endoscopic image sequences with an adaptive scale kernel consensus estimator. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2008)
Wu, C.H., Sun, Y.N., Chen, Y.C., Chang, C.C.: Endoscopic feature tracking and scale-invariant estimation of soft-tissue structures. IEICE Transactions on Information and Systems E91-D, 351–360 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chang, PL., Chen, D., Cohen, D., Edwards, P.“. (2012). 2D/3D Registration of a Preoperative Model with Endoscopic Video Using Colour-Consistency. In: Linte, C.A., Moore, J.T., Chen, E.C.S., Holmes, D.R. (eds) Augmented Environments for Computer-Assisted Interventions. AE-CAI 2011. Lecture Notes in Computer Science, vol 7264. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32630-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-32630-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32629-5
Online ISBN: 978-3-642-32630-1
eBook Packages: Computer ScienceComputer Science (R0)