Abstract
In this paper, we propose a smart deforming and/or cutting transcription algorithm for rheology objects such as human livers. Moreover, evaluation of performance and shape precision under the proposed algorithm are experimentally verified by deforming a real clay liver and/or cutting a gel block prepared at human body temperature. First, we capture the image of the liver of a patient by digital imaging and communication in medicine (DICOM) generated by magnetic resonance imaging (MRI) and/or computed tomography (CT) scanner. Then, the DICOM data is segmented and converted into four types of stereo-lithography (STL) polyhedra, which correspond to the whole liver and three blood vessels. Second, we easily overlap the virtual and real liver images in our mixed reality (MR) surgical navigation system using our initial position/orientation/shape adjustment system that uses color images to differentiate between real and virtual depth images. After overlapping, as long as the real liver is deformed and/or cut by a human (doctor), the liver is constantly captured by Kinect v2. Subsequently, by using the real depth image captured in real time, many vertices around the virtual polyhedral liver in STL format are pushed/pulled by viscoelastic elements called the Kelvin–Voigt materials located on the vertices. Finally, after determining the displacements of the vertices, we obtain an adequately shaped STL. The vertex position required for fixing the shape is calculated using the Runge–Kutta method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Peterhans, M., vom Berg, A., Dagon, B., Inderbitzin, D., Baur, C., Candinas, D., Weber, S.: A navigation system for open liver surgery: design, workflow and first clinical applications. Int J. Med. Robot. 7(1), 7–16 (Mar 2011). doi:10.1002/rcs.360. Epub 29 October 2010
Nicolas, C.B., Francesco, V., François, P., Christian, T., Matteo, F., Kate, G., Pietro, E.M., Matthias, P., Stefan, W., Philippe, M.: Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. J. Surg. Res. 184(2), 825–831 (2013)
Satou, S., Aoki, T., Kaneko, J., Sakamoto, Y., Hasegawa, K., Sugawara, Y., Arai, O., Mitake, T., Miura, K., Kokudo, N.: Initial experience of intraoperative three-dimensional navigation for liver resection using real-time virtual sonography. Surgery 155(2), 255–262 (2014)
Pessaux, P., Diana, M., Soler, L., Piardi, T., Mutter, D., Marescaux, J.: Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch. Surg. 400(3), 381–385 (2015)
Morita, Y., Takanishi, K., Matsumoto, J.: A new simple navigation for anatomic liver resection under intraoperative real-time ultrasound guidance. Hepatogastroenterology 61(34), 1734–1738 (2014)
Mahmud, N., Cohen, J., Tsourides, K., Berzin, T.M.: Computer vision and augmented reality in gastrointestinal endoscopy. Gastroenterol Rep. (Oxf) 3(3), 179–184 (2015). doi:10.1093/gastro/gov027. Epub 1 July 2015
Chen, X.-P., Zhang, W.-D., Wang, D., Cui, W., Yu, Y.-L.: Image classification of liver cancer surrounding right hepatic pedicle and its guide to precise liver resection. Int. J. Clin. Exp. Med. 8(7), 11093–11100 (2015)
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Zhang, Z.: Iterative point matching for registration of free-form surfaces. Int. J. Comput. Vis. 13(2), 119–152 (1994)
Granger, S., Pennec, X.: Multi-scale EM-ICP: a fast and robust approach for surface registration. In: 7th European Conference on Computer Vision, vol. 4, pp. 69–73 (2002)
Liu, Y.: Automatic registration of overlapping 3D point clouds using closest points: J. Image Vis. Comput. 24(7), 762–778 (2006)
Salvi, J., Matabosch, C., Fofi, D., Forest, J.: A review of recent range image registration methods with accuracy evaluation. J. Image Vis. Comput. 25, 578–596 (2007)
Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation, pp. 1–4 (2011)
Wu, Y.F., Wang, W., Lu, K.Q., Wei, Y.D., Chen, Z.C.: A new method for registration of 3D point sets with low overlapping ratios. In: 13th CIRP Conference on Computer Aided Tolerancing, pp. 202–206 (2015)
Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Onishi, K., Koeda, M., Nankaku, S., Matsui, K., Kon, M., Kaibori, M.: Image-based initial position/orientation adjustment system between real and virtual livers. Jurnal Teknologi Med. Eng. 77(6), 41–45 (2015). doi:10.11113/jt.v77.6225. Penerbit UTM Press
Noborio, H., Onishi, K., Koeda, M., Mizushino, K., Yagi, M., Kaibori, M., Kon, M.: Motion transcription algorithm by matching corresponding depth image and Z-buffer. In: 10th Anniversary Asian Conference on Computer Aided Surgery, pp. 60–61. Kyusyu University, Japan (2014)
Watanabe, K., Yagi, M., Ota, K., Onishi, K., Koeda, M., Nankaku, S., Noborio, H., Kon, M., Matsui, K., Kaibori, M.: Parameter identification of depth-depth-matching algorithm for liver following. Jurnal Teknologi Med. Eng. 77(6), 35–39 (2015). doi:10.11113/jt.v77.6224. Penerbit UTM Press
Watanabe, K., Yagi, M., Shintani, A., Nankaku, S., Onishi, K., Koeda, M., Noborio, H., Kon, M., Matsui, K., Kaibori, M.: A new 2D depth-depth matching algorithm whose translation and rotation freedoms are separated. In: International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS2015), Track 3: Bioinformatics, Medical Imaging and Neuroscience, pp. 271–278. Okinawa Institute of Science and Technology Graduate University (OIST), Okinawa, 28–30 November 2015
Noborio, H., Watanabe, K., Yagi, M., Ida, Y., Nankaku, S., Onishi, K., Koeda, H., Kon, M., Matsui, K., Kaibori, M.: Experimental results of 2D depth-depth matching algorithm based on depth camera Kinect v1. J. Bioinform. Neurosci 1(1), 38–44 (2015). ISSN:2188–8116
Acknowledgment
This study is supported in parts by 2014 Grants-in-aid for Scientific Research (No. 26289069) from the Ministry of Education, Culture, Sports, Science and Technology, Japan. It is also supported in parts by 2014 Cooperation research fund of graduate school in Osaka Electro-Communication University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Noborio, H. et al. (2016). Depth Image Matching Algorithm for Deforming and Cutting a Virtual Liver via Its Real Liver Image Captured Using Kinect v2. In: Ortuño, F., Rojas, I. (eds) Bioinformatics and Biomedical Engineering. IWBBIO 2016. Lecture Notes in Computer Science(), vol 9656. Springer, Cham. https://doi.org/10.1007/978-3-319-31744-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-31744-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31743-4
Online ISBN: 978-3-319-31744-1
eBook Packages: Computer ScienceComputer Science (R0)