Abstract
Surgical process analysis and modeling is a recent and important topic aiming at introducing a new generation of computer-assisted surgical systems. Among all of the techniques already in use for extracting data from the Operating Room, the use of image videos allows automating the surgeons’ assistance without altering the surgical routine. We proposed in this paper an application-dependent framework able to automatically extract the phases of the surgery only by using microscope videos as input data and that can be adaptable to different surgical specialties. First, four distinct types of classifiers based on image processing were implemented to extract visual cues from video frames. Each of these classifiers was related to one kind of visual cue: visual cues recognizable through color were detected with a color histogram approach, for shape-oriented visual cues we trained a Haar classifier, for texture-oriented visual cues we used a bag-of-word approach with SIFT descriptors, and for all other visual cues we used a classical image classification approach including a feature extraction, selection, and a supervised classification. The extraction of this semantic vector for each video frame then permitted to classify time series using either Hidden Markov Model or Dynamic Time Warping algorithms. The framework was validated on cataract surgeries, obtaining accuracies of 95%.
Chapter PDF
Similar content being viewed by others
References
Speidel, S., Sudra, G., Senemaud, J., Drentschew, M., Müller-stich, B., Gun, C., Dillmann, R.: Situation modeling and situation recognition for a context-aware augmented reality system. In: Progression in Biomed. Optics and Imaging, vol. 9(1), p. 35 (2008)
Lo, B., Darzi, A., Yang, G.: Episode Classification for the Analysis of Tissue/Instrument Interaction with Multiple Visual Cues. In: Ellis, R.E., Peters, T.M. (eds.) MICCAI 2003. LNCS, vol. 2878, pp. 230–237. Springer, Heidelberg (2003)
Klank, U., Padoy, N., Feussner, H., Navab, N.: Automatic feature generation in endoscopic images. Int. J. Comput. Assist. Radiol. Surg. 3(3-4), 331–339 (2008)
Blum, T., Feußner, H., Navab, N.: Modeling and Segmentation of Surgical Workflow from Laparoscopic Video. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A., et al. (eds.) MICCAI 2010. LNCS, vol. 6363, pp. 400–407. Springer, Heidelberg (2010)
Bhatia, B., Oates, T., Xiao, Y., Hu, P.: Real-time identification of operating room state from video. In: AAAI, pp. 1761–1766 (2007)
Padoy, N., Blum, T., Feuner, H., Berger, M.O., Navab, N.: On-line recognition of surgical activity for monitoring in the operating room. In: Proc’s of the 20th Conference on Innovative Applications of Artificial Intelligence (2008)
Voros, S., Hager, G.: Towards “real-time” tool-tissue interaction detection in robotically assisted laparoscopy. Biomed. Robotics and Biomechatronics, 562–567 (2008)
Reiley, C., Hager, G.: Decomposition of robotic surgical tasks: an analysis of subtasks and their correlation to skill. In: M2CAI Workshop, MICCAI 2009 (2009)
Lalys, F., Riffaud, L., Morandi, X., Jannin, P.: Surgical phases detection from microscope videos by combining SVM and HMM. In: Menze, B., Langs, G., Tu, Z., Criminisi, A. (eds.) MICCAI 2010. LNCS, vol. 6533, pp. 54–62. Springer, Heidelberg (2011)
Viola, P. and Jones, M.: Rapid real-time face detection. IJCV, 137-154 (2004)
Lowe, D.G.: Object recognition from scale-invariant features. In: ICCV 1999, vol. 2, pp. 1150–1157 (1999)
Lalys, F., Riffaud, L., Morandi, X., Jannin, P.: Automatic phases recognition in pituitary surgeries by microscope images classification. In: Navab, N., Jannin, P. (eds.) IPCAI 2010. LNCS, vol. 6135, pp. 34–44. Springer, Heidelberg (2010)
Haralick, R., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. on Systems, Man, and Cybernetics 3(6), 61–621 (1973)
Hu, M.: Visual pattern recognition by moment invariants. Trans. Inf. Theory 8(2), 79–87 (1962)
Ahmed, N., Natarajan, T., Rao, R.: Discrete Cosine Transform. IEEE Trans. Comp., 90–93 (1974)
Guyon, I., Weston, J., Barhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machine. Machine Learning 46, 389–422 (2002)
Hamming, R.W.: Coding and Information Theory. Prentice-Hall Inc., Englewood Cliffs (1980)
Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken work recognition. In: IEEE Trans. Acoustic, Speech and Signal Proc., vol. 26(1), pp. 43–49 (1978)
Ahmadi, S.-A., Sielhorst, T., Stauder, R., Horn, M., Feussner, H., Navab, N.: Recovery of surgical workflow without explicit models. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 420–428. Springer, Heidelberg (2006)
Bouarfa, L., Jonker, P., Dankelman, J.: Discovery of high-level tasks in the operating room. J. Biomedical Informatics 44(3), 455–462 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lalys, F., Riffaud, L., Bouget, D., Jannin, P. (2011). An Application-Dependent Framework for the Recognition of High-Level Surgical Tasks in the OR. In: Fichtinger, G., Martel, A., Peters, T. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2011. MICCAI 2011. Lecture Notes in Computer Science, vol 6891. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23623-5_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-23623-5_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23622-8
Online ISBN: 978-3-642-23623-5
eBook Packages: Computer ScienceComputer Science (R0)