Abstract
Purpose
Structured light scanning is a promising inexpensive and accurate intraoperative imaging modality. Integration of these scanners in surgical workflows has the potential to enable rapid registration and augment preoperative imaging, in a practical and timely manner in the operating theatre. Previously, we have demonstrated the intraoperative feasibility of such scanners to capture anatomical surface information with high accuracy. The purpose of this study was to investigate the feasibility of automatically characterizing anatomical tissues from textural and spatial information captured by such scanners using machine learning. Assisted or automatic identification of relevant components of a captured scan is essential for effective integration of the technology in surgical workflow.
Methods
During a clinical study, 3D surface scans for seven total knee arthroplasty patients were collected, and textural and spatial features for cartilage, bone, and ligament tissue were collected and annotated. These features were used to train and evaluate machine learning models. As part of our preliminary preparation, three fresh-frozen knee cadaver specimens were also used where 3D surface scans with texture information were collected during different dissection stages. The resulting models were manually segmented to isolate texture information for muscles, tendon, cartilage, and bone. This information, and detailed labels from dissections, provided an in-depth, finely annotated dataset for building machine learning classifiers.
Results
For characterizing bone, cartilage, and ligament in the intraoperative surface models, random forest and neural network-based models achieved an accuracy of close to 80%, whereas an accuracy of close to 90% was obtained when only characterizing bone and cartilage. Average accuracy of 76–82% was reached for cadaver data in two-, three-, and four-class tissue separation.
Conclusions
The results of this project demonstrate the feasibility of machine learning methods to accurately classify multiple types of anatomical tissue. The ability to automatically characterize tissues in intraoperatively collected surface models would streamline the surgical workflow of using structured light scanners—paving the way to applications such as 3D documentation of surgery in addition to rapid registration and augmentation of preoperative imaging.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hebal F, Port E, Hunter CJ, Malas B, Green J, Reynolds M (2019) A novel technique to measure severity of pediatric pectus excavatum using white light scanning. J Pediatr Surg 54(4):656–662. https://doi.org/10.1016/j.jpedsurg.2018.04.017
Szafer D, Taylor JS, Pei A, de Ruijter V, Hosseini H, Chao S, Wall J (2019) A simplified method for three-dimensional optical imaging and measurement of patients with chest wall deformities. J Laparoendosc Adv Surg Tech A 29(2):267–271. https://doi.org/10.1089/lap.2018.0191
Lain A, Garcia L, Gine C, Tiffet O, Lopez M (2017) New methods for imaging evaluation of chest wall deformities. Front Pediatr 5:257. https://doi.org/10.3389/fped.2017.00257
Michonski J, Walesiak K, Pakula A, Glinkowski W, Sitnik R (2016) Monitoring of spine curvatures and posture during pregnancy using surface topography—case study and suggestion of method. Scoliosis Spinal Disord 11(Suppl 2):31. https://doi.org/10.1186/s13013-016-0099-2
Seoud L, Ramsay J, Parent S, Cheriet F (2017) A novel fully automatic measurement of apparent breast volume from trunk surface mesh. Med Eng Phys 41:46–54. https://doi.org/10.1016/j.medengphy.2017.01.004
Sharma A, Sasaki D, Rickey DW, Leylek A, Harris C, Johnson K, Alpuche Aviles JE, McCurdy B, Egtberts A, Koul R, Dubey A (2018) Low-cost optical scanner and 3-dimensional printing technology to create lead shielding for radiation therapy of facial skin cancer: first clinical case series. Adv Radiat Oncol 3(3):288–296. https://doi.org/10.1016/j.adro.2018.02.003
Unkovskiy A, Spintzyk S, Axmann D, Engel EM, Weber H, Huettig F (2019) Additive manufacturing: a comparative analysis of dimensional accuracy and skin texture reproduction of auricular prostheses replicas. J Prosthodont 28(2):e460–e468. https://doi.org/10.1111/jopr.12681
Shamata A, Thompson T (2018) Documentation and analysis of traumatic injuries in clinical forensic medicine involving structured light three-dimensional surface scanning versus photography. J Forensic Leg Med 58:93–100. https://doi.org/10.1016/j.jflm.2018.05.004
Shamata A, Thompson T (2018) Using structured light three-dimensional surface scanning on living individuals: key considerations and best practice for forensic medicine. J Forensic Leg Med 55:58–64. https://doi.org/10.1016/j.jflm.2018.02.017
Buck U, Busse K, Campana L, Schyma C (2018) Validation and evaluation of measuring methods for the 3D documentation of external injuries in the field of forensic medicine. Int J Legal Med 132(2):551–561. https://doi.org/10.1007/s00414-017-1756-6
Venne G, Rasquinha BJ, Kunz M, Ellis RE (2017) Rectus capitis posterior minor: histological and biomechanical links to the spinal dura mater. Spine (Phila Pa 1976) 42(8):E466–E473. https://doi.org/10.1097/brs.0000000000001867
Hayes A, Easton K, Devanaboyina PT, Wu JP, Kirk TB, Lloyd D (2016) Structured white light scanning of rabbit Achilles tendon. J Biomech 49(15):3753–3758. https://doi.org/10.1016/j.jbiomech.2016.09.042
Chan B, Auyeung J, Rudan JF, Ellis RE, Kunz M (2016) Intraoperative application of hand-held structured light scanning: a feasibility study. Int J Comput Assist Radiol Surg 11(6):1101–1108. https://doi.org/10.1007/s11548-016-1381-8
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825–2830
Acknowledgements
The authors would like to thank the surgical and perioperative teams of Kingston Health Science Centre, and the Department of Biomedical and Molecular Sciences faculty at Queen’s University for their support throughout the study. The authors are grateful to the donor-cadaver-patients that made this research possible. We would also like to thank Alireza Sedghi and Shekoofeh Azizi for their helpful suggestions regarding data preprocessing, and Jason Auyeung for his help scanning and post-processing the specimens.
Funding
This work was supported in part by the Natural Sciences and Engineering Council of Canada, and thanks to funding from the Britton Smith Chair in Surgery.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
B Chan, JF. Rudan, P. Mousavi, and M. Kunz confirm that there are no known conflicts of interest associated with this publication.
Ethical standards
The study described in this manuscript was approved by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chan, B., Rudan, J.F., Mousavi, P. et al. Intraoperative integration of structured light scanning for automatic tissue classification: a feasibility study. Int J CARS 15, 641–649 (2020). https://doi.org/10.1007/s11548-020-02129-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-020-02129-8