Abstract
Early diagnosis is of great significance for the treatment of GI diseases. Endoscopic tissue biopsy of the GI tract is the standard means to diagnose whether the tumor will become cancerous or to confirm the stage of lesions. Confocal endoscopy, which can provide cell-level resolution and realize non-invasive real-time optical biopsy, has attracted much attention in the field of clinical disease diagnosis. However, because of the small field of vision, it is difficult to locate the probe again, which affects the efficiency and learning cost of confocal endoscopy. Based on the honeycomb shape of the original image of the confocal endoscope, the scale of the image pixel corresponding to the real world is obtained through the parameters of the endoscopy probe and the fiber bundle that make up the probe. Yolov3 is used as a crypt recognition algorithm, which assists the optical flow algorithm to predict the pixel displacement of crypt. Finally, the distance and angle of the endoscope lens motion are obtained through the scale. The moving angle and distance of the endoscope can help locate the probe position of the endoscope lens and record the probe path of the endoscope lens. After the exploration is completed, the original exploration path can also be restored by distance and angle information, which helps to accurately locate the lesion again. The Yolov3 Recognition Network was trained with 200 confocal endoscope images of rat colon. The \(\rm{map}_{0.5}\) was 99.84%. Compared with many other optical flow methods, the DISflow optical flow algorithm is finally selected. After testing, the angle error of the algorithm is less than 4°, the distance error is less than 8%. This work can restore the exploration path of confocal endoscopy and improve the diagnostic efficiency of confocal endoscopy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
王家福, 杨敏, 杨莉, 张云, 袁菁, 刘谦, 侯晓华, 付玲. 用于细胞成像的共聚焦内窥镜. Engineering 1(03), 152–171 (2015)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24673-2_3
Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)
Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2758-2766 (2015). https://doi.org/10.1109/ICCV.2015.316
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: CVPR (2017)
Sun, D., Yang, X., Liu, M.Y., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: CVPR (2018)
Teed, Z., Deng, J.: RAFT: recurrent all-pairs field transforms for optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 402–419. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_24
Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: CVPR (2016)
Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_44
Jonschkowski, R., Stone, A., Barron, J.T., Gordon, A., Konolige, K., Angelova, A.: What matters in unsupervised optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 557–572. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_33
Meister, S., Hur, J., Roth, S.: UnFlow: unsupervised learning of optical flow with a bidirectional census loss. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018). https://doi.org/10.1609/aaai.v32i1.12276
Luo, K., Wang, C., Liu, S., Fan, H., Wang, J., Sun, J.: UPFlow: upsampling pyramid for unsupervised optical flow learning In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1045 1054 (2021). https://doi.org/10.1109/CVPR46437.2021.00110
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unifified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016)
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7263–7271 (2017)
Redmon, J., Farhadi, A.: YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700–4708 (2017)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580–587 (2014)
Kroeger, T., Timofte, R., Dai, D., Van Gool, L.: Fast optical flow using dense inverse search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 471–488. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_29
李华. 大视场共聚焦内窥成像方法研究. 华中科技大学 (2021)
王家福. 近红外共聚焦内窥成像方法研究. 华中科技大学 (2018)
Clevers, H.: The intestinal crypt, a prototype stem cell compartment. Cell 154(2), 274–284 (2013). https://doi.org/10.1016/j.cell.2013.07.004. PMID: 23870119
Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: large displacement optical flow with deep matching. In: 2013 IEEE International Conference on Computer Vision, pp. 1385–1392 (2013). https://doi.org/10.1109/ICCV.2013.175
Farnebäck, G.: Two-Frame motion estimation based on polynomial expansion. In: Bigun, J., Gustavsson, T., (eds) Image Analysis. SCIA 2003. Lecture Notes in Computer Science, vol 2749 (2003). Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45103-X_50
Wulff, J., Black, M.J.: Efficient sparse-to-dense optical flow estimation using a learned basis and layers. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 120-130 (2015). https://doi.org/10.1109/CVPR.2015.7298607
OpenCV: Optical Flow Algorithms
Acknowledgments
Funding. Hainan Province Key Science and Technology Project (ZDKJ202006). (Supported by Major Special Science and Technology Project of Hainan Province, ZDKJ202006).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, H., Lu, Y., Liu, Q. (2023). Ranging of Confocal Endoscopy Probe Using Recognition and Optical Flow Algorithm. In: Yu, C., Zhou, J., Song, X., Lu, Z. (eds) Green, Pervasive, and Cloud Computing. GPC 2022. Lecture Notes in Computer Science, vol 13744. Springer, Cham. https://doi.org/10.1007/978-3-031-26118-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-26118-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26117-6
Online ISBN: 978-3-031-26118-3
eBook Packages: Computer ScienceComputer Science (R0)