[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Robot-assisted needle insertion guided by 2D ultrasound (US) can effectively improve the accuracy and success rate of clinical puncture. To this end, automatic and accurate needle-tracking methods are important for monitoring puncture processes, avoiding the needle deviating from the intended path, and reducing the risk of injury to surrounding tissues. This work aims to develop a framework for automatic and accurate detection of an inserted needle in 2D US image during the insertion process.

Methods

We propose a novel convolutional neural network architecture comprising of a two-channel encoder and single-channel decoder for needle segmentation using needle motion information extracted from two adjacent US image frames. Based on the novel network, we further propose an automatic needle detection framework. According to the prediction result of the previous frame, a region of interest of the needle in the US image was extracted and fed into the proposed network to achieve finer and faster continuous needle localization.

Results

The performance of our method was evaluated based on 1000 pairs of US images extracted from robot-assisted needle insertions on freshly excised bovine and porcine tissues. The needle segmentation network achieved 99.7% accuracy, 86.2% precision, 89.1% recall, and an F1-score of 0.87. The needle detection framework successfully localized the needle with a mean tip error of 0.45 ± 0.33 mm and a mean orientation error of 0.42° ± 0.34° and achieved a total processing time of 50 ms per image.

Conclusion

The proposed framework demonstrated the capability to realize robust, accurate, and real-time needle localization during robot-assisted needle insertion processes. It has a promising application in tracking the needle and ensuring the safety of robotic-assisted automatic puncture during challenging US-guided minimally invasive procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Soni NJ, Franco-Sadud R, Schnobrich D, Dancel R, Tierney DM, Salame G, Restrepo MI, McHardy P (2016) Ultrasound guidance for lumbar puncture. Neurol Clin Pract 6(4):358–368

    Article  Google Scholar 

  2. Chen SH, Wang F, Lin YP, Shi QS, Wang YL (2021) Ultrasound-guided needle insertion robotic system for percutaneous puncture. Int J Comput Assist Radiol Surg 16(3):475–484

    Article  Google Scholar 

  3. Morse J, Terrasini N, Wehbe M, Philippona C, Zaouter C, Cyr S, Hemmerling TM (2014) Comparison of success rates, learning curves, and inter-subject performance variability of robot-assisted and manual ultrasound-guided nerve block needle guidance in simulation. Br J Anaesth 112(6):1092–1097

    Article  CAS  Google Scholar 

  4. Zhao Y, Shen Y, Bernard A, Cachard C, Liebgott H (2017) Evaluation and comparison of current biopsy needle localization and tracking methods using 3D ultrasound. Ultrasonics 73:206–220

    Article  CAS  Google Scholar 

  5. Zhao Y, Liebgott H, Cachard C (2015) Comparison of the existing tool localisation methods on two-dimensional ultrasound images and their tracking results. IET Contr Theory Appl 9(7):1180–1180

    Article  Google Scholar 

  6. Zhao Z, Tse ZTH (2019) An electromagnetic tracking needle clip: an enabling design for low-cost image-guided therapy. Minim Invasive Ther Allied Technol 28(3):165–171

    Article  Google Scholar 

  7. Xia WF, West SJ, Finlay MC, Mari JM, Ourselin S, David AL, Desjardins AE (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7:3674. https://doi.org/10.1038/s41598-017-03886-4

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Ayvali E, Desai JP (2015) Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Ann Biomed Eng 43(8):1828–1840

    Article  Google Scholar 

  9. Barva M, Uhercik M, Mari JM, Kybic J, Duhamel JR, Liebgott H, Hlavac V, Cachard C (2008) Parallel integral projection transform for straight electrode localization in 3-D ultrasound images. IEEE Trans Ultrason Ferroelectr Freq Control 55(7):1559–1569

    Article  CAS  Google Scholar 

  10. Mathiassen K, Dall’Alba D, Muradore R, Fiorini P, Elle OJ (2017) Robust real-time needle tracking in 2-D ultrasound images using statistical filtering. IEEE Trans Control Syst Technol 25(3):966–978

    Article  Google Scholar 

  11. Kaya M, Bebek O (2014) Needle localization using Gabor filtering in 2D ultrasound images. In: IEEE international conference on robotics and automation (ICRA), pp 4881–4886

  12. Zhao Y, Cachard C, Liebgott H (2013) Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrason Imaging 35(4):283–306

    Article  Google Scholar 

  13. Beigi P, Rohling R, Salcudean SE, Ng GC (2016) Spectral analysis of the tremor motion for needle detection in curvilinear ultrasound via spatiotemporal linear sampling. Int J Comput Assist Radiol Surg 11(6):1183–1192

    Article  Google Scholar 

  14. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sanchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  15. Chen AI, Balter ML, Maguire TJ, Yarmush ML (2020) Deep learning robotic guidance for autonomous vascular access. Nat Mach Intell 2(2):104–115

    Article  Google Scholar 

  16. Park I, Kim HK, Chung WK, Kim K (2020) Deep learning based real-time OCT image segmentation and correction for robotic needle insertion systems. IEEE Robot Autom Let 5(3):4517–4524

    Article  Google Scholar 

  17. Maris B, Tenga C, Vicario R, Palladino L, Murr N, De Piccoli M, Calanca A, Puliatti S, Micali S, Tafuri A, Fiorini P (2021) Toward autonomous robotic prostate biopsy: a pilot study. Int J Comput Assist Radiol Surg 16(8):1393–1401

    Article  Google Scholar 

  18. Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolution neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657

    Article  Google Scholar 

  19. Lee JY, Islam M, Woh JR, Washeem TSM, Ngoh LYC, Wong WK, Ren H (2020) Ultrasound needle segmentation and trajectory prediction using excitation network. Int J Comput Assist Radiol Surg 15(3):437–443

    Article  Google Scholar 

  20. Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng GC, Korsten HHM, de With PHN (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333

    Article  Google Scholar 

  21. Arif M, Moelker A, van Walsum T (2019) Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver. Med Image Anal 53:104–110

    Article  Google Scholar 

  22. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI), pp 234–241

  23. Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of 2016 fourth international conference on 3d vision (3DV), pp 565–571

  24. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference in learning representations (ICLR)

Download references

Acknowledgements

We would like to thank Editage (www.editage.cn) for English language editing.

Funding

This study was funded by the National Natural Science Foundation of China (Grant number 52175020).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People’s Republic of China

    Shihang Chen, Yanping Lin & Qixin Cao

  2. School of Medicine, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai, People’s Republic of China

    Zhaojun Li

  3. Shanghai East Hospital, Tongji University School of Medicine, Shanghai, People’s Republic of China

    Fang Wang

  4. Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People’s Republic of China

    Qixin Cao

Authors
  1. Shihang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanping Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaojun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qixin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanping Lin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animals rights

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. No animal experiments were performed in this study.

Informed consent

Informed consent was obtained from all participants.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Lin, Y., Li, Z. et al. Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process. Int J CARS 17, 295–303 (2022). https://doi.org/10.1007/s11548-021-02519-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02519-6

Keywords

Search

Navigation