Abstract
Motion estimation and segmentation are both critical steps in identifying and assessing myocardial dysfunction, but are traditionally treated as unique tasks and solved as separate steps. However, many motion estimation techniques rely on accurate segmentations. It has been demonstrated in the computer vision and medical image analysis literature that both these tasks may be mutually beneficial when solved simultaneously. In this work, we propose a multi-task learning network that can concurrently predict volumetric segmentations of the left ventricle and estimate motion between 3D echocardiographic image pairs. The model exploits complementary latent features between the two tasks using a shared feature encoder with task-specific decoding branches. Anatomically inspired constraints are incorporated to enforce realistic motion patterns. We evaluate our proposed model on an in vivo 3D echocardiographic canine dataset. Results suggest that coupling these two tasks in a learning framework performs favorably when compared against single task learning and other alternative methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cheng, J., et al.: SegFlow: joint learning for video object segmentation and optical flow. In: IEEE International Conference on Computer Vision (2017)
Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 472–480. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_53
Ta, K., Ahn, S.S., Stendahl, J.C., Sinusas, A.J., Duncan, J.S.: A semi-supervised joint network for simultaneous left ventricular motion tracking and segmentation in 4D echocardiography. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 468–477. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_45
Chen, C., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7, 25 (2020)
Huang, X., et al.: Contour tracking in echocardiographic sequences via sparse representation and dictionary learning. Med. Image Anal. 18, 253–271 (2014)
Dong, S., et al.: A combined fully convolutional networks and deformable model for automatic left ventricle segmentation based on 3D echocardiography. BioMed Res. Int. 2018, 1–16 (2018)
Papademetris, X., et al.: Estimation of 3-D left ventricular deformation from medical images using biomechanical models. IEEE Trans. Med. Imaging 21, 786–800 (2002)
Parajuli, N., et al.: Flow network tracking for spatiotemporal and periodic point matching: applied to cardiac motion analysis. Med. Image Anal. 55, 116–135 (2019)
Qiu, H., Qin, C., Le Folgoc, L., Hou, B., Schlemper, J., Rueckert, D.: Deep learning for cardiac motion estimation: supervised vs. unsupervised training. In: Pop, M., et al. (eds.) STACOM 2019. LNCS, vol. 12009, pp. 186–194. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39074-7_20
Balakrishnan, G., et al.: An unsupervised learning model for deformable medical image registration. In: The IEEE Conference on Computer Vision and Pattern Recognition (2018)
Zhu, W., et al.: NeurReg: neural registration and its application to image segmentation. In: Winter Conference on Applications of Computer Vision (2020)
Ahn, S.S., et al.: Unsupervised motion tracking of left ventricle in echocardiography. In: Medical Imaging 2020: Ultrasonic Imaging and Tomography, International Society for Optics and Photonics (2020)
Ta, K., et al.: A semi-supervised joint learning approach to left ventricular segmentation and motion tracking in echocardiography. In: IEEE International Symposium on Biomedical Imaging (2020)
Tsai, Y.-H., et al.: Video segmentation via object flow. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Lu, A., et al.: Learning-based regularization for cardiac strain analysis with ability for domain adaptation. arXiv preprint arXiv:1807.04807 (2018)
Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv:1706.05098 (2017)
Stendahl, J.C., et al.: Regional myocardial strain analysis via 2D speckle tracking echocardiography: validation with sonomicrometry and correlation with regional blood flow in the presence of graded coronary stenoses and dobutamine stress. Cardiovasc. Ultrasound. 18(1), 2 (2020). PMID: 31941514; PMCID: PMC6964036. https://doi.org/10.1186/s12947-019-0183-x
Besnerais, G.L., et al.: Dense optical flow by iterative local window registration. In: IEEE International Conference on Image Processing (2005)
He, K., et al.: Deep residual learning for image recognition. arXiv:1512.03385 (2015)
Yushkevich, P.A., et al.: User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31, 1116–1128 (2006)
Wu, Y., He, K.: Group normalization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_1
Gibbons, J.D., et al.: Nonparametric Statistical Inference, 5th edn. Chapman & Hall/CRC Press, Taylor & Francis Group, Boca Raton (2011)
Yu, H., et al.: FOAL: fast online adaptive learning for cardiac motion estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020
Acknowledgements
This work was funded by the following grants: R01 HL121226, R01 HL137365, and HL 098069. Additionally, we are grateful toward Drs. Nripesh Parajuli and Allen Lu and the Fellows and staff of the Yale Translational Research Imaging Center, especially Drs. Nabil Boutagy and Imran Al Khalil, for their technical support and assistance with the in vivo canine imaging studies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Ta, K., Ahn, S.S., Stendahl, J.C., Langdon, J., Sinusas, A.J., Duncan, J.S. (2022). Simultaneous Segmentation and Motion Estimation of Left Ventricular Myocardium in 3D Echocardiography Using Multi-task Learning. In: Puyol Antón, E., et al. Statistical Atlases and Computational Models of the Heart. Multi-Disease, Multi-View, and Multi-Center Right Ventricular Segmentation in Cardiac MRI Challenge. STACOM 2021. Lecture Notes in Computer Science(), vol 13131. Springer, Cham. https://doi.org/10.1007/978-3-030-93722-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-93722-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93721-8
Online ISBN: 978-3-030-93722-5
eBook Packages: Computer ScienceComputer Science (R0)