Abstract
Image analysis challenges have considerably influenced the recent years in natural and biomedical computer vision. With several important architectures and training strategies having emerged from image analysis challenges, they are often interpreted as contests in model design and training, and much effort is put into optimization of these aspects.
This paper is to widen the focus beyond model architecture and training pipeline design by shedding a light on inference efficiency and the underestimated role of patch sampling strategies. A notable influence of the patch overlap on the challenge scores for successful MICCAI challenges of the previous year is found, in contrast to this parameter being systematically reported in rarely any challenge paper. These edge-overlap effects are shown to be etiologically related to varying dataset-specific intra-patch accuracies. Finally, novel strategies for inference-time patch sampling – other than strided cropping and including Monte Carlo - and uncertainty-based strategies – are proposed and examined, where special focus is put on effects that overarch the single-dataset level and, amongst other effects, an improved performance in the low patch number regimen is achieved.
Drawing on these findings, practical guidance is provided to the reader, and potential challenge participant, on how inference strategies can be optimized experimentally. Moreover, implications on the on-going best practice debate with respect to challenge design and reporting are discussed. In the hope it may stipulate interest in the undervalued topic of optimized sampling strategies, our inference framework and the source codes for the patch sampling strategies are made publicly available (https://github.com/IPMI-ICNS-UKE/inference-patch-sampling).
F. Madesta and R. Schmitz—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
StructSeg 2019: Automatic Structure Segmentation for Radiotherapy Planning Challenge 2019, August 2019. https://structseg2019.grand-challenge.org/Home/
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems, March 2016. arXiv:1603.04467
Abraham, N., Khan, N.M.: A novel focal Tversky loss function with improved attention U-net for lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 683–687. IEEE, Venice, April 2019
Bian, C., et al.: Pyramid network with online hard example mining for accurate left atrium segmentation. In: Pop, M., et al. (eds.) STACOM 2018. LNCS, vol. 11395, pp. 237–245. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-12029-0_26
Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 2843–2851. Curran Associates, Inc. (2012)
Han, X.: Automatic liver lesion segmentation using a deep convolutional neural network method. Med. Phys. 44(4), 1408–1419 (2017)
Isensee, F., Jaeger, P., Full, P.M., Wolf, I., Engelhardt, S., Maier-Hein, K.H.: Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. arXiv:1707.00587 (2018)
Isensee, F., Maier-Hein, K.H.: An attempt at beating the 3D U-Net. arXiv:1908.02182, October 2019
Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)
Larkin, K.G.: Reflections on shannon information: In search of a natural information-entropy for images. arXiv:1609.01117 (2016)
Maier-Hein, L., et al.: Why rankings of biomedical image analysis competitions should be interpreted with care. Nat. Commun. 9(1), 1–13 (2018)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Paszke, A., et al.: Automatic differentiation in PyTorch. NIPS 2017. https://openreview.net/pdf?id=BJJsrmfCZ
Reinke, A., et al.: How to exploit weaknesses in biomedical challenge design and organization. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 388–395. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_45
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schmitz, R., Madesta, F., Nielsen, M., Werner, R., Rösch, T.: Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture. arXiv:1909.10726, September 2019
Acknowledgments
This work was supported by DFG grant WE 6197/2-1, the European Fund for Regional Development (ERDF), the Free and Hanseatic City of Hamburg, the Forschungszentrum Medizintechnik Hamburg (02fmthh2017), and Olympus Co. Hamburg. Furthermore, RS gratefully acknowledges funding by the Studienstiftung des deutschen Volkes and the Günther Elin Krempel foundation. The authors would like to thank NVIDIA for the donation of graphics cards under the GPU Grant Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Madesta, F., Schmitz, R., Rösch, T., Werner, R. (2020). Widening the Focus: Biomedical Image Segmentation Challenges and the Underestimated Role of Patch Sampling and Inference Strategies. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12264. Springer, Cham. https://doi.org/10.1007/978-3-030-59719-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-59719-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59718-4
Online ISBN: 978-3-030-59719-1
eBook Packages: Computer ScienceComputer Science (R0)