Memory-Efficient High-Resolution OCT Volume Synthesis with Cascaded Amortized Latent Diffusion Models
Authors: 
Kun Huang, Xiao Ma, Yuhan Zhang, Na Su, Songtao Yuan, Yong Liu, Qiang Chen, Huazhu FuAuthors Info & Claims
Pages 478 - 487
Abstract
Optical coherence tomography (OCT) image analysis plays an important role in the field of ophthalmology. Current successful analysis models rely on available large datasets, which can be challenging to be obtained for certain tasks. The use of deep generative models to create realistic data emerges as a promising approach. However, due to limitations in hardware resources, it is still difficulty to synthesize high-resolution OCT volumes. In this paper, we introduce a cascaded amortized latent diffusion model (CA-LDM) that can synthesis high-resolution OCT volumes in a memory-efficient way. First, we propose non-holistic autoencoders to efficiently build a bidirectional mapping between high-resolution volume space and low-resolution latent space. In tandem with autoencoders, we propose cascaded diffusion processes to synthesize high-resolution OCT volumes with a global-to-local refinement process, amortizing the memory and computational demands. Experiments on a public high-resolution OCT dataset show that our synthetic data have realistic high-resolution and global features, surpassing the capabilities of existing methods. Moreover, performance gains on two down-stream fine-grained segmentation tasks demonstrate the benefit of the proposed method in training deep learning models for medical imaging tasks. The code is public available at: https://github.com/nicetomeetu21/CA-LDM.
References
[1]
Blattmann, A., et al.: Align your latents: high-resolution video synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22563–22575 (2023)
[2]
Chai, W., Guo, X., Wang, G., Lu, Y.: Stablevideo: text-driven consistency-aware diffusion video editing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 23040–23050 (2023)
[3]
Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, and Bharath AA Generative adversarial networks: an overview IEEE Signal Process. Mag. 2018 35 1 53-65
[4]
Deo, Y., Dou, H., Ravikumar, N., Frangi, A.F., Lassila, T.: Shape-guided conditional latent diffusion models for synthesising brain vasculature. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 164–173. Springer (2023).
[5]
Esser, P., Chiu, J., Atighehchian, P., Granskog, J., Germanidis, A.: Structure and content-guided video synthesis with diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7346–7356 (2023)
[6]
Gao C et al. Synthetic data accelerates the development of generalizable learning-based algorithms for x-ray image analysis Nature Mach. Intell. 2023 5 3 294-308
[7]
Han, K., et al.: Medgen3D: a deep generative framework for paired 3D image and mask generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 759–769. Springer (2023).
[8]
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017)
[9]
Ho J, Jain A, and Abbeel P Denoising diffusion probabilistic models Adv. Neural. Inf. Process. Syst. 2020 33 6840-6851
[10]
Hu, Q., Li, H., Zhang, J.: Domain-adaptive 3D medical image synthesis: an efficient unsupervised approach. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 495–504. Springer (2022).
[11]
Jiang Y, Chen H, Loew M, and Ko H Covid-19 CT image synthesis with a conditional generative adversarial network IEEE J. Biomed. Health Inform. 2020 25 2 441-452
[12]
Khader F et al. Denoising diffusion probabilistic models for 3D medical image generation Sci. Rep. 2023 13 1 7303
[13]
Kim, B., Ye, J.C.: Diffusion deformable model for 4D temporal medical image generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 539–548. Springer (2022).
[14]
Li, M., et al.: Octa-500: a retinal dataset for optical coherence tomography angiography study. Med. Image Anal. 103092 (2024)
[15]
Özbey, M., et al.: Unsupervised medical image translation with adversarial diffusion models. IEEE Trans. Med. Imaging (2023)
[16]
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition, pp. 10684–10695 (2022)
[17]
Rudin LI, Osher S, and Fatemi E Nonlinear total variation based noise removal algorithms Phys. D 1992 60 1–4 259-268
[18]
Saharia C et al. Photorealistic text-to-image diffusion models with deep language understanding Adv. Neural. Inf. Process. Syst. 2022 35 36479-36494
[19]
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)
[20]
Sun L, Chen J, Xu Y, Gong M, Yu K, and Batmanghelich K Hierarchical amortized Gan for 3d high resolution medical image synthesis IEEE J. Biomed. Health Inform. 2022 26 8 3966-3975
[21]
Xie Y et al. Fundus image-label pairs synthesis and retinopathy screening via Gans with class-imbalanced semi-supervised learning IEEE Trans. Med. Imaging 2023 42 9 2714-2725
[22]
Zhang Y, Li M, Yuan S, Liu Q, and Chen Q Robust region encoding and layer attribute protection for the segmentation of retina with multifarious abnormalities Med. Phys. 2021 48 12 7773-7789
[23]
Zhou Y, Wang B, He X, Cui S, and Shao L Dr-Gan: conditional generative adversarial network for fine-grained lesion synthesis on diabetic retinopathy images IEEE J. Biomed. Health Inform. 2020 26 1 56-66
Index Terms
- Memory-Efficient High-Resolution OCT Volume Synthesis with Cascaded Amortized Latent Diffusion Models
Index terms have been assigned to the content through auto-classification.
Recommendations
CoLa-Diff: Conditional Latent Diffusion Model for Multi-modal MRI Synthesis
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023AbstractMRI synthesis promises to mitigate the challenge of missing MRI modality in clinical practice. Diffusion model has emerged as an effective technique for image synthesis by modelling complex and variable data distributions. However, most diffusion-...
Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis
Advances in Knowledge Discovery and Data MiningAbstractWhile recent advances in large-scale foundational computer vision models show promising results, their application to the medical domain has not yet been explored in detail. In this paper, we progress into the realms of large-scale modeling in ...
CAVM: Conditional Autoregressive Vision Model for Contrast-Enhanced Brain Tumor MRI Synthesis
Medical Image Computing and Computer Assisted Intervention – MICCAI 2024AbstractContrast-enhanced magnetic resonance imaging (MRI) is pivotal in the pipeline of brain tumor segmentation and analysis. Gadolinium-based contrast agents, as the most commonly used contrast agents, are expensive and may have potential side effects, ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Oct 2024
835 pages
ISBN:978-3-031-72103-8
DOI:10.1007/978-3-031-72104-5
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 07 October 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Jan 2025