Abstract
Wireless Capsule Endoscopy (WCE) is highly valued for its non-invasive and painless approach, though its effectiveness is compromised by uneven illumination from hardware constraints and complex internal dynamics, leading to overexposed or underexposed images. While researchers have discussed the challenges of low-light enhancement in WCE, the issue of correcting for different exposure levels remains underexplored. To tackle this, we introduce EndoUIC, a WCE unified illumination correction solution using an end-to-end promptable diffusion transformer (DiT) model. In our work, the illumination prompt module shall navigate the model to adapt to different exposure levels and perform targeted image enhancement, in which the Adaptive Prompt Integration (API) and Global Prompt Scanner (GPS) modules shall further boost the concurrent representation learning between the prompt parameters and features. Besides, the U-shaped restoration DiT model shall capture the long-range dependencies and contextual information for unified illumination restoration. Moreover, we present a novel Capsule-endoscopy Exposure Correction (CEC) dataset, including ground-truth and corrupted image pairs annotated by expert photographers. Extensive experiments against a variety of state-of-the-art (SOTA) methods on four datasets showcase the effectiveness of our proposed method and components in WCE illumination restoration, and the additional downstream experiments further demonstrate its utility for clinical diagnosis and surgical assistance. The code and the proposed dataset are available at github.com/longbai1006/EndoUIC.
L. Bai, T. Chen, Q. Tan—Co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baek, J.H., Kim, D., Choi, S.M., Lee, H.j., Kim, H., Koh, Y.J.: Luminance-aware color transform for multiple exposure correction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6156–6165 (2023)
Bai, L., Chen, T., Wu, Y., Wang, A., Islam, M., Ren, H.: LLCaps: learning to illuminate low-light capsule endoscopy with curved wavelet attention and reverse diffusion. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. MICCAI 2023. LNCS, vol. 14229. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43999-5_4
Coelho, P., Pereira, A., Leite, A., Salgado, M., Cunha, A.: A deep learning approach for red lesions detection in video capsule endoscopies. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 553–561. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_63
García-Vega, A., et al.: A novel hybrid endoscopic dataset for evaluating machine learning-based photometric image enhancement models. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds.) Advances in Computational Intelligence. MICAI 2022. LNCS(), vol. 13612. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19493-1_22
García-Vega, A., et al.: Multi-scale structural-aware exposure correction for endoscopic imaging. In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2023)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)
Huang, J., et al.: Exposure normalization and compensation for multiple-exposure correction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6043–6052 (2022)
Huang, J., et al.: Deep fourier-based exposure correction network with spatial-frequency interaction. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision - ECCV 2022. ECCV 2022. LNCS, vol. 13679. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19800-7_10
Jiang, H., Luo, A., Fan, H., Han, S., Liu, S.: Low-light image enhancement with wavelet-based diffusion models. ACM Trans. Graph. 42(6), 1–14 (2023)
Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., Peng, X.: All-in-one image restoration for unknown corruption. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17452–17462 (2022)
Li, Y., Hou, Q., Zheng, Z., Cheng, M.M., Yang, J., Li, X.: Large selective kernel network for remote sensing object detection (2023). arXiv preprint arXiv:2303.09030
Li, Z., Lei, Y., Ma, C., Zhang, J., Shan, H.: Prompt-in-prompt learning for universal image restoration (2023). arXiv preprint arXiv:2312.05038
Liu, Y., et al.: VMamba: Visual state space model (2024). arXiv preprint arXiv:2401.10166
Long, M., Li, Z., Xie, X., Li, G., Wang, Z.: Adaptive image enhancement based on guide image and fraction-power transformation for wireless capsule endoscopy. IEEE Trans. Biomed. Circuits Syst. 12(5), 993–1003 (2018)
Ma, Y., et al.: Cycle structure and illumination constrained GAN for medical image enhancement. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 667–677. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_64
Moghtaderi, S., Yaghoobian, O., Wahid, K.A., Lukong, K.E.: Endoscopic image enhancement: wavelet transform and guided filter decomposition-based fusion approach. J. Imaging 10(1), 28 (2024)
Mou, E., et al.: Global and local enhancement of low-light endoscopic images (2023)
Potlapalli, V., Zamir, S.W., Khan, S., Khan, F.S.: PromptiR: Prompting for all-in-one blind image restoration. arXiv preprint arXiv:2306.13090 (2023)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rukundo, O., Pedersen, M., Hovde, Ø., et al.: Advanced image enhancement method for distant vessels and structures in capsule endoscopy. Comput. Math. Methods Med. 2017, 9813165 (2017)
Smedsrud, P.H., et al.: Kvasir-capsule, a video capsule endoscopy dataset. Sci. Data 8(1), 142 (2021)
Wang, G., Bai, L., Wu, Y., Chen, T., Ren, H.: Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy-based mini-batch pseudo-replay. Comput. Biol. Med. 165, 107412 (2023)
Wang, L., Yang, Q., Wang, C., Wang, W., Pan, J., Su, Z.: Learning a coarse-to-fine diffusion transformer for image restoration (2023). arXiv preprint arXiv:2308.08730
Wang, L., Wu, B., Wang, X., Zhu, Q., Xu, K.: Endoscopic image luminance enhancement based on the inverse square law for illuminance and retinex. Int. J. Med. Robot. Comput. Assist. Surg. 18(4), e2396 (2022)
Xue, M., He, J., He, Y., Liu, Z., Wang, W., Zhou, M.: Low-light image enhancement via clip-fourier guided wavelet diffusion (2024). arXiv preprint arXiv:2401.03788
Yang, K.F., Cheng, C., Zhao, S.X., Yan, H.M., Zhang, X.S., Li, Y.J.: Learning to adapt to light. Int. J. Comput. Vision 131(4), 1022–1041 (2023)
Yin, Y., Xu, D., Tan, C., Liu, P., Zhao, Y., Wei, Y.: CLE diffusion: controllable light enhancement diffusion model. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 8145–8156 (2023)
Yue, G., Gao, J., Cong, R., Zhou, T., Li, L., Wang, T.: Deep pyramid network for low-light endoscopic image enhancement. IEEE Trans. Circ. Syst. Video Technol. 34(5), 3834–3845 (2023)
Zamir, S.W., et al.: Learning enriched features for fast image restoration and enhancement. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 1934–1948 (2022)
Zhang, Y., Bai, L., Liu, L., Ren, H., Meng, M.Q.H.: Deep reinforcement learning-based control for stomach coverage scanning of wireless capsule endoscopy. In: 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 01–06. IEEE (2022)
Zhou, D., Yang, Z., Yang, Y.: Pyramid diffusion models for low-light image enhancement (2023). arXiv preprint arXiv:2305.10028
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (2017)
Acknowledgments
This work was supported by Hong Kong RGC GRF 14211420, CRF C4063-18G, NSFC/RGC Joint Research Scheme N_CUHK420/22; Shenzhen-HK-Macau Technology Research Programme (Type C) STIC Grant 202108233000303; Regional Joint Fund Project 2021B1515120035 (B.02.21.00101) of Guangdong Basic and Applied Research Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bai, L. et al. (2024). EndoUIC: Promptable Diffusion Transformer for Unified Illumination Correction in Capsule Endoscopy. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15007. Springer, Cham. https://doi.org/10.1007/978-3-031-72104-5_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-72104-5_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72103-8
Online ISBN: 978-3-031-72104-5
eBook Packages: Computer ScienceComputer Science (R0)