A data-centric framework for combating domain shift in underwater object detection with image enhancement: A data-centric framework for combating domain shift in underwater object detection...
Abstract
Underwater object detection has numerous applications in protecting, exploring, and exploiting aquatic environments. However, underwater environments pose a unique set of challenges for object detection including variable turbidity, colour casts, and light conditions. These phenomena represent a domain shift and need to be accounted for during design and evaluation of underwater object detection models. Although methods for underwater object detection have been extensively studied, most proposed approaches do not address challenges of domain shift inherent to aquatic environments. In this work we propose a data-centric framework for combating domain shift in underwater object detection with image enhancement. We show that there is a significant gap in accuracy of popular object detectors when tested for their ability to generalize to new aquatic domains. We used our framework to compare 14 image processing and enhancement methods in their efficacy to improve underwater domain generalization using three diverse real-world aquatic datasets and two widely used object detection algorithms. Using an independent test set, our approach superseded the mean average precision performance of existing model-centric approaches by 1.7–8.0 percentage points. In summary, the proposed framework demonstrated a significant contribution of image enhancement to underwater domain generalization.
References
[1]
Aguirre-Castro OA, García-Guerrero EE, López-Bonilla OR, et al. Evaluation of underwater image enhancement algorithms based on Retinex and its implementation on embedded systems Neurocomputing 2022 494 148-159
[2]
Akkaynak D, Treibitz T (2018) A Revised underwater image formation model. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. pp 6723–673
[3]
Atlas WI, Ma S, Chou YC et al (2023) Wild salmon enumeration and monitoring using deep learning empowered detection and tracking. Front Mar Sci 1.
[4]
Balaji Y, Sankaranarayanan S, Chellappa R (2018) MetaReg: towards domain generalization using meta-regularization. In: Advances in neural information processing systems
[5]
Bradski G (2000) The OpenCV library. Dr Dobb’s journal of software tools
[6]
Cai L, McGuire NE, Hanlon R, et al. Semi-supervised visual tracking of marine animals using autonomous underwater vehicles Int J Comput Vision 2023 131 6 1406-1427
[7]
Carlucci FM, D’Innocente A, Bucci S et al (2019) Domain generalization by Solving jigsaw puzzles. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 2224–2233.
[8]
Chaitin GJ On the length of programs for computing finite binary sequences J ACM 1966 13 4 547-56
[9]
Chen X, Lu Y, Wu Z et al (2021) Reveal of domain effect: how visual restoration contributes to object detection in aquatic scenes. In: Visual perception and control of underwater robots. CRC Press
[10]
Chen Y, Song P, Liu H, et al. Achieving domain generalization for underwater object detection by domain mixup and contrastive learning Neurocomputing 2023 528 20-3
[11]
Cheng N, Xie H, Zhu X, et al. Joint image enhancement learning for marine object detection in natural scene Eng Appl Artif Intell 2023 120 10590
[12]
Chiang JY and Chen YC Underwater image enhancement by wavelength compensation and dehazing IEEE Trans Image Process 2012 21 4 1756-176
[13]
Clark A, others (2023) Pillow (PIL Fork) documentation. https://pillow.readthedocs.io
[14]
Cong R, Yang W, Zhang W, et al. PUGAN: physical model-guided underwater image enhancement using GAN with dual-discriminators IEEE Trans Image Process 2023 32 4472-4485
[15]
Connolly RM, Jinks KI, Herrera C, et al. Fish surveys on the move: Adapting automated fish detection and classification frameworks for videos on a remotely operated vehicle in shallow marine waters Front Mar Sci 2022 9 91850
[16]
Connolly RM, Herrera C, Rasmussen J, et al. Estimating enhanced fish production on restored shellfish reefs using automated data collection from underwater videos J Appl Ecol 2024 61 4 633-646
[17]
Costello C, Cao L, Gelcich S, et al. The future of food from the sea Nature 2020 588 7836 95-10
[18]
Dai L, Liu H, Song P, et al. A gated cross-domain collaborative network for underwater object detection Pattern Recogn 2024 149 110222
[19]
Ditria EM, Lopez-Marcano S, Sievers M et al (2020) Automating the analysis of fish abundance using object detection: optimizing animal ecology with deep learning. Front Mar Sci 7.
[20]
Dou Q, Castro DC, Kamnitsas K et al (2019) Domain generalization via model-agnostic learning of semantic features. In: Proceedings of the 33rd international conference on neural information processing systems, vol. 579. p 6450–6461
[21]
Du D, Li E, Si L, et al. UIEDP: Boosting underwater image enhancement with diffusion prior Expert Syst Appl 2025 259 125271
[22]
Eger AM, Marzinelli EM, Beas-Luna R, et al. The value of ecosystem services in global marine kelp forests Nat Commun 2023 14 1 1894
[23]
Fu C, Liu R, Fan X, et al. Rethinking general underwater object detection: datasets, challenges, and solutions Neurocomputing 2023 517 243-25
[24]
Galdran A, Pardo D, Picón A, et al. Automatic Red-Channel underwater image restoration J Vis Commun Image Represent 2015 26 132-14
[25]
Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks J Mach Learn Res 2016 17 1 2096-2030
[26]
Gao M, Li S, Wang K, et al. Real-time jellyfish classification and detection algorithm based on improved YOLOv4-tiny and improved underwater image enhancement algorithm Sci Rep 2023 13 1 12989
[27]
Harary S, Schwartz E, Arbelle A et al (2022) Unsupervised domain generalization by Learning a bridge across domains. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 5280–5290
[28]
He K, Sun J, Tang X (2009) Single image haze removal using dark channel prior. In: 2009 IEEE conference on computer vision and pattern recognition. pp 1956–196.
[29]
Hu K, Zhang Y, Lu F, et al. An underwater image enhancement algorithm based on MSR parameter optimization J Marine Sci Eng 2020 8 10 74
[30]
Huang H, Zhou H, Yang X, et al. Faster R-CNN for marine organisms detection and recognition using data augmentation Neurocomputing 2019 337 372-38
[31]
Islam MJ, Xia Y, and Sattar J Fast underwater image enhancement for improved visual perception IEEE Robot Autom Lett 2020 5 2 3227-323
[32]
Jaffe J Computer modeling and the design of optimal underwater imaging systems IEEE J Oceanic Eng 1990 15 2 101-111
[33]
Jia C and Zhang Y Meta-learning the invariant representation for domain generalization Mach Learn 2024 113 4 1661-1681
[34]
Jiang L, Wang Y, Jia Q et al (2021) Underwater species detection using channel sharpening attention. In: Proceedings of the 29th ACM international conference on multimedia. association for computing machinery. pp 4259–426
[35]
Jobson D, Rahman Z, and Woodell G A multiscale retinex for bridging the gap between color images and the human observation of scenes IEEE Trans Image Process 1997 6 7 965-97
[36]
Jocher G, Chaurasia A, Qiu J (2023) Ultralytics YOLO. https://github.com/ultralytics/ultralytics
[37]
Kabir H and Garg N Machine learning enabled orthogonal camera goniometry for accurate and robust contact angle measurements Sci Rep 2023 13 1 149
[38]
Kang Y, Jiang Q, Li C, et al. A perception-aware decomposition and fusion framework for underwater image enhancement IEEE Trans Circuits Syst Video Technol 2023 33 3 988-1002
[39]
Katija K, Roberts PLD, Daniels J, et al (2021) Visual tracking of deepwater animals using machine learning-controlled robotic underwater vehicles. In: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). pp 859–86.
[40]
Katija K, Orenstein E, Schlining B, et al. FathomNet: a global image database for enabling artificial intelligence in the ocean Sci Rep 2022 12 1 1591
[41]
Kolmogorov AN Three approaches to the quantitative definition of information Int J Comput Math 1968 2 1–4 157-168
[42]
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, pp 1097–1105
[43]
Land EH The retinex theory of color vision Sci Am 1977 237 6 108-12
[44]
Lee W, Hong D, Lim H et al (2024) Object-aware domain generalization for object detection. arXiv:2312.12133 [cs]
[45]
Li C, Guo J, and Guo C Emerging from water: underwater image color correction based on weakly supervised color transfer IEEE Signal Process Lett 2018 25 3 323-327
[46]
Li C, Guo C, Ren W, et al. An underwater image enhancement benchmark dataset and beyond IEEE Trans Image Process 2019 29 4376-438
[47]
Li C, Anwar S, and Porikli F Underwater scene prior inspired deep underwater image and video enhancement Pattern Recogn 2020 98 107038
[48]
Li CY, Guo JC, Cong RM, et al. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior IEEE Trans Image Process 2016 25 12 5664-567
[49]
Li D and Du L Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish Artif Intell Rev 2022 55 5 4077-411
[50]
Li H, Pan SJ, Wang S et al (2018) Domain generalization with adversarial feature learning. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. pp 5400–5409.
[51]
Li J, Skinner KA, Eustice RM et al (2017) WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot Autom Lett 1.
[52]
Li P, Li D, Li W et al (2021) A simple feature augmentation for domain generalization. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 8886–8895
[53]
Li Y, Tian X, Gong M et al (2018) Deep domain generalization via conditional invariant adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV). pp 624–639
[54]
Lin C, Yuan Z, Zhao S et al (2021) Domain-invariant disentangled network for generalizable object detection. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp 8751–8760.
[55]
Lin TY, Maire M, Belongie S et al (2014) Microsoft COCO: Common Objects in Context. In: Fleet D, Pajdla T, Schiele B, et al (eds) Computer vision - ECCV 2014, Lecture notes in computer science. pp 740–755.
[56]
Liu C, Li H, Wang S et al (2021) A dataset and benchmark of underwater object detection for robot picking. In: 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp 1–6.
[57]
Liu H, Song P, Ding R (2020) Towards domain generalization in underwater object detection. In: 2020 IEEE International Conference on Image Processing (ICIP). pp 1971–1975.
[58]
Liu P, Qian W, and Wang Y YWnet: a convolutional block attention-based fusion deep learning method for complex underwater small target detection Eco Inform 2024 79 102401
[59]
Liu R, Fan X, Zhu M, et al. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light IEEE Trans Circuits Syst Video Technol 2020 30 12 4861-487
[60]
Liu Z, Wang B, Li Y, et al. UnitModule: a lightweight joint image enhancement module for underwater object detection Pattern Recogn 2024 151 11043
[61]
Lopez-Marcano S, L. Jinks E, Buelow CA et al (2021) Automatic detection of fish and tracking of movement for ecology. Ecol Evol 11(12):8254–826.
[62]
Lyu L, Liu Y, Xu X, et al. EFP-YOLO: a quantitative detection algorithm for marine benthic organisms Ocean Coastal Manag 2023 243 10677
[63]
Ma H, Zhang Y, Sun S, et al. Weighted multi-error information entropy based you only look once network for underwater object detection Eng Appl Artif Intell 2024 130 107766
[64]
Mandal R, Connolly RM, Schlacher TA et al (2018) Assessing fish abundance from underwater video using deep neural networks. In: 2018 International Joint Conference on Neural Networks (IJCNN). pp 1–6.
[65]
Marrable D, Barker K, Tippaya S et al (2022) Accelerating species recognition and labelling of fish from underwater video with machine-assisted deep learning. Front Mar Sci 9.
[66]
Meng R, Li X, Chen W et al (2022) Attention diversification for domain generalization. In: Avidan S, Brostow G, Cissé M, et al (eds) Computer Vision - ECCV 2022, Lecture notes in computer science. pp 322–34.
[67]
Mittal A, Soundararajan R, and Bovik AC Making a “completely blind” image quality analyzer IEEE Signal Process Lett 2013 20 3 209-212
[68]
Måløy H, Aamodt A, and Misimi E A spatio-temporal recurrent network for salmon feeding action recognition from underwater videos in aquaculture Comput Electron Agric 2019 167 105087
[69]
Motiian S, Piccirilli M, Adjeroh DA et al (2017) Unified deep supervised domain adaptation and generalization. In: 2017 IEEE International Conference on Computer Vision (ICCV). pp 5716–5726.
[70]
Muksit AA, Hasan F, Hasan Bhuiyan Emon MF, et al. YOLO-Fish: a robust fish detection model to detect fish in realistic underwater environment Eco Inform 2022 72 101847
[71]
NVIDIA (2023) The NVIDIA Data Loading Library (DALI). https://github.com/NVIDIA/DALI
[72]
Ottaviani E, Francescangeli M, Gjeci N et al (2022) Assessing the image concept drift at the OBSEA coastal underwater cabled observatory. Front Mar Sci.
[73]
Pal SK, Pramanik A, Maiti J, et al. Deep learning in multi-object detection and tracking: state of the art Appl Intell 2021 51 9 6400-642
[74]
Panetta K, Gao C, and Agaian S Human-visual-system-inspired underwater image quality measures IEEE J Oceanic Eng 2016 41 3 541-551
[75]
Panetta K, Kezebou L, Oludare V, et al. Comprehensive underwater object tracking benchmark dataset and underwater image enhancement with GAN IEEE J Oceanic Eng 2022 47 1 59-75
[76]
Peng L, Zhu C, Bian L (2023) U-shape transformer for underwater image enhancement. In: Karlinsky L, Michaeli T, Nishino K (eds) Computer Vision - ECCV 2022 workshops. Springer Nature Switzerland, Cham, pp 290–307.
[77]
Pizer SM, Amburn EP, Austin JD, et al. Adaptive histogram equalization and its variations Comput Vis Graph Image Process 1987 39 3 355-36
[78]
Pu H, Zhang D, Xu K et al (2024) BNN-SAM: Improving generalization of binary object detector by Seeking Flat Minima. Appl Intell.
[79]
Redmon J, Divvala S, Girshick R et al (2016) You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 779–788.
[80]
Ren S, He K, Girshick R et al (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems
[81]
Rizzi A, Gatta C, and Marini D A new algorithm for unsupervised global and local color correction Pattern Recogn Lett 2003 24 11 1663-1677
[82]
Rousseeuw PJ Silhouettes: a graphical aid to the interpretation and validation of cluster analysis J Comput Appl Math 1987 20 53-6
[83]
Ruiz-Frau A, Martin-Abadal M, Jennings CL, et al. The potential of Jellytoring 2.0 smart tool as a global jellyfish monitoring platform Ecol Evol 2022 12 11 e947
[84]
Saleh A, Laradji IH, Konovalov DA, et al. A realistic fish-habitat dataset to evaluate algorithms for underwater visual analysis Sci Rep 2020 10 1 1467
[85]
Saleh A, Sheaves M, Jerry D, et al. Applications of deep learning in fish habitat monitoring: A tutorial and survey Expert Syst Appl 2024 238 12184
[86]
Schechner Y, Karpel N (2004) Clear underwater vision. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. pp I–I.
[87]
Schmitt R The Ocean’s role in climate Oceanography 2018 31 2 100
[88]
Shankar S, Piratla V, Chakrabarti S et al (2018) Generalizing across domains via cross-gradient training. In: International conference on learning representations
[89]
Shui C, Wang B, and Gagné C On the benefits of representation regularization in invariance based domain generalization Mach Learn 2022 111 3 895-91
[90]
Sicilia A, Zhao X, and Hwang SJ Domain adversarial neural networks for domain generalization: when it works and how to improve Mach Learn 2023 112 7 2685-272
[91]
Volpi R, Namkoong H, Sener O et al (2018) Generalizing to unseen domains via adversarial data augmentation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18. pp 5339–5349
[92]
Svd Walt, Schönberger JL, Nunez-Iglesias J, et al. scikit-image: image processing in Python PeerJ 2014 2 e45
[93]
Wang J, Lan C, Liu C, et al. Generalizing to unseen domains: a survey on domain generalization IEEE Trans Knowl Data Eng 2023 35 8 8052-8072
[94]
Wang P, Zhang Z, Lei Z et al (2023) Sharpness-aware gradient matching for domain generalization. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 3769–3778.
[95]
Wang S, Yu L, Li C et al (2020) Learning from extrinsic and intrinsic supervisions for domain generalization. In: Vedaldi A, Bischof H, Brox T, et al (eds) Computer Vision - ECCV 2020, vol 12354. p 159–176.
[96]
Wang Z, Bovik A, Sheikh H, et al. Image quality assessment: from error visibility to structural similarity IEEE Trans Image Process 2004 13 4 600-61
[97]
Wu A, Deng C (2022) Single-domain generalized object detection in urban scene via cyclic-disentangled self-distillation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 837–84.
[98]
Wu X, Zhang L, Huang J, et al. Underwater image enhancement via modeling white degradation IEEE J Oceanic Eng 2024 49 4 1220-123
[99]
Wu Y, Kirillov A, Massa F et al (2019) Detectron2. https://github.com/facebookresearch/detectron2
[100]
Xu S, Zhang M, Song W, et al. A systematic review and analysis of deep learning-based underwater object detection Neurocomputing 2023 527 204-232
[101]
Yang X, Zhang S, Liu J, et al. Deep learning for smart fish farming: applications, opportunities and challenges Rev Aquac 2021 13 1 66-9
[102]
Yeh CH, Lin CH, Kang LW et al (2022) Lightweight deep neural network for joint learning of underwater object detection and color conversion. IEEE Trans Neural Netw Learn Syst 33(11):6129–614.
[103]
Zhang J, Zhu L, Xu L et al (2020) Research on the correlation between image enhancement and underwater object detection. In: 2020 Chinese Automation Congress (CAC). pp 5928–5933.
[104]
Zhang J, Zhang J, Zhou K et al (2023) An improved YOLOv5-based underwater object-detection framework. Sensors 23(7):3693.
[105]
Zhang W, Zhuang P, Sun HH, et al. Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement IEEE Trans Image Process 2022 31 3997-401
[106]
Zhang W, Zhou L, Zhuang P, et al. Underwater image enhancement via weighted wavelet visual perception fusion IEEE Trans Circuits Syst Video Technol 2024 34 4 2469-2483
[107]
Zhang X, Xu Z, Xu R et al (2022) Towards domain generalization in object detection. arXiv:2203.14387 [cs]
[108]
Zhou K, Yang Y, Hospedales T et al (2020) Learning to generate novel domains for domain generalization. In: Vedaldi A, Bischof H, Brox T, et al (eds) Computer Vision - ECCV 2020, vol 12361. p 561–578.
[109]
Zhou K, Liu Z, Qiao Y, et al. Domain generalization: a survey IEEE Trans Pattern Anal Mach Intell 2023 45 4 4396-4415
[110]
Zhu Q, Mai J, and Shao L A fast single image haze removal algorithm using color attenuation prior IEEE Trans Image Process 2015 24 11 3522-353
[111]
Zion B The use of computer vision technologies in aquaculture - a review Comput Electron Agric 2012 88 125-132
Index Terms
- A data-centric framework for combating domain shift in underwater object detection with image enhancement: A data-centric framework for combating domain shift in underwater object detection...
Index terms have been assigned to the content through auto-classification.
Recommendations
Underwater image enhancement by color correction and color constancy via Retinex for detail preserving
AbstractIn underwater, light attenuation causes non-uniform illumination that degrades underwater image. To enhance the degraded image, we propose an underwater image enhancement method that includes color correction, color constancy, multi-...
Graphical abstractDisplay Omitted
Highlights- The formulation of color correction compensates the red and blue channels by masking.
UDUIE: Unpaired Domain-Irrelevant Underwater Image Enhancement
PRICAI 2024: Trends in Artificial IntelligenceAbstractUnderwater image enhancement is fundamental to ocean exploration, as it can reduce the distortion caused by refraction and absorption in underwater imaging. Although many deep-learning-based underwater image enhancement methods achieved good ...
Review on underwater image restoration and enhancement algorithms
ICIMCS '15: Proceedings of the 7th International Conference on Internet Multimedia Computing and ServiceThe serious degradation of underwater image is the result of light scattering by particles, and light absorption causes the color attenuation. Thus, the task of underwater enhancement is improving the visibility and restoring the color by removing ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s) 2025.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 04 January 2025
Accepted: 21 December 2024
Author Tags
Author Tag
Qualifiers
- Research-article
Funding Sources
- Griffith University
Open Access funding enabled and organized by CAUL and its Member Institutions
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025