这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Underwater images are often affected by light refraction and absorption, reducing visibility and interfering with subsequent applications. Existing underwater image enhancement methods primarily focus on improving visual quality while overlooking practical implications. To strike a balance between visual quality and application, we propose a heuristic invertible network for underwater perception enhancement, dubbed HUPE, which enhances visual quality and demonstrates flexibility in handling other downstream tasks. Specifically, we introduced a information-preserving reversible transformation with embedded Fourier transform to establish a bidirectional mapping between underwater images and their clear images. Additionally, a heuristic prior is incorporated into the enhancement process to better capture scene information. To further bridges the feature gap between vision-based enhancement images and application-oriented images, a semantic collaborative learning module is applied in the joint optimization process of the visual enhancement task and the downstream task, which guides the proposed enhancement model to extract more task-oriented semantic features while obtaining visually pleasing images. Extensive experiments, both quantitative and qualitative, demonstrate the superiority of our HUPE over state-of-the-art methods. The source code is available at https://github.com/ZengxiZhang/HUPE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  • Bouguer, P. (1729). Essai D’optique sur la Gradation de la Lumière, .

  • Brigham, E. O., & Morrow, R. (1967). The fast Fourier transform. IEEE spectrum, 4(12), 63–70.

    Article  Google Scholar 

  • Cai, L., McGuire, N. E., Hanlon, R., Mooney, T. A., & Girdhar, Y. (2023). Semi-supervised visual tracking of marine animals using autonomous underwater vehicles. International Journal of Computer Vision, 131(6), 1406–1427.

    Article  Google Scholar 

  • Cai, M., Wang, Y., Wang, S., Wang, R., Ren, Y., & Tan, M. (2020). Grasping marine products with hybrid-driven underwater vehicle-manipulator system. IEEE Transactions on Automation Science and Engineering, 17(3), 1443–1454.

    Google Scholar 

  • Chen, X., Zhang, P., Quan, L., Yi, C., & Lu, C. (2021). Underwater image enhancement based on deep learning and image formation model. arXiv preprint arXiv:2101.00991 .

  • Chen, R., Mihaylova, L., Zhu, H., & Bouaynaya, N. C. (2020). A deep learning framework for joint image restoration and recognition. Circuits, Systems, and Signal Processing, 39(3), 1561–1580.

    Article  Google Scholar 

  • Chi, Z., Wang, Y., Yu, Y., & Tang, J. (2021). Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition (pp. 9137-9146).

  • Chiang, J. Y., & Chen, Y.-C. (2011). Underwater image enhancement by wavelength compensation and dehazing. IEEE Transactions on Image Processing, 21(4), 1756–1769.

    Article  MathSciNet  Google Scholar 

  • Cong, X., Gui, J., & Hou, J. (2024). Underwater organism color fine-tuning via decomposition and guidance. In Proceedings of the AAAI conference on artificial intelligence 38, (pp. 1389–1398).

  • Drews, P., Nascimento, E., Moraes, F., Botelho, S., & Campos, M. (2013). Transmission estimation in underwater single images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 825–830 .

  • Fan, G.-D., Fan, B., Gan, M., Chen, G.-Y., & Chen, C. P. (2022). Multiscale low-light image enhancement network with illumination constraint. IEEE Transactions on Circuits and Systems for Video Technology, 32(11), 7403–7417.

    Article  Google Scholar 

  • Fang, Y., Ma, K., Wang, Z., Lin, W., Fang, Z., & Zhai, G. (2014). No-reference quality assessment of contrast-distorted images based on natural scene statistics. IEEE Signal Processing Letters, 22(7), 838–842.

    Google Scholar 

  • Ghani, A. S. A., & Isa, N. A. M. (2015). Underwater image quality enhancement through integrated color model with Rayleigh distribution. Applied Soft Computing, 27, 219–230.

    Article  Google Scholar 

  • Han, J., Shoeiby, M., Malthus, T., Botha, E., Anstee, J., Anwar, S., Wei, R., Armin, M. A., Li, H., & Petersson, L. (2022). Underwater image restoration via contrastive learning and a real-world dataset. Remote Sensing, 14, 4297.

    Article  Google Scholar 

  • Hitam, M.S., Awalludin, E.A., Yussof, W.N.J.H.W., & Bachok, Z. (2013). Mixture contrast limited adaptive histogram equalization for underwater image enhancement. In 2013 International Conference on Computer Applications Technology (ICCAT) (pp. 1–5). IEEE.

  • Hsiao, Y.-H., Chen, C.-C., Lin, S.-I., & Lin, F.-P. (2014). Real-world underwater fish recognition and identification, using sparse representation. Ecological informatics, 23, 13–21.

    Article  Google Scholar 

  • Huang, S., Wang, K., Liu, H., Chen, J., & Li, Y. (2023). Contrastive semi-supervised learning for underwater image restoration via reliable bank. arXiv preprint arXiv:2303.09101 .

  • Hughes, B., & Burghardt, T. (2017). Automated visual fin identification of individual great white sharks. International Journal of Computer Vision, 122, 542–557.

    Article  Google Scholar 

  • Iqbal, K., Odetayo, M., James, A., Salam, R.A., & Talib, A.Z.H. (2010). Enhancing the low quality images using unsupervised colour correction method. In 2010 IEEE International conference on systems, man and cybernetics (pp. 1703–1709). IEEE.

  • Islam, M.J., Edge, C., Xiao, Y., Luo, P., Mehtaz, M., Morse, C., Enan, S.S., & Sattar, J. (2020). Semantic segmentation of underwater imagery: Dataset and benchmark. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

  • Islam, M. J., Xia, Y., & Sattar, J. (2020). Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 5(2), 3227–3234.

    Article  Google Scholar 

  • Jiang, Z., Li, Z., Yang, S., Fan, X., & Liu, R. (2022). Target oriented perceptual adversarial fusion network for underwater image enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 32, 6584–6598.

    Article  Google Scholar 

  • Kim, K., & Lee, H.S. (2020). Probabilistic anchor assignment with iou prediction for object detection. In European conference on computer vision (pp. 355–371) Springer.

  • Kingma, D.P., & Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems 31 .

  • Lee, S., Cho, D., Kim, J., & Kim, T.H. (2020). Self-supervised fast adaptation for denoising via meta-learning. arXiv preprint arXiv:2001.02899.

  • Li, H., Li, J., & Wang, W. (2019). A fusion adversarial underwater image enhancement network with a public test dataset. arXiv preprint arXiv:1906.06819 .

  • Li, H., Li, J., Zhao, D., & Xu, L. (2021). Dehazeflow: Multi-scale conditional flow network for single image dehazing. In Proceedings of the 29th ACM International conference on multimedia (pp. 2577–2585).

  • Li, C., Quo, J., Pang, Y., Chen, S., & Wang, J. (2016). Single underwater image restoration by blue-green channels dehazing and red channel correction. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1731–1735). IEEE.

  • Li, C., Anwar, S., Hou, J., Cong, R., Guo, C., & Ren, W. (2021). Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 30, 4985–5000.

    Article  Google Scholar 

  • Li, L., Dong, B., Rigall, E., Zhou, T., Dong, J., & Chen, G. (2021). Marine animal segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 32(4), 2303–2314.

    Article  Google Scholar 

  • Li, C., Guo, J., Guo, C., Cong, R., & Gong, J. (2017). A hybrid method for underwater image correction. Pattern Recognition Letters, 94, 62–67.

    Article  Google Scholar 

  • Li, C., Guo, C., Ren, W., Cong, R., Hou, J., Kwong, S., & Tao, D. (2019). An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29, 4376–4389.

    Article  Google Scholar 

  • Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (pp. 2980–2988).

  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A.C. (2016). Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, October 11–14, 2016, Proceedings, Part I 14 (pp. 21–37). Springer.

  • Liu, R., Gao, J., Liu, X., & Fan, X. (2024). Learning with constraint learning: New perspective, solution strategy and various applications. IEEE Transactions on Pattern Analysis and Machine Intelligence .

  • Liu, R., Liu, Z., Liu, J., Fan, X., & Luo, Z. (2024). A task-guided, implicitly-searched and metainitialized deep model for image fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence

  • Liu, R., Liu, X., Zeng, S., Zhang, J., & Zhang, Y. (2023). Hierarchical optimization-derived learning. IEEE Transactions on Pattern Analysis and Machine Intelligence.

  • Liu, R., Liu, X., Zeng, S., Zhang, J., & Zhang, Y. (2023). Value-function-based sequential minimization for bi-level optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence .

  • Liu, R., Fan, X., Zhu, M., Hou, M., & Luo, Z. (2020). Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Transactions on Circuits and Systems for Video Technology, 30(12), 4861–4875.

    Article  Google Scholar 

  • Liu, R., Jiang, Z., Yang, S., & Fan, X. (2022). Twin adversarial contrastive learning for underwater image enhancement and beyond. IEEE Transactions on Image Processing, 31, 4922–4936.

    Article  Google Scholar 

  • Liu, J., Lin, R., Wu, G., Liu, R., Luo, Z., & Fan, X. (2024). Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion. International Journal of Computer Vision, 132(5), 1748–1775.

    Article  Google Scholar 

  • Liu, J., Shang, J., Liu, R., & Fan, X. (2022). Attention-guided global-local adversarial learning for detail-preserving multi-exposure image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 32(8), 5026–5040.

    Article  Google Scholar 

  • Li, K., Wu, L., Qi, Q., Liu, W., Gao, X., Zhou, L., & Song, D. (2022). Beyond single reference for training: Underwater image enhancement via comparative learning. IEEE Transactions on Circuits and Systems for Video Technology, 33, 2561–2576.

    Article  Google Scholar 

  • McEver, R. A., Zhang, B., Levenson, C., Iftekhar, A., & Manjunath, B. (2023). Context-driven detection of invertebrate species in deep-sea video. International Journal of Computer Vision, 131(6), 1367–1388.

    Article  Google Scholar 

  • Mu, P., Qian, H., & bBai, C. (2022). Structure-inferred bi-level model for underwater image enhancement. In Proceedings of the 30th ACM International conference on multimedia (pp. 2286–2295).

  • Panetta, K., Gao, C., & Agaian, S. (2015). Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 41(3), 541–551.

    Article  Google Scholar 

  • Peng, Y.-T., Cao, K., & Cosman, P. C. (2018). Generalization of the dark channel prior for single image restoration. IEEE Transactions on Image Processing, 27(6), 2856–2868.

    Article  MathSciNet  Google Scholar 

  • Prytula, S. (2020). Underwater Object Detection Dataset. https://public.roboflow.com/object-detection/aquarium.

  • Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019). Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition (pp. 658–666).

  • Shen, H., Zhao, Z.-Q., Zhang, Y., & Zhang, Z. (2023). Mutual information-driven triple interaction network for efficient image dehazing. In Proceedings of the 31st ACM international conference on multimedia (pp. 7–16).

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 .

  • Sun, S., Ren, W., Wang, T., & Cao, X. (2022). Rethinking image restoration for object detection. Advances in Neural Information Processing Systems, 35, 4461–4474.

    Google Scholar 

  • Walther, D., Edgington, D.R., & Koch, C. (2004). Detection and tracking of objects in underwater video. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2004., vol. 1. IEEE.

  • Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., & Latecki, L.J. (2019). Lednet: A lightweight encoder-decoder network for real-time semantic segmentation. In 2019 IEEE International Conference on Image Processing (ICIP) (pp. 1860–1864). IEEE.

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wang, Y., Liu, H., & Chau, L.-P. (2017). Single underwater image restoration using adaptive attenuation-curve prior. IEEE Transactions on Circuits and Systems I: Regular Papers, 65(3), 992–1002.

    Google Scholar 

  • Wu, G., Fu, H., Liu, J., Ma, L., Fan, X., & Liu, R. (2024). Hybrid-supervised dual-search: Leveraging automatic learning for loss-free multi-exposure image fusion. In Proceedings of the AAAI conference on artificial intelligence 38, (pp. 5985–5993.

  • Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2020). Cgnet: A light-weight context guided network for semantic segmentation. IEEE Transactions on Image Processing, 30, 1169–1179.

    Article  Google Scholar 

  • Xu, Q., Zhang, R., Zhang, Y., Wang, Y., & Tian, Q. (2021). A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14383–14392).

  • Xue, X., Li, Z., Ma, L., Jia, Q., Liu, R., & Fan, X. (2023). Investigating intrinsic degradation factors by multi-branch aggregation for real-world underwater image enhancement. Pattern Recognition, 133, 109041.

    Article  Google Scholar 

  • Yang, M., & Sowmya, A. (2015). An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 24(12), 6062–6071.

    Article  MathSciNet  Google Scholar 

  • Yao, Z., Fan, G., Fan, J., Gan, M., & Philip Chen, C. L. (2024). Spatial–frequency dual-domain feature fusion network for low-light remote sensing image enhancement. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–16.

    Google Scholar 

  • Ye, T., Zhang, Y., Jiang, M., Chen, L., Liu, Y., Chen, S., & Chen, E. (2022). Perceiving and modeling density for image dehazing. In European conference on computer vision (pp. 130–145). Springer.

  • Yeh, C.-H., Lin, C.-H., Kang, L.-W., Huang, C.-H., Lin, M.-H., Chang, C.-Y., & Wang, C.-C. (2021). Lightweight deep neural network for joint learning of underwater object detection and color conversion. IEEE Transactions on Neural Networks and Learning Systems, 33(11), 6129–6143.

    Article  Google Scholar 

  • You, S., Tezcan, K.C., Chen, X., & Konukoglu, E. (2019). Unsupervised lesion detection via image restoration with a normative prior. In International conference on medical imaging with deep learning (pp. 540–556). PMLR.

  • Zeng, L., Sun, B., & Zhu, D. (2021). Underwater target detection based on faster R-CNN and adversarial occlusion network. Engineering Applications of Artificial Intelligence, 100, 104190.

    Article  Google Scholar 

  • Zhang, Z., Jiang, Z., Liu, J., Fan, X., & Liu, R. (2023). Waterflow: Heuristic normalizing flow for underwater image enhancement and beyond. In Proceedings of the 31st ACM International conference on multimedia (pp. 7314–7323) .

  • Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S.Z. (2018). Single-shot refinement neural network for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4203–4212 .

  • Zhang, D., Zhou, J., Guo, C., Zhang, W., & Li, C. (2024). Synergistic multiscale detail refinement via intrinsic supervision for underwater image enhancement. In Proceedings of the AAAI conference on artificial intelligence 38, (pp. 7033–7041).

  • Zhao, W., Xie, S., Zhao, F., He, Y., & Lu, H. (2023). Metafusion: Infrared and visible image fusion via meta-feature embedding from object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13955–13965).

  • Zhou, J., Sun, J., Li, C., Jiang, Q., Zhou, M., Lam, K.-M., Zhang, W., & Fu, X. (2024). Hclr-net: Hybrid contrastive learning regularization with locally randomized perturbation for underwater image enhancement. International Journal of Computer Vision .

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Nos. U22B2052, 12326605, 62027826, 62302078, 62450072, 62372080); in part by the National Key Research and Development Program of China (No. 2022YFA1 004101); and in part by China Postdoctoral Science Foundation (No. 2023M730741).

Author information

Authors and Affiliations

  1. School of Software Engineering, Dalian University of Technology, Dalian, 116024, China

    Zengxi Zhang, Long Ma, Jinyuan Liu, Xin Fan & Risheng Liu

  2. College of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China

    Zhiying Jiang

  3. School of Software Engineering, Pazhou Laboratory (Huangpu), Guangzhou, 510555, China

    Risheng Liu

Authors
  1. Zengxi Zhang
    View author publications

    Search author on:PubMed Google Scholar

  2. Zhiying Jiang
    View author publications

    Search author on:PubMed Google Scholar

  3. Long Ma
    View author publications

    Search author on:PubMed Google Scholar

  4. Jinyuan Liu
    View author publications

    Search author on:PubMed Google Scholar

  5. Xin Fan
    View author publications

    Search author on:PubMed Google Scholar

  6. Risheng Liu
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Risheng Liu.

Additional information

Communicated by Yue Gao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Jiang, Z., Ma, L. et al. HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning. Int J Comput Vis 133, 3259–3277 (2025). https://doi.org/10.1007/s11263-024-02318-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-024-02318-x

Keywords