这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

Saliency optimization fused background feature with frequency domain features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the non-deep learning-based salient object detection methods known so far, the detection effect and robustness based on the background detection method are good. However, results are not desirable in small objects and complex scene images. This paper proposes a salient object detection algorithm, which employs a fusion framework to fuse background and frequency domain features to improve the accuracy of salient object detection. First, an improved background model is proposed for salient object detection to extract the background feature of the image. Simultaneously, the frequency domain features are obtained by the proposed frequency domain algorithm, which combines global information and local details by the Gaussian pyramid algorithm and different filters. Then, within our fusion framework, the fusion operations are guided by the self-attention mechanism to fuse background and multi-scale frequency domain features to obtain the self-attention maps. Finally, this paper introduces a fusion algorithm to derive the final saliency map from the self-attention maps. The results demonstrate that the proposed method consistently outperforms state-of-the-art approaches in four evaluation metrics on six challenging and complicated datasets and improves the accuracy of salient object detection in complex and small object scene images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  1. Borji A (2015) What is a salient object? a dataset and a baseline model for salient object detection. IEEE Transactions on Image Processing 24(2):742–756. https://doi.org/10.1109/TIP.2014.2383320

    Article  MathSciNet  Google Scholar 

  2. Jia-Ying WU, Sai Y, Jun DU, Hong-Da L (2019) Review of bottom-up salient object detection. Computer Science

  3. Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33:898–916

    Article  Google Scholar 

  4. Nyo MT, Mebarek-Oudina F, Hlaing SS, Khan NA (2022) Otsu’s thresholding technique for mri image brain tumor segmentation. Multimedia Tools and Applications 1–13

  5. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-3(6):610–621. https://doi.org/10.1109/TSMC.1973.4309314

  6. Huo L, Rao T, Zhang L (2019) Fused feature encoding in convolutional neural network. Multimedia Tools and Applications 78:1635–1648

    Article  Google Scholar 

  7. Xu Q, Li M, Yu M (2019) Learning to rank with relational graph and pointwise constraint for cross-modal retrieval. Soft Computing 23:9413–9427

    Article  Google Scholar 

  8. Pillai MS, Chaudhary G, Khari M, Crespo RG (2021) Real-time image enhancement for an automatic automobile accident detection through cctv using deep learning. Soft Computing 25(18):11929–11940

    Article  Google Scholar 

  9. Rong Li, X Jilkov VP (2003) Survey of maneuvering target tracking. part i. dynamic models. IEEE Transactions on Aerospace and Electronic Systems 39(4):1333–1364. https://doi.org/10.1109/TAES.2003.1261132

  10. Li C, Chen Z, Wu QJ, Liu C (2019) Saliency object detection: integrating reconstruction and prior. Machine Vision and Applications 30(3):397–406

    Article  Google Scholar 

  11. Tong N, Lu H, Zhang Y, Ruan X (2015) Salient object detection via global and local cues. Pattern Recognition 48(10):3258–3267

    Article  Google Scholar 

  12. Yuan Y, Li C, Kim J, Cai W, Feng DD (2017) Reversion correction and regularized random walk ranking for saliency detection. IEEE Transactions on Image Processing 27(3):1311–1322

    Article  MathSciNet  Google Scholar 

  13. Qin, Y, Lu, H, Xu, Y, Wang, H.: Saliency detection via cellular automata. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 110–119 (2015)

  14. Perazzi F, Krähenbühl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: 2012 IEEE Conference on computer vision and pattern recognition, pp 733–740. IEEE

  15. Xu Q, Wang F, Gong Y, Wang Z, Zeng K, Li Q, Luo X (2019) A novel edge-oriented framework for saliency detection enhancement. Image and Vision Computing 87:1–12

    Article  Google Scholar 

  16. Li X, Zhao L, Wei L, Yang MH, Wu F, Zhuang Y, Ling H, Wang J (2016) Deepsaliency: Multi-task deep neural network model for salient object detection. IEEE Transactions on Image Processing 25(8):3919–3930

    Article  MathSciNet  Google Scholar 

  17. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5455–5463

  18. Lee G, Tai YW, Kim J (2016) Deep saliency with encoded low level distance map and high level features. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 660–668

  19. Wei J, Wang S, Huang Q (2020) F3net: Fusion, feedback and focus for salient object detection. Proceedings of the AAAI Conference on artificial intelligence 34:12321–12328

    Article  Google Scholar 

  20. Gu Y, Wang L, Wang Z, Liu Y, Cheng MM, Lu SP (2020) Pyramid constrained self-attention network for fast video salient object detection. Proceedings of the AAAI Conference on artificial intelligence 34:10869–10876

    Article  Google Scholar 

  21. Liu S, Huang D, Wang Y (2019) Learning spatial fusion for single-shot object detection. arXiv:1911.09516

  22. Wei Y, Wen F, Zhu W, Sun J (2012) Geodesic saliency using background priors. In: European conference on computer vision, pp 29–42. Springer

  23. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 2814–2821

  24. Li J, Levine MD, An X, Xu X, He H (2012) Visual saliency based on scalespace analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(4):996–1010

    Article  Google Scholar 

  25. Li W, Yang X, Li C, Lu R, Xie X (2020) Fast visual saliency based on multi-scale difference of gaussians fusion in frequency domain. IET Image Processing 14(16):4039–4048

    Article  Google Scholar 

  26. Liu Z, Yang X, Liu Y, Qian Z (2019) Smoke-detection framework for highdefinition video using fused spatial-and frequency-domain features. IEEE Access 7:89687–89701

    Article  Google Scholar 

  27. Song S, Jia Z, Yang J, Kasabov N (2022) Salient detection via the fusion of background-based and multiscale frequency-domain features. Information Sciences 618:53–71

    Article  Google Scholar 

  28. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Advances in Neural Information Processing Systems 30:5998–6008

    Google Scholar 

  29. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image superresolution using very deep residual channel attention networks. In: Proceedings of the european conference on computer vision (ECCV), pp 286–301

  30. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H,Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 3156–3164

  31. Chen L, Zhang H, Xiao J, Nie L, Shao J, Liu W, Chua TS (2017) Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 5659–5667

  32. Wang W, Shen J, Shao L, Porikli F (2016) Correspondence driven saliency transfer. IEEE Transactions on Image Processing 25(11):5025–5034. https://doi.org/10.1109/TIP.2016.2601784

    Article  MathSciNet  Google Scholar 

  33. Zhang Z, Liang Y, Zheng J, Li K, Ding Z, Sun D (2019) Saliency optimization integrated robust background detection with global ranking. In: International conference on intelligent science and big data engineering, pp 517–528. Springer

  34. Yan Y, Ren J, Sun G, Zhao H, Han J, Li X, Marshall S, Zhan J (2018) Unsupervised image saliency detection with gestalt-laws guided optimization and visual attention based renement. Pattern Recognition 79:65–78

    Article  Google Scholar 

  35. Hou, X, Zhang, L.: Saliency detection: A spectral residual approach. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8 (2007)

  36. Chenlei Guo, Qi Ma, Liming Zhang (2008) Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: 2008 IEEE Conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2008.4587715

  37. Wang W, Lai Q, Fu H, Shen J, Ling H, Yang R (2021) Salient object detection in the deep learning era: An in-depth survey. IEEE Transactions on pattern analysis and machine intelligence 1–1. https://doi.org/10.1109/TPAMI.2021.3051099

  38. Chen S, Tan X, Wang B, Lu H, Hu X, Fu Y (2020) Reverse attentionbased residual network for salient object detection. IEEE Transactions on Image Processing 29:3763–3776

    Article  Google Scholar 

  39. Li J, Pan Z, Liu Q, Cui Y, Sun Y (2020) Complementarity-aware attention network for salient object detection. IEEE Transactions on Cybernetics

  40. Li G, Yu Y (2016) Visual saliency detection based on multiscale deep cnn features. IEEE Transactions on Image Processing 25(11):5012–5024

    Article  MathSciNet  Google Scholar 

  41. Xu Q, Wang Z, Wang F, Gong Y (2019) Multi-feature fusion cnns for drosophila embryo of interest detection. Physica A: Statistical Mechanics and its Applications 531:121808

    Article  Google Scholar 

  42. Wang C, Dong S, Zhao X, Papanastasiou G, Zhang H, Yang G (2020) Saliencygan: Deep learning semisupervised salient object detection in the fog of iot. IEEE Transactions on Industrial Informatics 16(4):2667–2676. https://doi.org/10.1109/TII.2019.2945362

    Article  Google Scholar 

  43. Kim KS, Zhang D, Kang MC, Ko SJ (2013) Improved simple linear iterative clustering superpixels. In: 2013 IEEE International symposium on consumer electronics (ISCE), pp 259–260. IEEE

  44. Gonzalez RC, Woods RE (2008) Digital image processing. Prentice Hall. International 28(4):484–486

    Google Scholar 

  45. Ramsey JD, Sanchez-Romero R, Glymour C (2014) Non-gaussian methods and high-pass filters in the estimation of effective connections. Neuroimage 84:986–1006

    Article  Google Scholar 

  46. Wang S, Li W, Wang Y, Jiang Y, Jiang S, Zhao R (2012) An improved difference of gaussian filter in face recognition. Journal of Multimedia 7(6):429–433

    Article  Google Scholar 

  47. Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid methods in image processing. RCA Engineer 29(6):33–41

    Google Scholar 

  48. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 7794–7803

  49. Cheng MM, Mitra NJ, Huang X, Torr PHS, Hu SM (2015) Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(3):569–582

    Article  Google Scholar 

  50. Shi J, Yan Q, Xu L, Jia J (2015) Hierarchical image saliency detection on extended cssd. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(4):717–729

    Article  Google Scholar 

  51. Alpert S, Galun M, Brandt A, Basri R (2011) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE transactions on pattern analysis and machine intelligence 34(2):315–327

    Article  Google Scholar 

  52. Movahedi V, Elder JH (2010) Design and perceptual validation of performance measures for salient object segmentation. In: 2010 IEEE Computer society conference on computer vision and pattern recognition-workshops, pp 49–56. IEEE

  53. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3166–3173

  54. Fan DP, Cheng MM, Liu JJ, Gao SH, Hou Q, Borji A (2018) Salient objects in clutter: Bringing salient object detection to the foreground. In: Proceedings of the european conference on computer vision (ECCV), pp 186–202

Download references

Acknowledgements

This research was funded by the National Natural Science Foundation of China with Grant U1803261 and 62261053, and Scientific research plan of universities in Xinjiang Uygur Autonomous Region under grant XJEDU2019Y006.

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Key Laboratory of Signal Detection and Processing, Xinjiang University, Urumqi, 830046, Xinjiang, China

    Sensen Song, Zhenhong Jia, Fei Shi & Junnan Wang

  2. College of Mathematics and System Science, Xinjiang University, Urumqi, Xinjiang, China

    Sensen Song

  3. Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, 200400, China

    Jie Yang

  4. Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Auckland, 1020, New Zealand

    Nikola Kasabov

  5. George Moore Chair in Data Analytics, Ulster University, Maggy, BT48 7JL, UK

    Nikola Kasabov

Authors
  1. Sensen Song
    View author publications

    Search author on:PubMed Google Scholar

  2. Zhenhong Jia
    View author publications

    Search author on:PubMed Google Scholar

  3. Fei Shi
    View author publications

    Search author on:PubMed Google Scholar

  4. Junnan Wang
    View author publications

    Search author on:PubMed Google Scholar

  5. Jie Yang
    View author publications

    Search author on:PubMed Google Scholar

  6. Nikola Kasabov
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Zhenhong Jia.

Ethics declarations

Conflicts of interest

The authors declare that they have no Conflict of Interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, S., Jia, Z., Shi, F. et al. Saliency optimization fused background feature with frequency domain features. Multimed Tools Appl 83, 40509–40528 (2024). https://doi.org/10.1007/s11042-023-16760-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11042-023-16760-5

Keywords