+
Skip to main content

Advertisement

Springer Nature Link
Log in

HASN: hybrid attention separable network for efficient image super-resolution

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Recently, lightweight methods for single-image super-resolution have gained significant popularity and achieved impressive performance due to limited hardware resources. These methods demonstrate that adopting residual feature distillation is an effective way to enhance performance. However, we find that using residual connections after each block increases the model’s storage and computational cost. Therefore, to simplify the network structure and learn higher-level features and relationships between features, we use depth-wise separable convolutions, fully connected layers, and activation functions as the basic feature extraction modules. This significantly reduces computational load and the number of parameters while maintaining strong feature extraction capabilities. To further enhance model performance, we propose the hybrid attention separable block, which combines channel attention and spatial attention, thus making use of their complementary advantages. Additionally, we use depth-wise separable convolutions instead of standard convolutions, significantly reducing the computational load and the number of parameters while maintaining strong feature extraction capabilities. During the training phase, we also adopt a warm-start retraining strategy to exploit the potential of the model further. Extensive experiments demonstrate the effectiveness of our approach. Our method achieves a smaller model size and reduced computational complexity without compromising performance. Code can be available at https://github.com/nathan66666/HASN.git

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

All original codes have been deposited at Zenodo (https://doi.org/10.5281/zenodo.12730191) [54].

References

  1. Dong, C., Loy, C.C., He, K., et al.: Learning a deep convolutional network for image super-resolution. In: ECCV (4), Lecture Notes in Computer Science, vol 8692. Springer, pp 184–199 (2014)

  2. Liang, J., Cao, J., Sun, G., et al.: Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1833–1844 (2021)

  3. Chen, H., Gu, J., Zhang, Z.: Attention in Attention Network for Image Super-Resolution. arXiv preprint arXiv:2104.09497 (2021)

  4. Dong, C., Loy, C.C., He, K., et al.: Learning a deep convolutional network for image super-resolution. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13, Springer, pp 184–199 (2014)

  5. Zhang, K., Zuo, W., Gu, S., et al.: Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3929–3938 (2017)

  6. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1646–1654 (2016)

  7. Lim, B., Son, S., Kim, H., et al.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 136–144 (2017)

  8. Zhang, Y., Li, K., Li, K., et al.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 286–301 (2018)

  9. Niu, B., Wen, W., Ren, W., et al.: Single image super-resolution via a holistic attention network. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, Springer, pp 191–207 (2020)

  10. Zhang, Y., Li, K., Li, K., et al.: Residual Non-local Attention Networks for Image Restoration. arXiv preprint arXiv:1903.10082 (2019)

  11. Wang, X., Yu, K., Wu, S., et al.: Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on computer Vision (ECCV) Workshops, pp 0–0 (2018)

  12. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2), Lecture Notes in Computer Science, vol 9906. Springer, pp 391–407 (2016)

  13. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR. IEEE Computer Society, pp 1646–1654 (2016a)

  14. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: CVPR. IEEE Computer Society, pp 1637–1645 (2016b)

  15. Lai, W., Huang, J., Ahuja, N., et al.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: CVPR. IEEE Computer Society, pp 5835–5843 (2017)

  16. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR. IEEE Computer Society, pp 2790–2798 (2017a)

  17. Tai, Y., Yang, J., Liu, X., et al.: Memnet: A persistent memory network for image restoration. In: ICCV. IEEE Computer Society, pp 4549–4557 (2017b)

  18. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: CVPR. IEEE Computer Society, pp 723–731 (2018)

  19. Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: CVPR. IEEE Computer Society, pp 3262–3271 (2018)

  20. Ahn, N., Kang, B., Sohn, K.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: ECCV (10), Lecture Notes in Computer Science, vol 11214. Springer, pp 256–272 (2018)

  21. Hui, Z., Gao, X., Yang, Y., et al.: Lightweight image super-resolution with information multi-distillation network. In: ACM Multimedia. ACM, pp 2024–2032 (2019)

  22. Liu, J., Tang, J., Wu, G.: Residual feature distillation network for lightweight image super-resolution. In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, Springer, pp 41–55 (2020)

  23. Kong, F., Li, M., Liu, S., et al.: Residual local feature network for efficient super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 766–776 (2022)

  24. Yu, L., Li, X., Li, Y., et al.: Dipnet: efficiency distillation and iterative pruning for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1692–1701 (2023)

  25. Wan, C., Yu, H., Li, Z., et al.: Swift parameter-free attention network for efficient super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6246–6256 (2024)

  26. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1637–1645 (2016)

  27. Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 252–268 (2018)

  28. Liu, J., Zhang, W., Tang, Y., et al.: Residual feature aggregation network for image super-resolution. In: CVPR. IEEE, pp 2356–2365 (2020)

  29. Li, Z., Liu, Y., Chen, X., et al.: Blueprint separable residual network for efficient image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 833–843 (2022)

  30. Zhang, Y., Tian, Y., Kong, Y., et al.: Residual dense network for image super-resolution. In: CVPR. IEEE Computer Society, pp 2472–2481 (2018)

  31. Chen, X., Wang, X., Zhou, J., et al.: Activating more pixels in image super-resolution transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 22367–22377 (2023)

  32. Zhang, Y., Li, K., Li, K., et al.: Image super-resolution using very deep residual channel attention networks. In: ECCV (7), Lecture Notes in Computer Science, vol 11211. Springer, pp 294–310 (2018)

  33. Zhao, H., Kong, X., He, J., et al.: Efficient image super-resolution using pixel attention. In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, Springer, pp 56–72 (2020)

  34. Liu, Z., Lin, Y., Cao, Y., et al.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10012–10022 (2021)

  35. Deng, W., Yuan, H., Deng, L., et al.: Reparameterized residual feature network for lightweight image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1712–1721 (2023)

  36. Hou, B., Li, G.: Pccformer: Parallel Coupled Convolutional Transformer for Image Super-Resolution. The Visual Computer pp 1–12 (2024)

  37. Lin, X., Sun, S., Huang, W., et al.: Eapt: efficient attention pyramid transformer for image processing. IEEE Trans. Multimedia 25, 50–61 (2021)

    Article  MATH  Google Scholar 

  38. Zhou, Y., Chen, Z., Li, P., et al.: Fsad-net: feedback spatial attention dehazing network. IEEE Transact. Neural Netw. Learn. Syst. 34(10), 7719–7733 (2022)

    Article  MATH  Google Scholar 

  39. Huang, S., Liu, X., Tan, T., et al.: Transmrsr: transformer-based self-distilled generative prior for brain mri super-resolution. Vis. Comput. 39(8), 3647–3659 (2023)

    Article  MATH  Google Scholar 

  40. Zhang, X., Zeng, H., Zhang, L.: Edge-oriented convolution block for real-time super resolution on mobile devices. In: Proceedings of the 29th ACM International Conference on Multimedia, pp 4034–4043 (2021)

  41. Ding, X., Zhang, X., Ma, N., et al.: Repvgg: making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13733–13742 (2021)

  42. Du Zongcai, L.D., Jie, L., Jie, T., et al.: Fast and memory-efficient network towards efficient image super-resolution. In: NTIRE (CVPR Workshop) (2022)

  43. Timofte, R., Agustsson, E., Van Gool, L., et al.: Ntire 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 114–125 (2017)

  44. Bevilacqua, M., Roumy, A., Guillemot, C., et al.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC. BMVA Press, pp 1–10 (2012)

  45. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Curves and Surfaces, Lecture Notes in Computer Science, vol 6920. Springer, pp 711–730 (2010)

  46. Martin, D.R., Fowlkes, C.C., Tal, D., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, pp 416–425 (2001)

  47. Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR. IEEE Computer Society, pp 5197–5206 (2015)

  48. Matsui, Y., Ito, K., Aramaki, Y., et al.: Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl. 76(20), 21811–21838 (2017)

    Article  Google Scholar 

  49. Wang, Y., Zhang, T.: Osffnet: Omni-stage feature fusion network for lightweight image super-resolution. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 5660–5668 (2024)

  50. Ma, X., Dai, X., Bai, Y., et al.: Rewrite the stars. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5694–5703 (2024)

  51. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp 807–814 (2010)

  52. Maas, A.L., Hannun, A.Y., Ng, A.Y., et al.: Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml, Atlanta, GA, p 3 (2013)

  53. Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520 (2018)

  54. nathan66666 (2024) Hasn: v1.0.1. Zenodo, https://doi.org/10.5281/zenodo.12730191

Download references

Acknowledgements

This work is supported in part by Graduate Education Reform Project of Henan Province (2023SJGLX037Y), National Natural Science Foundation of China (62076223), Key Science and Technology Program of Henan Province (232102211018), and Key Research Project of Henan Province Universities (24ZX005).

Funding

No funds, grants, or other support was received.

Author information

Authors and Affiliations

  1. The School of Electrical and Information Engineering, Zhengzhou University of Light Industry, No.5 Dongfeng Road, Zhengzhou, 450002, Henan, China

    Weifeng Cao, Xiaoyan Lei, Jun Shi, Wanyong Liang, Jie Liu & Zongfei Bai

Authors
  1. Weifeng Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoyan Lei
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jun Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Wanyong Liang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jie Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Zongfei Bai
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Conceptualization was performed by [Xiaoyan Lei], [Weifeng Cao]; methodology by [Xiaoyan Lei], [Weifeng Cao], [Jun Shi], [Wanyong Liang]; formal analysis and investigation by [Xiaoyan Lei], [Jie Liu], [Zongfei Bai]; writing—original draft preparation—by [Xiaoyan Lei]; writing— review and editing—by [Weifeng Cao], [Xiaoyan Lei], [Jun Shi], [Wanyong Liang], [Jie Liu], [Zongfei Bai]; supervision by [Xiaoyan Lei], [Jun Shi], [Wanyong Liang], [Jie Liu], [Zongfei Bai].

Corresponding author

Correspondence to Xiaoyan Lei.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, W., Lei, X., Shi, J. et al. HASN: hybrid attention separable network for efficient image super-resolution. Vis Comput 41, 3423–3435 (2025). https://doi.org/10.1007/s00371-024-03610-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-024-03610-0

Keywords

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载