这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

Advanced Binary Neural Network for Single Image Super Resolution

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Binary neural network (BNN) is an effective approach to accelerate the model inference and has been initially applied in the field of single image super resolution (SISR). However, the optimization of efficiency and accuracy remains a major challenge for achieving further improvements. While existing BNN-based SR methods solve the SISR problems by proposing a residual block-oriented quantization mechanism, the quantization process in the up-sampling stage and the representation tendency of binary super resolution networks are ignored. In this paper, we propose an Advanced Binary Super Resolution (ABSR) method to optimize the binary generator in terms of quantization mechanism and up-sampling strategy. Specifically, we first design an excitation-selection mechanism for binary inference, which could distinctively implement self-adjustment of activation and significantly reduce inference errors. Furthermore, we construct a binary up-sampling strategy that achieves performance almost equal to that of real-valued up-sampling modules, and fully frees up the inference speed of the binary network. Extensive experiments show that the ABSR not only reaches state-of-the-art BNN-based SR performance in terms of objective metrics and visual quality, but also reduces computational consumption drastically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  • Ahn, N., Kang, B., & Sohn, KA .(2018). Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV).

  • Antoni, B., Joan, D., & Julia, N. (2019). Motion-compensated Spatio-temporal filtering for multi-image and multimodal super-resolution. International Journal of Computer Vision, 127(10), 1474–1500.

    Article  Google Scholar 

  • Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, ML. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding pp. 1–10.

  • Bulat, A., & Tzimiropoulos, G. (2019). Xnor-net++: Improved binary neural networks. arXiv preprint arXiv:1909.13863.

  • Chen, TQ., Rubanova, Y., Bettencourt, J., & Duvenaud, DK. (2018). Neural ordinary differential equations. In: Advances in neural information processing systems, pp. 6571–6583.

  • Courbariaux, M., Bengio, Y., & David, JP. (2015). Binaryconnect: Training deep neural networks with binary weights during propagations. In: Advances in .neural information processing systems, pp. 3123–3131

  • Dong, C., Loy, C. C., He, K., & Tang, X. (2015). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295–307.

    Article  Google Scholar 

  • Greenspan, H. (2009). Super-resolution in medical imaging. The Computer Journal, 52(1), 43–63.

    Article  Google Scholar 

  • Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020). Single path one-shot neural architecture search with uniform sampling. In: Proceedings of the European conference on computer vision (ECCV), Springer, vol 12361, pp. 544–560.

  • He, X., Mo, Z., Wang, P., Liu, Y., Yang, M., & Cheng, J. (2019). Ode-inspired network design for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Huang, JB., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206.

  • Huang, Y., Shao, L., & Frangi, AF. (2017). Simultaneous super-resolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016). Binarized neural networks. In: Advances in neural information processing systems, pp 4107–4115.

  • Hui, Z., Gao, X., Yang, Y., & Wang, X. (2019). Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM international conference on multimedia (ACM MM).

  • Jiang, X., Wang, N., Xin, J., Li, K., Yang, X., & Gao, X. (2021). Training binary neural network without batch normalization for image super-resolution. In: Proceedings of the thirty-fifth AAAI conference on artificial intelligence (AAAI).

  • Jiang, X., Wang, N., Xin, J., Li, K., Yang, X., Li, X., Wang, Jie, & Gao, X. (2022). Fabnet: Frequency-aware binarized network for single image super-resolution. In: IEEE Transactions on neural networks and learning systems. pp. 1–11.

  • Kim, J., Kwon, Lee, J., & Mu, Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Lai, WS., Huang, JB., Ahuja, N,. & Yang, MH. (2017). Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., & Wang, Z. et al (2017). Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Lei, Z., Peng, W., Chunhua, S., Lingqiao, L., Wei, W., Yanning, Z., & van den Hengel, A. (2020). Adaptive importance learning for improving lightweight image super-resolution network. International Journal of Computer Vision, 128(2), 479–499.

    Article  MathSciNet  Google Scholar 

  • Li, K., Wang, N., Xin, J., Jiang, X., Li, J., Gao, X., Han, K., & Wang, Y. (2022). Local means binary networks for image super-resolution. IEEE Transactions on Neural Networks and Learning Systems pp 1–11, 10.1109/TNNLS.2022.3212827.

  • Li, Y., Dong, X., Zhang, SQ., Bai, H., Chen, Y., & Wang, W. (2020). Rtn: Reparameterized ternary network. Proceedings of the thirty-fourth AAAI conference on artificial intelligence (AAAI).

  • Lim, B., Son, S., Kim, H., Nah, S., & Mu, Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW).

  • Lin, F., Fookes, C., Chandran, V., & Sridharan, S. (2007). Super-resolved faces for improved face recognition from surveillance video. In: Proceedings of the international conference on advances in biometrics.

  • Liu, J., Zhang, W., Tang, Y., Tang, J., & Wu, G. (2020a). Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2359–2368.

  • Liu, P., Zhang, H., Wei, L., & Zuo, W. (2019). Multi-level wavelet convolutional neural networks. IEEE Access, 7, 74973–74985.

    Article  Google Scholar 

  • Liu, Z., Wu, B., Luo, W., Yang, X., Liu, W., & Cheng, KT. (2018). Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. In: Proceedings of the European conference on computer vision (ECCV).

  • Liu, Z., Shen, Z., Savvides, M., & Cheng, KT. (2020b). Reactnet: Towards precise binary neural network with generalized activation functions. arXiv preprint arXiv:2003.03488,

  • Ma, Y., Xiong, H., Hu, Z., & Ma, L. (2019). Efficient super resolution using binarized neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW).

  • Maas, AL., Hannun, AY., & Ng, AY. (2013). Rectifier nonlinearities improve neural network acoustic models. In: Proceeding ICML, vol 30, p. 3.

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. ICCV, IEEE, 2, 416–423.

    Google Scholar 

  • Qin, H., Gong, R., Liu, X., Wei, Z., Yu, F., & Song, J. (2020). Ir-net: Forward and backward information retention for highly accurate binary neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. In: Proceedings of the European conference on computer vision (ECCV).

  • Rasti, P., Uiboupin, T., Escalera, S., & Anbarjafari, G. (2016). Convolutional neural network super resolution for face recognition in surveillance monitoring. In: International conference on articulated motion and deformable objects.

  • Timofte, R., Lee, KM., Wang, X,, Tian, Y., Ke, Y., Zhang, Y., Wu, S., Chao, D., Liang, L., & Yu, Q. (2017). Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  • Wang, Y., Lu, Y., & Blankevoort, T. (2020). Differentiable joint pruning and quantization for hardware efficiency. In: Proceedings of the European conference on computer vision (ECCV), Springer, vol 12374, pp 259–277

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

  • Xin, J., Li, J., Jiang, X., Wang, N., Huang, H., & Gao, X. (2020a). Wavelet-based dual recursive network for image super-resolution. IEEE Transactions on Neural Networks and Learning Systems.

  • Xin, J., Wang, N., Jiang, X., Li, J., Huang, H., & Gao, X. (2020b). Binarized neural network for single image super resolution. In: Proceedings of the European conference on computer vision (ECCV).

  • Xu, Z., & Cheung, RC. (2019). Accurate and compact convolutional neural networks with trained binarization. arXiv preprint arXiv:1909.11366

  • Zeyde, R., Elad, M., & Protter, M. (2012). On single image scale-up using sparse-representations. In: Proceedings of the international conference on curves and surfaces.

  • Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018a). Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV).

  • Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018b). Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant 2018AAA0103202; in part by the National Natural Science Foundation of China under Grants 62206211, 62036007, U22A2096, 62176195, 62176198 and 62106184; in part by the Technology Innovation Leading Program of Shaanxi under Grant 2022QFY01-15; in part by the Natural Science Basic Research Plan in Shaanxi Province of China under Grant 2022JQ-682; in part by Open Research Projects of Zhejiang Lab under Grant 2021KG0AB01 and in part by the Fundamental Research Funds for the Central Universities under Grant QTZX23042, XJS220119.

Author information

Authors and Affiliations

  1. State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, 710071, China

    Jingwei Xin, Nannan Wang & Xinrui Jiang

  2. State Key Laboratory of Integrated Services Networks, School of Electronic Engineering, Xidian University, Xi’an, 710071, China

    Jie Li

  3. Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Xinbo Gao

Authors
  1. Jingwei Xin
    View author publications

    Search author on:PubMed Google Scholar

  2. Nannan Wang
    View author publications

    Search author on:PubMed Google Scholar

  3. Xinrui Jiang
    View author publications

    Search author on:PubMed Google Scholar

  4. Jie Li
    View author publications

    Search author on:PubMed Google Scholar

  5. Xinbo Gao
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Nannan Wang.

Additional information

Communicated by Shaodi You.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xin, J., Wang, N., Jiang, X. et al. Advanced Binary Neural Network for Single Image Super Resolution. Int J Comput Vis 131, 1808–1824 (2023). https://doi.org/10.1007/s11263-023-01789-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-023-01789-8

Keywords