Abstract
Existing deep blind image super-resolution (SR) methods usually depend on the paired training data, which is difficult to obtain in real applications. In this paper, we propose an effective unpaired learning method to solve the blind image SR problem. The proposed method first estimates the blur kernel and intermediate high-resolution (HR) image from the low-resolution (LR) input image in a self-supervised learning manner. With the estimated blur kernels and intermediate HR images, we develop an effective variational model based on the image formation of SR to improve the quality of the intermediate HR images. To better learn the LR-to-HR mapping, we further develop an exemplar distillation module that is able to explore useful information from exemplar HR images to constrain the deep models learned from the self-supervised learning module for the final HR image restoration. We jointly train the proposed method and show that it performs favorably against state-of-the-art methods on benchmark datasets and real-world images.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
26 February 2024
A Correction to this paper has been published: https://doi.org/10.1007/s11263-023-01980-x
References
Bell-Kligler, S., Shocher, A., & Irani, M. (2019). Blind super-resolution kernel estimation using an internal-gan. In NeurIPS (pp. 284–293).
Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, M. L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC (pp. 1–10).
Bulat, A., Yang, J., & Tzimiropoulos, G. (2018). To learn image super-resolution, use a GAN to learn how to do image degradation first. In ECCV (pp. 187–202).
Chen, X., Zhang, J., Xu, C., Wang, Y., Wang, C., & Liu, Y. (2023). Better ”CMOS” produces clearer images: Learning space-variant blur estimation for blind image super-resolution. In CVPR (pp. 1651–1661).
Chen, X., Zhang, J., Xu, C., Wang, Y., Wang, C., & Liu, Y. (2023). Better "CMOS" produces clearer images: Learning space-variant blur estimation for blind image super-resolution. In CVPR (pp. 1651–1661).
Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. In ECCV (pp. 184–199).
Efrat, N., Glasner, D., Apartsin, A., Nadler, B., & Levin, A. (2013). Accurate blur models vs. image priors in single image super-resolution. In ICCV (pp. 2832–2839).
Gao, Q., Zhao, Y., Li, G., & Tong, T. (2018). Image super-resolution using knowledge distillation. In ACCV (pp. 527–541).
Gu, J., Lu, H., Zuo, W., & Dong, C. (2019). Blind super-resolution with iterative kernel correction. In CVPR (pp. 1604–1613).
Haris, M., Shakhnarovich, G., & Ukita, N. (2018). Deep back-projection networks for super-resolution. In CVPR (pp. 1664–1673 ).
He, Y., Yap, K. H., Chen, L., & Chau, L. P. (2009). A soft MAP framework for blind super-resolution image reconstruction. IVC, 27(4), 364–373.
He, Z., Dai, T., Lu, J., Jiang, Y., & Xia, S. T. (2020). FAKD: Feature-affinity based knowledge distillation for efficient image super-resolution. In ICIP (pp. 518–522).
Huang, J. B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In CVPR (pp. 5197–5206).
Hui, Z., Li, J., Wang, X., & Gao, X. (2021). Learning the non-differentiable optimization for blind super-resolution. In CVPR (pp. 2093–2102).
Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In CVPR (pp. 1646–1654).
Kim, S. Y., Sim, H., & Kim, M. (2021). Koalanet: Blind super-resolution using kernel-oriented adaptive local adjustment. In CVPR (pp. 10,611–10,620).
Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In ICLR.
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., & Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In CVPR (pp. 105–114).
Lee, W., Lee, J., Kim, D., & Ham, B. (2020). Learning with privileged information for efficient image super-resolution. In ECCV (pp. 465–482).
Li, X., Zuo, W., & Loy, C. C. (2023). Learning generative structure prior for blind text image super-resolution. In CVPR (pp. 10,103–10,113).
Liang, J., Sun, G., Zhang, K., Gool, L. V., & Timofte, R. (2021). Mutual affine network for spatially variant kernel estimation in blind image super-resolution. In ICCV (pp. 4076–4085).
Liang, J., Zhang, K., Gu, S., Van Gool, L., & Timofte, R. (2021). Flow-based kernel prior with application to blind super-resolution. In CVPR (pp. 10,601–10,610).
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017). Enhanced deep residual networks for single image super-resolution. In CVPR (pp. 1132–1140).
Luo, Z., Huang, H., Yu, L., Li, Y., Fan, H., & Liu, S. (2022). Deep constrained least squares for blind image super-resolution. In CVPR (pp. 17,621–17,631).
Luo, Z., Huang, Y., Li, S., Wang, L., & Tan, T. (2020). Unfolding the alternating optimization for blind super resolution. In NeurIPS.
Maeda, S. (2020). Unpaired image super-resolution using pseudo-supervision. In CVPR (pp. 288–297).
Martin, D. R., Fowlkes, C. C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV (pp. 416–425).
Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., & Aizawa, K. (2017). Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, 76(20), 21811–21838.
Michaeli, T., & Irani, M. (2013). Nonparametric blind super-resolution. In ICCV (pp. 945–952).
Pan, J., Sun, D., Pfister, H., & Yang, M. H. (2018). Deblurring images via dark channel prior. IEEE TPAMI, 40(10), 2315–2328.
Protter, M., Elad, M., Takeda, H., & Milanfar, P. (2009). Generalizing the nonlocal-means to super-resolution reconstruction. IEEE TIP, 18(1), 36–51.
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. InCVPR (pp. 1874–1883).
Shocher, A., Cohen, N., & Irani, M. (2018). “Zero-shot” super-resolution using deep internal learning. In CVPR (pp. 3118–3126).
Soh, J. W., Cho, S., & Cho, N. I. (2020). Meta-transfer learning for zero-shot super-resolution. In CVPR (pp. 3513–3522).
Timofte, R., Agustsson, E., Gool, L. V., Yang, M. H., Zhang, L., et al. (2017). NTIRE 2017 challenge on single image super-resolution: Methods and results. In CVPR Workshops (pp. 1110–1121).
Wang, L., Wang, Y., Dong, X., Xu, Q., Yang, J., An, W., & Guo, Y. (2021). Unsupervised degradation representation learning for blind super-resolution. In CVPR (pp. 10,581–10,590)
Wang, Q., Tang, X., & Shum, H. (2005). Patch based blind image super resolution. In ICCV (pp. 709–716 ).
Wang, Z., Chen, J., & Hoi, S. C. H. (2021). Deep learning for image super-resolution: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10), 3365–3387.
Wu, X., Liu, M., Cao, Y., Ren, D., & Zuo, W. (2020). Unpaired learning of deep image denoising. In ECCV.
Yue, Z., Zhao, Q., Xie, J., Zhang, L., Meng, D., & Wong, K. K. (2022). Blind image super-resolution with elaborate degradation modeling on noise and kernel. In CVPR (pp. 2118–2128).
Zeyde, R., Elad, M., & Protter, M. (2010). On single image scale-up using sparse-representations. In ICCS (pp. 711–730).
Zhang, K., Gool, L. V., & Timofte, R. (2020). Deep unfolding network for image super-resolution. In CVPR (pp. 3214–3223).
Zhang, K., Liang, J., Gool, L. V., & Timofte, R. (2021). Designing a practical degradation model for deep blind image super-resolution. In ICCV (pp. 4771–4780).
Zhang, K., Zuo, W., & Zhang, L. (2018). Learning a single convolutional super-resolution network for multiple degradations. In CVPR (pp. 3262–3271).
Zhang, Y., Chen, H., Chen, X., Deng, Y., Xu, C., & Wang, Y. (2021). Data-free knowledge distillation for image super-resolution. In CVPR (pp. 7852–7861).
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In ECCV (pp. 294–310).
Zhao, N., Wei, Q., Basarab, A., Dobigeon, N., Kouame, D., & Tourneret, J. Y. (2016). Fast single image super-resolution using a new analytical solution for l2–l2 problems. IEEE TIP, 25(8), 3683-3697.
Zhou, R., & Süsstrunk, S. (2019). Kernel modeling super-resolution on real low-resolution images. In ICCV (pp. 2433–2443).
Acknowledgements
This work has been supported in part by the National Natural Science Foundation of China under Grants 62272233, 61922043, 61925204, U22B2049 and 62332010, and the Fundamental Research Funds for the Central Universities under Grants 30922010910 and 30920041109.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Chongyi Li.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original article has been corrected to update figure citation.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dong, J., Bai, H., Tang, J. et al. Deep Unpaired Blind Image Super-Resolution Using Self-supervised Learning and Exemplar Distillation. Int J Comput Vis 133, 8253–8266 (2025). https://doi.org/10.1007/s11263-023-01957-w
Received:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s11263-023-01957-w