这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

Reblurring-Guided Single Image Defocus Deblurring: A Learning Framework with Misaligned Training Pairs

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

For single image defocus deblurring, acquiring well-aligned training pairs (or training triplets), i.e., a defocus blurry image, an all-in-focus sharp image (and a defocus blur map), is a challenging task for developing effective deblurring models. Existing image defocus deblurring methods typically rely on training data collected by specialized imaging equipment, with the assumption that these pairs or triplets are perfectly aligned. However, in practical scenarios involving the collection of real-world data, direct acquisition of training triplets is infeasible, and training pairs inevitably encounter spatial misalignment issues. In this work, we introduce a reblurring-guided learning framework for single image defocus deblurring, enabling the learning of a deblurring network even with misaligned training pairs. By reconstructing spatially variant isotropic blur kernels, our reblurring module ensures spatial consistency between the deblurred image, the reblurred image and the input blurry image, thereby addressing the misalignment issue while effectively extracting sharp textures from the all-in-focus sharp image. Moreover, spatially variant blur can be derived from the reblurring module, and serve as pseudo supervision for defocus blur map during training, interestingly transforming training pairs into training triplets. To leverage this pseudo supervision, we propose a lightweight defocus blur estimator coupled with a fusion block, which enhances deblurring performance through seamless integration with state-of-the-art deblurring networks. Additionally, we have collected a new dataset for single image defocus deblurring (SDD) with typical misalignments, which not only validates our proposed method but also serves as a benchmark for future research. The effectiveness of our method is validated by notable improvements in both quantitative metrics and visual quality across several datasets with real-world defocus blurry images, including DPDD, RealDOF, DED, and our SDD. The source code and dataset are available at https://github.com/ssscrystal/Reblurring-guided-JDRL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. https://github.com/xytmhy/DID-ANet-Defocus-Deblurring

  2. https://github.com/ssscrystal/Reblurring-guided-JDRL

References

  • Abuolaim, A., Brown, M. S. (2020). Defocus deblurring using dual-pixel data. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, (pp. 111-126). Springer.

  • Abuolaim, A., Delbracio, M., Kelly, D., Brown, MS., Milanfar, P. (2021). Learning to reduce defocus blur by realistically modeling dual-pixel data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, (pp 2289–2298).

  • Ai, Y., Huang, H., Zhou, X., Wang, J., He, R. (2024). Multimodal prompt perceiver: Empower adaptiveness generalizability and fidelity for all-in-one image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 25432–25444).

  • Blau, Y., Michaeli, T. (2018). The perception-distortion tradeoff. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 6228–6237).

  • Carreira, J., Sminchisescu, C. (2010). Constrained parametric min-cuts for automatic object segmentation. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (pp 3241–3248). IEEE.

  • Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M. (1994). Two deterministic half-quadratic regularization algorithms for computed imaging. In Proceedings of 1st International Conference on Image Processing, vol 2, (pp 168–172). IEEE.

  • Chen, H., Gu, J., Gallo, O., Liu, M., Veeraraghavan, A., Kautz, J. (2018). Reblur2deblur: Deblurring videos via self-supervised learning. In 2018 IEEE International Conference on Computational Photography (ICCP), (pp 1–9). IEEE.

  • Chen, L., Tian, X., Xiong, S., Lei, Y., Ren., C. (2024). Unsupervised blind image deblurring based on self-enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 25691–25700).

  • DP, K., Ba, J. (2015). Adam: A method for stochastic optimization. In Proc. of the 3rd International Conference for Learning Representations (ICLR).

  • Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y. (2017). Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision, (pp 764–773).

  • Fish, D. A., Brinicombe, A. M., Pike, E. R., & Walker, J. G. (1995). Blind deconvolution by means of the richardson-lucy algorithm. JOSA A, 12(1), 58–65.

    Article  Google Scholar 

  • Goldstein, A., Fattal, R. (2012). Blur-kernel estimation from spectral irregularities. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12, (pp 622–635). Springer.

  • Gu, J., Cai, H., Chen, H., Ye, X., S, RJ., Dong, C. (2020). Pipal: a large-scale image quality assessment dataset for perceptual image restoration. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, (pp 633–651). Springer.

  • Hariharan, B., Arbeláez, P., Girshick, R., Malik, J. (2015). Hypercolumns for object segmentation and fine-grained localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 447–456).

  • He, K., Zhang, X., Ren, S., Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, (pp 1026–1034).

  • Krishnan, D., Fergus, R. (2009). Fast image deconvolution using hyper-laplacian priors. Advances in neural information processing systems, 22.

  • Lee, J., Son, H., Rim, J., Cho, S., Lee, S. (2021). Iterative filter adaptive network for single image defocus deblurring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 2034–2042).

  • Li, Y., Fan, Y., Xiang, X., Demandolx, D., Ranjan, R., Timofte, R., Van Gool, L. (2023). Efficient and explicit modelling of image hierarchies for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 18278–18289).

  • Li, Y., Hou, X., Koch, C., Rehg, JM., Yuille, AL. (2014) The secrets of salient object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 280–287).

  • Li, Yu., Ren, Dongwei, Shu, Xinya, & Zuo, Wangmeng. (2023). Learning single image defocus deblurring with misaligned training pairs. In Proceedings of the AAAI Conference on Artificial Intelligence, 37, 1495–1503.

    Article  Google Scholar 

  • Lin, J., Zhang, Z., Wei, Y., Ren, D., Jiang, D., Tian, Q., Zuo, W. (2024). Improving image restoration through removing degradations in textual representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 2866–2878).

  • Ma, Haoyu, Liu, Shaojun, Liao, Qingmin, Zhang, Juncheng, & Xue, Jing-Hao. (2022). Defocus image deblurring network with defocus map estimation as auxiliary task. IEEE Transactions on Image Processing, 31, 216–226.

    Article  Google Scholar 

  • Mao, X., Wang, J., Xie, X., Li, Q., Wang, Y. (2024). Loformer: Local frequency transformer for image deblurring. In ACM Multimedia 2024.

  • Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., Lee, KM. (2019). Ntire 2019 challenge on video deblurring and super-resolution: Dataset and study. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, (pp 0–0).

  • Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P. (2005). Light field photography with a hand-held plenoptic camera. PhD thesis, Stanford university.

  • Pang, L., Rui, X., Cui, L., Wang, H., Meng, D.., Cao, X. (2024). Hir-diff: Unsupervised hyperspectral image restoration via improved diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 3005–3014).

  • Papageorgiou, CP., Oren, M., Poggio, T. (1998). A general framework for object detection. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), (pp 555–562). IEEE.

  • Park, D., Lee, BH., Chun, SY. (2023). All-in-one image restoration for unknown degradations using adaptive discriminative filters for specific degradations. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (pp 5815–5824). IEEE.

  • Park, J., Tai, Y., Cho, D., Kweon, IS. (2017). A unified approach of multi-scale deep and hand-crafted features for defocus estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp 1736–1745).

  • Potlapalli, V., Zamir, SW., Khan, SH., Khan, FS. (2024). Promptir: Prompting for all-in-one image restoration. Advances in Neural Information Processing Systems, 36.

  • Quan, Y., Wu, Z., Ji, H. (2023). Neumann network with recursive kernels for single image defocus deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 5754–5763).

  • Quan, Y., Yao, X., Ji, H. (2023).Single image defocus deblurring via implicit neural inverse kernels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, (pp 12600–12610).

  • Ronneberger, O., Fischer, P., Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention, (pp 234–241). Springer.

  • Shi, J., Xu, L., Jia, J. (2015). Just noticeable defocus blur detection and estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp 657–665).

  • Son, H., Lee, J., Cho, S., Lee, S. (2021). Single image defocus deblurring using kernel-sharing parallel atrous convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, (pp 2642–2650).

  • Sun, L., Cho, S., Wang, J., Hays, J. (2013). Edge-based blur kernel estimation using patch priors. In IEEE international conference on computational photography (ICCP), (pp 1–8). IEEE.

  • Sun, D., Yang, X., Liu, MY., Kautz, J. (2018). Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 8934–8943).

  • Tang, X., Zhao, X., Liu, J., Wang, J., Miao, Y., Zeng, T. (2023). Uncertainty-aware unsupervised image deblurring with deep residual prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 9883–9892).

  • Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H. (2022). Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 17683–17693).

  • Wang, Zhou, Bovik, Alan C., Sheikh, Hamid R., & Simoncelli, Eero P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600–612.

    Article  Google Scholar 

  • Xia, Z., Pan, X., Song, S., Li, LE., Huang, G. (2022). Vision transformer with deformable attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 4794–4803).

  • Xu, L., Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part I 11, (pp 157–170). Springer.

  • Yang, H., Pan, L., Yang, Y., Hartley, R., Liu, M. (2024). Ldp: Language-driven dual-pixel image defocus deblurring network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 24078–24087).

  • Ye, Q., Suganuma, M., Okatani, T. (2023). Accurate single-image defocus deblurring based on improved integration with defocus map estimation. In 2023 IEEE International Conference on Image Processing (ICIP), (pp 750–754). IEEE.

  • Zamir, SW., Arora, A., Khan, S., Hayat, M., Khan, FS., Yang, MH., Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 14821–14831).

  • Zamir, SW., Arora, A., Khan, S., Hayat, M., Khan, FS., Yang, M. (2022). Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 5728–5739).

  • Zhang, H., Dai, Y., Li, H., Koniusz, P. (2019). Deep stacked hierarchical multi-patch network for image deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 5978–5986).

  • Zhang, R., Isola, P., Efros, AA., Shechtman, E., Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 586–595).

  • Zhang, K., Luo, W., Zhong, Y., Ma, L., Stenger, B., Liu, W., Li, H. (2020). Deblurring by realistic blurring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (pp 2737–2746).

  • Zhang, Y., Zheng, P., Yan, W., Fang, C., Cheng, SS. (2024). A unified framework for microscopy defocus deblur with multi-pyramid transformer and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp 11125–11136).

  • Zhang, Xin, Song, Yingze, Song, Tingting, Yang, Degang, Ye, Yichen, Zhou, Jie, & Zhang, Liming. (2024). Ldconv: Linear deformable convolution for improving convolutional neural networks. Image and Vision Computing, 149, Article 105190.

    Article  Google Scholar 

  • Zhao, W., Hu, G., Wei, F., Wang, H., He, Y., Lu, H. (2023). Attacking defocus detection with blur-aware transformation for defocus deblurring. IEEE Transactions on Multimedia.

  • Zhao, Zhong-Qiu., Zheng, Peng, Shou-tao, Xu., & Xindong, Wu. (2019). Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems, 30(11), 3212–3232.

    Article  Google Scholar 

  • Zou, Zhengxia, Chen, Keyan, Shi, Zhenwei, Guo, Yuhong, & Ye, Jieping. (2023). Object detection in 20 years: A survey. Proceedings of the IEEE, 111(3), 257–276.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (62172127 and U22B2035), and the Natural Science Foundation of Heilongjiang Province (YQ2022F004).

Author information

Author notes

  1. Dongwei Ren and Xinya Shu have equally contributed to this work.

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, China

    Dongwei Ren

  2. Faculty of Computing, Harbin Institute of Technology, Harbin, China

    Xinya Shu, Yu Li, Xiaohe Wu & Wangmeng Zuo

  3. School of Electrical and Information Engineering, Tianjin University, Tianjin, China

    Jin Li

Authors
  1. Dongwei Ren
    View author publications

    Search author on:PubMed Google Scholar

  2. Xinya Shu
    View author publications

    Search author on:PubMed Google Scholar

  3. Yu Li
    View author publications

    Search author on:PubMed Google Scholar

  4. Xiaohe Wu
    View author publications

    Search author on:PubMed Google Scholar

  5. Jin Li
    View author publications

    Search author on:PubMed Google Scholar

  6. Wangmeng Zuo
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Dongwei Ren.

Additional information

Communicated by Rynson W.H. Lau.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, D., Shu, X., Li, Y. et al. Reblurring-Guided Single Image Defocus Deblurring: A Learning Framework with Misaligned Training Pairs. Int J Comput Vis 133, 6953–6970 (2025). https://doi.org/10.1007/s11263-025-02522-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-025-02522-3

Keywords

Profiles

  1. Dongwei Ren View author profile