这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

Making the Invisible Visible: Toward High-Quality Terahertz Tomographic Imaging via Physics-Guided Restoration

  • S.I. : Physics-Based Vision meets Deep Learning
  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Terahertz (THz) tomographic imaging has recently attracted significant attention thanks to its non-invasive, non-destructive, non-ionizing, material-classification, and ultra-fast nature for object exploration and inspection. However, its strong water absorption nature and low noise tolerance lead to undesired blurs and distortions of reconstructed THz images. The diffraction-limited THz signals highly constrain the performances of existing restoration methods. To address the problem, we propose a novel multi-view Subspace-Attention-guided Restoration Network (SARNet) that fuses multi-view and multi-spectral features of THz images for effective image restoration and 3D tomographic reconstruction. To this end, SARNet uses multi-scale branches to extract intra-view spatio-spectral amplitude and phase features and fuse them via shared subspace projection and self-attention guidance. We then perform inter-view fusion to further improve the restoration of individual views by leveraging the redundancies between neighboring views. Here, we experimentally construct a THz time-domain spectroscopy (THz-TDS) system covering a broad frequency range from 0.1 to 4 THz for building up a temporal/spectral/spatial/material THz database of hidden 3D objects. Complementary to a quantitative evaluation, we demonstrate the effectiveness of our SARNet model on 3D THz tomographic reconstruction applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. Project site: https://github.com/wtnthu/THz_Tomography.

  2. Dataset site: https://github.com/wtnthu/THz_Data.

References

  • Abbas, A., Abdelsamea, M., & Gaber, M. M. (2021). Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Applied Intelligence, 51(2), 854–864.

    Article  Google Scholar 

  • Abraham, E., Younus, A., Delagnes, T. C., & Mounaix, P. (2010). Non-invasive investigation of art paintings by terahertz imaging. Applied Physics A, 100(3), 585–590.

    Article  Google Scholar 

  • Born, M., & Wolf, E. (2013). Principles of optics: Electromagnetic theory of propagation, interference and diffraction of light.

  • Bowman, T., Chavez, T., Khan, K., Wu, J., Chakraborty, A., Rajaram, N., Bailey, K., & El-Shenawee, M. (2018). Pulsed terahertz imaging of breast cancer in freshly excised murine tumors. Journal of Biomedical Optics, 23(2), 026004.

    Article  Google Scholar 

  • Calvin, Y., Shuting, F., Yiwen, S., & Emma, P.-M. (2012). The potential of terahertz imaging for cancer diagnosis: A review of investigations to date. Quantitative Imaging in Medicine and Surgery, 2(1), 33.

    Google Scholar 

  • Cao, J., Li, Y., Zhang, K., & Van Gool, L. (2021). Video super-resolution transformer. arXiv preprint arXiv:2106.06847.

  • Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. In Proceedings of European conference on computer vision (pp. 213–229). Springer.

  • Chapman, D., Homlinson, W., Johnston, R., Washburn, D., Pisano, E., Gmür, N., Zhong, Z., Menk, R., Arfelli, F., & Sayers, D. (1997). Diffraction enhanced X-ray imaging. Journal Physics in Medicine & Biology, 42(11), 2015.

    Article  Google Scholar 

  • Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021). Pre-trained image processing transformer. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition (pp. 12299–12310).

  • Cheng, S., Wang, Y., Huang, H., Liu, D., Fan, H., & Liu, S. (2021). NBNet: Noise basis learning for image denoising with subspace projection. In Proceedings of IEEE/CVF international conference on computer vision and pattern recognition (pp. 4896–4906).

  • Clarke, L., Velthuizen, R., Camacho, M., Heine, J., Vaidyanathan, M., Hall, L., Thatcher, R., & Silbiger, M. (1995). MRI segmentation: Methods and applications. Magnetic Resonance Imaging, 13(3), 343–368.

    Article  Google Scholar 

  • Cloetens, P., Barrett, R., Baruchel, J., Guigay, J.-P., & Schlenker, M. (1996). Phase objects in synchrotron radiation hard X-ray imaging. Journal of Physics D: Applied Physics, 29(1), 133.

    Article  Google Scholar 

  • de Gonzalez, A. B., & Darby, S. (2004). Risk of cancer from diagnostic X-rays: Estimates for the UK and 14 other countries. The Lancet, 363(9406), 345–351.

    Article  Google Scholar 

  • Dorney, T. D., Baraniuk, R. G., & Mittleman, D. M. (2001). Material parameter estimation with terahertz time-domain spectroscopy. Journal of the Optical Society of America A, 18(7), 1562–1571.

    Article  Google Scholar 

  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., & Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

  • Fitzgerald, R. (2000). Phase-sensitive X-ray imaging. Physics Today, 53(7), 23–26.

    Article  Google Scholar 

  • Fukunaga, K. (2016). THz technology applied to cultural heritage in practice

  • Geladi, P., Burger, J., & Lestander, T. (2004). Hyperspectral imaging: Calibration problems and solutions. Chemometrics and Intelligent Laboratory Systems, 72(2), 209–217.

    Article  Google Scholar 

  • Hack, E., & Zolliker, P. (2014). Terahertz holography for imaging amplitude and phase objects. Optics Express, 22(13), 16079–16086.

    Article  Google Scholar 

  • He, K., Zhang, X., Ren, S., Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of IEEE/CVF international conference on computer vision (pp. 1026–1034).

  • Hung, Y.-C., & Yang, S.-H. (2019a). Kernel size characterization for deep learning terahertz tomography (pp. 1–2).

  • Hung, Y.-C., & Yang, S.-H. (2019b). Terahertz deep learning computed tomography. In Proceedings of international infrared, millimeter, and terahertz waves (pp. 1–2). IEEE.

  • Janke, C., Först, M., Nagel, M., Kurz, H., & Bartels, A. (2005). Asynchronous optical sampling for high-speed characterization of integrated resonant terahertz sensors. Optics Letters, 30(11), 1405–1407.

    Article  Google Scholar 

  • Jansen, C., Wietzke, S., Peters, O., Scheller, M., Vieweg, N., Salhi, M., Krumbholz, N., Jördens, C., Hochrein, T., & Koch, M. (2010). Terahertz imaging: Applications and perspectives. Applied Optics, 49(19), 48–57.

    Article  Google Scholar 

  • Jin, K. H., McCann, M. T., Froustey, E., & Unser, M. (2017). Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9), 4509–4522.

    Article  MathSciNet  MATH  Google Scholar 

  • Kak, A. C. (2001). Algorithms for reconstruction with nondiffracting sources. Principles of Computerized Tomographic Imaging, 49–112.

  • Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2011). Application of NIR hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104(3), 332–340.

    Article  Google Scholar 

  • Kang, E., Min, J., & Ye, J. C. (2017). A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Journal of Medical physics, 44(10), 360–375.

    Article  Google Scholar 

  • Kawase, K., Ogawa, Y., Watanabe, Y., & Inoue, H. (2003). Non-destructive terahertz imaging of illicit drugs using spectral fingerprints. Optics Express, 11(20), 2549–2554.

    Article  Google Scholar 

  • Kim, J., Lim, H., Ahn, S.C., Lee, S. (2018). RGBD camera based material recognition via surface roughness estimation. In: Proceedings of IEEE Winter Conference Applied Computer Vision (pp. 1963–1971).

  • Li, X., & Jarrahi, M. (2020). A 63-pixel plasmonic photoconductive terahertz focal-plane array. In Proceedings of IEEE/MTT-S international microwave symposium (IMS) (pp. 91–94).

  • Liu, F., Jang, H., Kijowski, R., Bradshaw, T., & McMillan, A. B. (2018). Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology, 286(2), 676–684.

    Article  Google Scholar 

  • Ljubenovic, M., Bazrafkan, S., Beenhouwer, J. D., & Sijbers, J. (2020). CNN-based deblurring of terahertz images (pp. 323–330).

  • Mao, X., Shen, C., & Yang, Y.-B. (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proceedings of the advances in neural information processing systems (pp. 2802–2810).

  • Meyer, C. D. (2000). Matrix analysis and applied linear algebra. SIAM.

    Book  Google Scholar 

  • Mittleman, D. M. (2018). Twenty years of terahertz imaging. Optics Express, 26(8), 9417–9431.

    Article  Google Scholar 

  • Mittleman, D., Gupta, M., Neelamani, R., Baraniuk, R., Rudd, J., & Koch, M. (1999). Recent advances in terahertz imaging. Applied Physics B, 68(6), 1085–1094.

    Article  Google Scholar 

  • Nunes-Pereira, E., Peixoto, H., Teixeira, J., & Santos, J. (2020). Polarization-coded material classification in automotive LIDAR aiming at safer autonomous driving implementations. Applied Optics, 59(8), 2530–2540.

    Article  Google Scholar 

  • Ozdemir, A., & Polat, K. (2020). Deep learning applications for hyperspectral imaging: A systematic review. Journal of the Institute of Electronics and Computer, 2(1), 39–56.

    Article  Google Scholar 

  • Peterson, J., Paerels, F., Kaastra, J., Arnaud, M., Reiprich, T., Fabian, A., Mushotzky, R., Jernigan, J., & Sakelliou, I. (2001). X-ray imaging-spectroscopy of Abell 1835. Journal of Astronomy & Astrophysics, 365(1), 104–109.

    Article  Google Scholar 

  • Popescu, D. C., & Ellicar, A. D. (2010). Point spread function estimation for a terahertz imaging system. EURASIP Journal on Advances in Signal Processing, 2010(1), 575817.

    Article  Google Scholar 

  • Popescu, D.C., Hellicar, A., & Li, Y. (2009). Phantom-based point spread function estimation for terahertz imaging system (pp. 629–639).

  • Qin, X., Wang, X., Bai, Y., Xie, X., & Jia, H. (2020). FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, pp. 11908–11915).

  • Recur, B., Guillet, J.-P., Manek-Hönninger, I., Delagnes, J.-C., Benharbone, W., Desbarats, P., Domenger, J.-P., Canioni, L., & Mounaix, P. (2012). Propagation beam consideration for 3D THz computed tomography. Optics Express, 20(6), 5817–5829.

    Article  Google Scholar 

  • Recur, B., Younus, A., Salort, S., Mounaix, P., Chassagne, B., Desbarats, P., Caumes, J., & Abraham, E. (2011). Investigation on reconstruction methods applied to 3D terahertz computed tomography. Optics Express, 19(6), 5105–5117.

    Article  Google Scholar 

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (pp. 234–241).

  • Rotermund, H. H., Engel, W., Jakubith, S., Von Oertzen, A., & Ertl, G. (1991). Methods and application of UV photoelectron microscopy in heterogenous catalysis. Ultramicroscopy, 36(1–3), 164–172.

    Article  Google Scholar 

  • Round, A. R., Wilkinson, S. J., Hall, C. J., Rogers, K. D., Glatter, O., Wess, T., & Ellis, I. O. (2005). A preliminary study of breast cancer diagnosis using laboratory based small angle x-ray scattering. Physics in Medicine & Biology, 50(17), 4159.

    Article  Google Scholar 

  • Saeedkia, D. (2013). Handbook of terahertz technology for imaging, sensing and communications (pp. 542–578). Cambridge: Woodhead Publishing.

    Book  Google Scholar 

  • Sakdinawat, A., & Attwood, D. (2010). Nanoscale X-ray imaging. Nature Photonics, 4(12), 840.

    Article  Google Scholar 

  • Schultz, R., Nielsen, T., Zavaleta, R. J., Wyatt, R., & Garner, H. (2001). Hyperspectral imaging: A novel approach for microscopic analysis. Cytometry, 43(4), 239–247.

    Article  Google Scholar 

  • Su, W.-T., Hung, Y.-C., Yu, P.-J., Lin, C.-W., & Yang, S.-H. (2023). Physics-guided terahertz computational imaging: A tutorial on sate-of-the-art techniques. IEEE Signal Processing Magazine, 40(2), 32–45.

    Article  Google Scholar 

  • Su, W.-T., Hung, Y.-C., Yu, P.-J., Yang, S.-H., & Lin, C.-W. (2022). Seeing through a black box: Toward high-quality terahertz tomographic imaging via multi-scale spatio-spectral image fusion. In Proceedings of the European conference on computer vision.

  • Tuan, T. M., Fujita, H., Dey, N., Ashour, A. S., Ngoc, T. N., & Chu, D.-T. (2018). Dental diagnosis from X-ray images: An expert system based on fuzzy computing. Biomedical Signal Processing and Control, 39, 64–73.

    Article  Google Scholar 

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., & Polosukhin, I. (2017) Attention is all you need. In Proceedings of advances in neural information processing systems (vol. 30).

  • Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022). Uformer: A general U-shaped transformer for image restoration. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition (pp. 17683–17693).

  • Wong, T. M., Kahl, M., & Bolívar, P.H., Kolb, A. (2019). Computational image enhancement for frequency modulated continuous wave (FMCW) THz image. Journal of Infrared, Millimeter, and Terahertz Waves, 40(7), 775–800.

  • Wong, T. M., Kahl, M., Haring-Bolívar, P., Kolb, A., & Möller, M. (2019). Training auto-encoder-based optimizers for terahertz image reconstruction (pp. 93–106).

  • Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020). Visual transformers: Token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.03677.

  • Xie, H., Yao, H., Zhang, S. P., Zhou, S. C., & Sun, W. X. (2020). Pix2Vox++: Multi-scale context-aware 3D object reconstruction from single and multiple images. International Journal of Computer Vision, 128(12), 2919–2935.

    Article  Google Scholar 

  • Xie, X. (2008). A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electronic Letters on Computer Vision and Image Analysis, 1–22

  • Yujiri, L., Shoucri, M., & Moffa, P. (2003). Passive millimeter wave imaging. IEEE Microwave Magazine, 4(3), 39–50.

    Article  Google Scholar 

  • Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019). Self-attention generative adversarial networks. In Proceedings of international conference on machine learning (pp. 7354–7363).

  • Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, K., Zuo, W. M., & Zhang, L. (2018). FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9), 4608–4622.

    Article  MathSciNet  Google Scholar 

  • Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2020). Residual dense network for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence.

  • Zhou, S., Zhang, J., Pan, J., Xie, H., Zuo, W., & Ren, J. (2019). Spatio-temporal filter adaptive network for video deblurring. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 2482–2491).

  • Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, R. B., & Rosen, M. S. (2018). Image reconstruction by domain-transform manifold learning. Nature, 555(7697), 487–492.

    Article  Google Scholar 

Download references

Acknowledgements

This work was financially supported in part by the National Science and Technology Council (NSTC), Taiwan, under Grants 111-2221-E-007-046-MY3, 111-2634-F-002-023, and 110-2636-E-007-017.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Tsing Hua University, Kuang-Fu Road, Hsinchu, 30048, Taiwan

    Weng-Tai Su, Yi-Chun Hung, Po-Jen Yu, Shang-Hua Yang & Chia-Wen Lin

Authors
  1. Weng-Tai Su
    View author publications

    Search author on:PubMed Google Scholar

  2. Yi-Chun Hung
    View author publications

    Search author on:PubMed Google Scholar

  3. Po-Jen Yu
    View author publications

    Search author on:PubMed Google Scholar

  4. Shang-Hua Yang
    View author publications

    Search author on:PubMed Google Scholar

  5. Chia-Wen Lin
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Chia-Wen Lin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 804 KB)

Supplementary file 2 (mp4 982 KB)

Supplementary file 3 (mp4 625 KB)

Supplementary file 4 (mp4 1601 KB)

Supplementary file 5 (mp4 1057 KB)

Supplementary file 6 (mp4 715 KB)

Supplementary file 7 (mp4 397 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, WT., Hung, YC., Yu, PJ. et al. Making the Invisible Visible: Toward High-Quality Terahertz Tomographic Imaging via Physics-Guided Restoration. Int J Comput Vis 131, 2388–2407 (2023). https://doi.org/10.1007/s11263-023-01812-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-023-01812-y

Keywords