+
Skip to main content
Springer Nature Link
Log in

Classification of Interventional Radiology Reports into Technique Categories with a Fine-Tuned Large Language Model

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

The aim of this study is to develop a fine-tuned large language model that classifies interventional radiology reports into technique categories and to compare its performance with readers. This retrospective study included 3198 patients (1758 males and 1440 females; age, 62.8 ± 16.8 years) who underwent interventional radiology from January 2018 to July 2024. Training, validation, and test datasets involved 2292, 250, and 656 patients, respectively. Input data involved texts in clinical indication, imaging diagnosis, and image-finding sections of interventional radiology reports. Manually classified technique categories (15 categories in total) were utilized as reference data. Fine-tuning of the Bidirectional Encoder Representations model was performed using training and validation datasets. This process was repeated 15 times due to the randomness of the learning process. The best-performed model, which showed the highest accuracy among 15 trials, was selected to further evaluate its performance in the independent test dataset. The report classification involved one radiologist (reader 1) and two radiology residents (readers 2 and 3). The accuracy and macrosensitivity (average of each category’s sensitivity) of the best-performed model in the validation dataset were 0.996 and 0.994, respectively. For the test dataset, the accuracy/macrosensitivity were 0.988/0.980, 0.986/0.977, 0.989/0.979, and 0.988/0.980 in the best model, reader 1, reader 2, and reader 3, respectively. The model required 0.178 s required for classification per patient, which was 17.5–19.9 times faster than readers. In conclusion, fine-tuned large language model classified interventional radiology reports into technique categories with high accuracy similar to readers within a remarkably shorter time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data Availability

The datasets generated and/or analyzed during the current study are not publicly available due to patients' confidentiality.

References

  1. Reig M, Forner A, Rimola J et al (2022) BCLC strategy for prognosis prediction and treatment recommendation: The 2022 update. J Hepatol 76(3):681-693. https://doi.org/10.1016/j.jhep.2021.11.018.

    Article  PubMed  Google Scholar 

  2. Sugawara S, Sone M, Sakamoto N et al (2023) Guidelines for central venous port placement and management (Abridged Translation of the Japanese Version). Interv Radiol (Higashimatsuyama) 8(2):105-117. https://doi.org/10.22575/interventionalradiology.2022-0015.

    Article  PubMed  Google Scholar 

  3. Horattas MC, Trupiano J, Hopkins S, Pasini D, Martino C, Murty A (2001) Changing concepts in long-term central venous access: catheter selection and cost savings. Am J Infect Control 29(1):32-40. https://doi.org/10.1067/mic.2001.111536.

    Article  CAS  PubMed  Google Scholar 

  4. Yu Q, Funaki B, Ahmed O (2024) Twenty years of embolization for acute lower gastrointestinal bleeding: a meta-analysis of rebleeding and ischaemia rates. Br J Radiol 97(1157):920-932. https://doi.org/10.1093/bjr/tqae037.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Jablonska B, Mrowiec S (2024) Endovascular treatment of hepatic artery pseudoaneurysm after pancreaticoduodenectomy: a literature review. Life (Basel) 14(8)https://doi.org/10.3390/life14080920.

    Article  PubMed  Google Scholar 

  6. Awwad A, Dhillon PS, Ramjas G, Habib SB, Al-Obaydi W (2018) Trans-arterial embolisation (TAE) in haemorrhagic pelvic injury: review of management and mid-term outcome of a major trauma centre. CVIR Endovasc 1(1):32. https://doi.org/10.1186/s42155-018-0031-3.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Fernandez MG, Coutinho de Carvalho SF, Martins BA et al (2024) Uterine artery embolization versus hysterectomy in postpartum hemorrhage: a systematic review with meta-analysis. J Endovasc Ther:15266028241252730. https://doi.org/10.1177/15266028241252730.

  8. Kanagawa H, Mima S, Kouyama H, Gotoh K, Uchida T, Okuda K (1996) Treatment of gastric fundal varices by balloon-occluded retrograde transvenous obliteration. J Gastroenterol Hepatol 11(1):51-58. https://doi.org/10.1111/j.1440-1746.1996.tb00010.x.

    Article  CAS  PubMed  Google Scholar 

  9. Brookmeyer CE, Bhatt S, Fishman EK, Sheth S (2022) Multimodality imaging after liver transplant: top 10 important complications. Radiographics 42(3):702-721. https://doi.org/10.1148/rg.210108.

    Article  PubMed  Google Scholar 

  10. Thornburg B, Katariya N, Riaz A et al (2017) Interventional radiology in the management of the liver transplant patient. Liver Transpl 23(10):1328-1341. https://doi.org/10.1002/lt.24828.

    Article  PubMed  Google Scholar 

  11. Almansour H, Li N, Murphy MC, Healy GM (2023) Interventional radiology training: international variations. Radiology 308(1):e230040. https://doi.org/10.1148/radiol.230040.

    Article  PubMed  Google Scholar 

  12. Kachura JR (2023) Rules for Interventional Radiology. Can Assoc Radiol J 74(1):172-179. https://doi.org/10.1177/08465371221121338.

    Article  PubMed  Google Scholar 

  13. Chng SY, Tern PJW, Kan MRX, Cheng LTE (2023) Automated labelling of radiology reports using natural language processing: comparison of traditional and newer methods. Health Care Sci 2(2):120-128. https://doi.org/10.1002/hcs2.40.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Yasaka K, Abe O (2018) Deep learning and artificial intelligence in radiology: current applications and future directions. PLoS Med 15(11):e1002707. https://doi.org/10.1371/journal.pmed.1002707.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Chartrand G, Cheng PM, Vorontsov E et al (2017) Deep learning: a primer for radiologists. Radiographics 37(7):2113-2131. https://doi.org/10.1148/rg.2017170077.

    Article  PubMed  Google Scholar 

  16. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O (2018) Deep learning with convolutional neural network in radiology. Jpn J Radiol 36(4):257-272. https://doi.org/10.1007/s11604-018-0726-3.

    Article  PubMed  Google Scholar 

  17. Kiryu S, Akai H, Yasaka K et al (2023) Clinical impact of deep learning reconstruction in MRI. Radiographics 43(6):e220133. https://doi.org/10.1148/rg.220133.

    Article  PubMed  Google Scholar 

  18. Yasaka K, Kanzawa J, Nakaya M et al (2024) Super-resolution deep learning reconstruction for 3D Brain MR imaging: improvement of cranial nerve depiction and interobserver agreement in evaluations of neurovascular conflict. Acad Radiol https://doi.org/10.1016/j.acra.2024.06.010.

    Article  PubMed  Google Scholar 

  19. Tajima T, Akai H, Sugawara H et al (2021) Breath-hold 3D magnetic resonance cholangiopancreatography at 1.5 T using a deep learning-based noise-reduction approach: Comparison with the conventional respiratory-triggered technique. Eur J Radiol 144:109994. https://doi.org/10.1016/j.ejrad.2021.109994.

    Article  PubMed  Google Scholar 

  20. Yasaka K, Uehara S, Kato S et al (2024) Super-resolution deep learning reconstruction cervical spine 1.5T MRI: improved interobserver agreement in evaluations of neuroforaminal stenosis compared to conventional deep learning reconstruction. J Imaging Inform Med https://doi.org/10.1007/s10278-024-01112-y.

  21. Kiryu S, Yasaka K, Akai H et al (2019) Deep learning to differentiate parkinsonian disorders separately using single midsagittal MR imaging: a proof of concept study. Eur Radiol https://doi.org/10.1007/s00330-019-06327-0.

    Article  PubMed  Google Scholar 

  22. Yasaka K, Akai H, Abe O, Kiryu S (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286(3):887-896. https://doi.org/10.1148/radiol.2017170706.

    Article  PubMed  Google Scholar 

  23. Hamada T, Yasaka K, Nakai Y et al (2024) Computed tomography-based prediction of pancreatitis following biliary metal stent placement with the convolutional neural network. Endosc Int Open 12(6):E772-E780. https://doi.org/10.1055/a-2298-0147.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Yasaka K, Kamagata K, Ogawa T et al (2021) Parkinson's disease: deep learning with a parameter-weighted structural connectome matrix for diagnosis and neural circuit disorder investigation. Neuroradiology https://doi.org/10.1007/s00234-021-02648-4.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Bobba PS, Sailer A, Pruneski JA et al (2023) Natural language processing in radiology: Clinical applications and future directions. Clin Imaging 97:55-61. https://doi.org/10.1016/j.clinimag.2023.02.014.

    Article  PubMed  Google Scholar 

  26. Lopez-Ubeda P, Martin-Noguerol T, Escartin J, Luna A (2024) Role of natural language processing in automatic detection of unexpected findings in radiology reports: a comparative study of RoBERTa, CNN, and ChatGPT. Acad Radiol https://doi.org/10.1016/j.acra.2024.07.057.

    Article  PubMed  Google Scholar 

  27. Stade EC, Stirman SW, Ungar LH et al (2024) Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation. Npj Ment Health Res 3(1):12. https://doi.org/10.1038/s44184-024-00056-z.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Can E, Uller W, Vogt K et al (2024) Large language models for simplified interventional radiology reports: a comparative analysis. Acad Radiol https://doi.org/10.1016/j.acra.2024.09.041.

    Article  PubMed  Google Scholar 

  29. Glielmo P, Fusco S, Gitto S et al (2024) Artificial intelligence in interventional radiology: state of the art. Eur Radiol Exp 8(1):62. https://doi.org/10.1186/s41747-024-00452-2.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Gorenstein L, Konen E, Green M, Klang E (2024) Bidirectional encoder representations from transformers in radiology: a systematic review of natural language processing applications. J Am Coll Radiol 21(6):914-941. https://doi.org/10.1016/j.jacr.2024.01.012.

    Article  PubMed  Google Scholar 

  31. Mukherjee P, Hou B, Lanfredi RB, Summers RM (2023) Feasibility of using the privacy-preserving large language model vicuna for labeling radiology reports. Radiology 309(1):e231147. https://doi.org/10.1148/radiol.231147.

    Article  PubMed  Google Scholar 

  32. Kanemaru N, Yasaka K, Fujita N, Kanzawa J, Abe O (2024) The fine-tuned large language model for extracting the progressive bone metastasis from unstructured radiology reports. J Imaging Inform Med https://doi.org/10.1007/s10278-024-01242-3.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Yasaka K, Kanzawa J, Kanemaru N, Koshino S, Abe O (2024) Fine-tuned large language model for extracting patients on pretreatment for lung cancer from a picture archiving and communication system based on radiological reports. J Imaging Inform Med https://doi.org/10.1007/s10278-024-01186-8.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kanzawa J, Yasaka K, Fujita N, Fujiwara S, Abe O (2024) Automated classification of brain MRI reports using fine-tuned large language models. Neuroradiology https://doi.org/10.1007/s00234-024-03427-7.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Nakamura Y, Hanaoka S, Nomura Y et al (2021) Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers. BMC Med Inform Decis Mak 21(1):262. https://doi.org/10.1186/s12911-021-01623-6.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hu D, Zhang H, Li S, Wang Y, Wu N, Lu X (2021) Automatic extraction of lung cancer staging information from computed tomography reports: deep learning approach. JMIR Med Inform 9(7):e27955. https://doi.org/10.2196/27955.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Walston SL, Seki H, Takita H et al (2024) Data set terminology of deep learning in medicine: a historical review and recommendation. Jpn J Radiol 42(10):1100-1109. https://doi.org/10.1007/s11604-024-01608-1.

    Article  PubMed  Google Scholar 

  38. Su TH, Hsu SJ, Kao JH (2022) Paradigm shift in the treatment options of hepatocellular carcinoma. Liver Int 42(9):2067-2079. https://doi.org/10.1111/liv.15052.

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This work was not funded.

Author information

Authors and Affiliations

  1. Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8655, Japan

    Koichiro Yasaka, Takuto Nomura, Jun Kamohara, Hiroshi Hirakawa, Takatoshi Kubo & Osamu Abe

  2. Department of Radiology, International University of Health and Welfare Narita Hospital, 852 Hatakeda, Narita, Chiba, 286-0124, Japan

    Shigeru Kiryu

Authors
  1. Koichiro Yasaka
    View author publications

    Search author on:PubMed Google Scholar

  2. Takuto Nomura
    View author publications

    Search author on:PubMed Google Scholar

  3. Jun Kamohara
    View author publications

    Search author on:PubMed Google Scholar

  4. Hiroshi Hirakawa
    View author publications

    Search author on:PubMed Google Scholar

  5. Takatoshi Kubo
    View author publications

    Search author on:PubMed Google Scholar

  6. Shigeru Kiryu
    View author publications

    Search author on:PubMed Google Scholar

  7. Osamu Abe
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Koichiro Yasaka.

Ethics declarations

Ethics Approval

This retrospective study was approved by our research ethics committee.

Informed Consent

The requirement for informed consent was waived due to the retrospective nature of this study.

Conflict of Interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yasaka, K., Nomura, T., Kamohara, J. et al. Classification of Interventional Radiology Reports into Technique Categories with a Fine-Tuned Large Language Model. J Digit Imaging. Inform. med. 38, 3366–3374 (2025). https://doi.org/10.1007/s10278-024-01370-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s10278-024-01370-w

Keywords

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载