Introduction

Idiopathic pulmonary fibrosis (IPF) is a rare and dangerous disease with high mortality1. IPF is characterized by irreversible fibrotic remodeling of the bronchial system without an identifiable cause, ultimately leading to insufficient gas exchange or heart failure2,3. In addition to pulmonary function tests and microscopic parenchymal assessment, CT imaging has an important role in the diagnosis and quantification of disease extent. Prior to physiological assessment and imaging, progressive exertional dyspnea is often misdiagnosed, which may lead to delayed diagnosis of IPF4. The greater extent of fibrotic tissue changes at the time of delayed diagnosis leads to a worse prognosis5.

Reliable quantification of fibrotic lung alterations by radiological imaging is increasingly demanded for the diagnosis and monitoring of the progression of pulmonary fibrosis, especially in IPF6,7. However, visual quantifications of fibrotic lung disease are subjective, which can lead to inter-reader differences. Furthermore, assessment requires expertise in pulmonary imaging, however the availability of qualified radiologists/pulmonologists may be limited in some regions7,8,9. The recently introduced subtype of progressive fibrosis in interstitial lung (PF-ILD) is defined by the presence of at least 10% fibrosis that shows signs of progressive fibrotic alterations and worsening of clinical symptoms8. Inter-reader differences in visual assessment of pulmonary fibrosis can therefore lead to misinterpretations that influence therapy decisions. Automated objective pulmonary fibrosis quantification tools that ensure general reproducibility and traceability of results, are cost-effective and widely available, are therefore of high interest.

Recent studies suggest that automated software-based quantifications offer benefits over purely visual assessment of fibrotic alterations in diffuse lung disease7,10,11,12. In principle, CT pulmonary parenchymal analysis tools can vary in complexity ranging from simple binary discrimination of lung tissue in normal or pathological up to sophisticated analyses of parenchymal changes by distinguishing multiple characteristic lung texture features (i.e. ground glass, reticulation, honeycombing). As most of these changes found in pulmonary fibrosis alter pulmonary opacity, these sophisticated analyses might not be necessary compared to a simple opacity-based, binary differentiation of healthy and diseased lung parenchyma. Thus, it was the aim of this study to compare the diagnostic and prognostic value of a simpler opacity-based quantification of pulmonary fibrosis to a more sophisticated multi-texture lung analysis algorithm.

Methods

This retrospective study was performed in accordance with the Declaration of Helsinki and approved by the institutional review board with a waiver of written informed consent. Patients with biopsy or imaging-based diagnosis of idiopathic pulmonary fibrosis that received CT scans between 2012 and 2020 were included. Patients were excluded from this study, if CT scans were not available in a dedicated lung kernel, if slice thickness was > 1.5 mm, if scans were acquired with intravenous contrast and if extensive motion artifacts were present in the respective scans. All patients with image-based diagnosis presented a usual interstitial pneumonia pattern and were otherwise unremarkable in interstitial lung disease workup13. Details regarding patient selection are listed in Fig. 1.

Fig. 1
figure 1

Cohort selection flow chart. The retrospective study included patients with biopsy or imaging-based diagnosis of idiopathic pulmonary fibrosis who received high-resolution and non-contrast CT scans.

Full size image

Quantifications for assessing pulmonary fibrosis were performed using two software tools:

  1. 1.

    Opacity based quantification of fibrosis: For determining OFL, a CT-based pneumonia analysis tool was used, which identifies voxels with opacified lung (including reticulation, ground glass and consolidation) in % for the entire lung14. Additionally, severity of pulmonary affection is estimated with a lung severity score (LSS) that assesses local severity for each lobe based on the percentage of affected pulmonary parenchyma (0: <1%, 1: ≤25%, 2: ≤50%, 3: ≤75%). LSS is calculated by accumulation of all local scores15. The tool is currently a research only prototype and included in the SyngoVia software (Siemens Healthineers).

  2. 2.

    Multi-texture based quantification of fibrosis: For calculation of MTFL a dedicated lung-texture analysis tool was applied that identifies a variety of sub-features of fibrotic alteration that are typical or at least predominant in patients with IPF16. MTFL was defined as the combined amount of ground glass, honeycombing and reticular abnormalities. The computer-aided lung informatics for pathology evaluation and ratings (CALIPER) tool is part of the Imbio lung texture analysis software (Imbio) and was created by the Biomedical Imaging Resource Laboratory of the Mayo Clinic, Minnesota.

Additionally, duration of data extraction necessary for analysis, image analysis and manual corrections of analysis (if possible) were assessed for the two tools.

Statistical analysis

Estimation of lung fibrosis by OFL and LSS by the Pneumonia tool required some manual interactions. Analyses were performed by two radiologists with 11 (D.K.) and 5 years (D.C.) of experience in chest imaging. To assess inter-reader reproducibility intra-class correlation coefficient (ICC) with 95% confidence intervals (CI) was determined. Estimation of lung fibrosis by MTFL was a fully automated process without option of manual corrections and was conducted by (D.K.). Datasets were anonymized and randomly presented to the readers preventing direct correlation between the two analysis tools. To further avoid bias, assessment of OFL was performed first and subsequently, after an interval of 2 weeks, MTFL was determined.

Statistical data analysis was performed in Python v3.9.12. Correlation and mean differences between OFL and MTFL were assessed with linear regression analysis with Pearson correlation (r) and Bland-Altman analysis using Scikit-learn v1.1.2 and SciPy v1.9.017,18. Differences in OFL and MTFL were assessed with Mann-Whitney U-test. Also, regression analysis was used to investigate correlation between OFL, LSS and MTFL with spirometry parameters for the subset of patients for whom these information were available. These parameters were percent predicted total lung capacity (TLC), percent predicted vital capacity (VC), percent predicted forced expiratory volume in 1 s (FEV1), diffusing capacity of the lungs for carbon monoxide (DLCO), partial pressure of oxygen (PO2) and carbon dioxide (PCO2). Correction for multiple comparison of 19 regression analysis was conducted by Bonferroni method. Statistical significance was defined as P-value < 0.05.

The prognostic value of OFL, LSS and MTFL for 3-year survival was investigated in multivariable Cox-proportional-hazards (CPH) models together with basic clinical attributes (sex, age) and with TLC, and separately with sex, age and VC. We aimed for a low number of variables in CPH analysis to mitigate effects of overfitting. Correction for multiple comparison in three CPH models was also conducted by Bonferroni method. CPH analysis was performed in SPSS Statistics v27.0. Additionally, Kaplan-Meier analysis was conducted for OFL, LSS and MTFL with log rank test between patient groups separated by median using lifelines v0.28.019.

Results

A total of 45 patients (mean age 74 ± 9 years, 37 male) with biopsy (12/45) or imaging-based (33/45) proven IPF were included in the study. Exemplary images of fibrotic quantification are depicted in Fig. 2. Analysis of OFL and LSS using the opacity-based Pneumonia tool took an average of 611.75 ± 60.8s, analysis of MTFL is automated and required 312.61 ± 25.19s (P < 0.001). Inter-observer variability analysis showed very high agreement (ICC: OFL 0.91 [CI: 0.84–0.95], LSS 0.92 [CI: 0.86–0.96])20 for manual refinement of OFL segmentation in 24 of 45 cases.

Fig. 2
figure 2

Exemplary results of the multi-texture lung analysis tool and the opacity-based lung analysis tool. CT in coronal view of a 62 year old patient with idiopathic pulmonary fibrosis (A), analyzed by the opacity-based Pneumonia tool (Siemens Healthineers), the red lines represent the segmented fibrotic areas (B) with additional 3D reconstruction (C). (D) Analysis with the multi-texture lung analysis tool (CALIPER, Biomedical imaging resource laboratory, Mayo Clinic) with the yellow areas representing fibrotic altered tissue.

Full size image

Median with inter quartile ranges of OFL, LSS and MTFL, as well as general information on patient characteristics can be found in Table 1. The quantifications of OFL and MTFL did not differ (median and interquartile range: OFL 29% [20-38%], MTFL 31% [19-45%]; P = 0.44). Also, high correlation was observed between OFL and MTFL (r = 0.93, p < 0.01). Table 2 shows the results of the regression analysis between OFL, LSS and MTFL and the subsets of patients for which the spirometry parameters TLC, VC, FEV1, DLCO, PO2 and PCO2 were available. For OFL (P = 0.03) and LSS (P = 0.05) significant correlation to VC was observed. No significant correlation was found MTFL after Bonferroni correction. Figure 3 illustrates linear regression analysis of OFL and MTFL to spirometry parameters. Bland-Altman analysis (Fig. 4) reveals a bias of -3.78% and 95% limits of agreement of -21.87–14.31% with OFL being lower than MTFL.

Table 1 General characteristics and imaging quantifications of the patients included in the study.
Full size table
Table 2 Linear regression analyses.
Full size table
Fig. 3
figure 3

Linear regression analyses. Illustration of regression analysis between the opacity-based fibrotic lung (OFL) and the multi-texture fibrotic lung (MTFL) quantification, as well as regression analysis of OFL and MTFL with spirometry parameters with Pearson correlation coefficient (r) bigger than 0.25. 95% confidence intervals are indicated by light-colored areas. TLC percent predicted total lung capacity, VC percent predicted vital capacity and DLCO diffusing capacity of the lungs for carbon monoxide.

Full size image
Fig. 4
figure 4

Bland-Altman analysis. The opacity-based fibrotic lung (OFL) quantifications was compared to the multi-texture fibrotic lung (MTFL) measurements.

Full size image

As shown in Table 3, only the image-based markers (OFL P = 0.03, LSS P < 0.01 and MTFL P = 0.02) showed significant association with the survival of the patients in multivariable CPH analyses with age, sex and TLC. The CPH model, which solely included patients with reported VC, resulted in neither the image-based markers (OFL P = 1, LSS P = 0.73, and MTFL P = 1) nor sex, age, or VC showing a significant association with patient hazard. In univariable CPH analysis, VC (0.97 [0.93-1.00], P = 0.07) showed a greater, yet also non-significant, association with patient hazard compared to TLC (1.00 [0.97-1.02], P = 0.82).

Table 3 Multivariable cox-proportional-hazards analysis.
Full size table

Figure 5 shows the Kaplan-Meier analysis for patient groups separated by median OFL, LSS and MTFL with log rank test of significant differences. For OFL (P = 0.03) and LSS (P = 0.03) significant differences were observed but not for MTFL (P = 0.17).

Fig. 5
figure 5

Kaplan-Meier survival plots. Patients were separated by the median of the opacity-based fibrotic lung (OFL) quantification, by the lung severity score (LSS) and by the multi-texture fibrotic lung (MTFL) quantification. Differences between groups were tested by log rank test.

Full size image

Discussion

In this study, an opacity-based analysis tool, initially developed for identification of inflammatory infiltrates, was compared to an established multi-texture lung analysis tool. The quantitative assessment of pulmonary fibrosis and prognosis estimation in patients with IPF was evaluated.

Employing automated pulmonary analysis for quantification of pulmonary fibrosis holds several advantages. Unlike visual approximation, which is prone to inter-observer variability and subjectivity, automated tools offer the prospect of a more objective and standardized approach. Nevertheless, visual approximation remains the current clinical standard for quantification of fibrosis.

The clinical need for affordable and simple tools to quantify fibrosis in CT imaging is paramount, especially since introduction of progressive pulmonary fibrosis as an independent fibrotic entity21. While specialized tools may offer more targeted analysis for certain pathologies and use cases, such as the discrimination of sub-textures of fibrotic lung, tools with more widespread and general application to multiple use cases might be preferable, particularly in resource-constrained environments.

Initially designed to quantify inflammatory infiltrates, specifically pneumonia, the opacity-based Pneumonia tool might have potential to the assessment of various pulmonary abnormalities characterized by reduced aeration or ‘consolidations’.

The results of the current study show that OFL quantification of this tool did not significantly differ and showed high correlation in linear regression analysis to MTFL measured by the sophisticated CALIPER lung texture analysis tool. Bland-Altman analysis reveal a small systematic difference of OFL to MTFL of -3.78%, with OFL being lower. As already indicated, the CALIPER tool was trained explicitly for the detection of fibrotic subtypes, and the Pneumonia tool was originally conceptualized to quantify the extent of pulmonary infectious disease. Therefore, slight differences between both tools were to be expected. Considering that results from visual approximation of fibrotic extent data severely differ from automated approaches22, as recently reported, smaller differences between the two automatic approaches are most likely acceptable for clinical purposes.

For quantification of the amount of MTFL, the combined quantification of lung with ground glass, reticulation and honeycombing features was calculated to allow for comparability with the quantified OFL, which represents an opacity-based, binary differentiation of diseased and to non-diseased lung. Recently it has been shown that the combination of fibrotic features in a single score appears to have superior correlation with clinical features such as lung function and outcome23, thus disadvantages are not to be expected in outcome prediction when solely employing binary opacity detection by the Pneumonia tool without discrimination of various fibrotic features. Nevertheless, fibrotic subtype classification and estimation of inflammation are easier when using a tool that allows for discrimination of fibrotic features24. Inter-reader reproducibility was only assessed for the Pneumonia tool, as the CALIPER multi-texture lung analysis tool does not enable modification of segmentations. Reproducibility of the Pneumonia tool for opacity detection was higher compared to recently reported results of visual assessment22.

However, manual interaction resulted in opacity detection by the Pneumonia tool taking considerably longer. Although not examined in the current study, differences in duration of analysis will be even longer when compared to the visual approximation, as data extraction, computation time and manual corrections are not necessary when visually approximating the extent of fibrosis. A faster analysis could improve the clinical applicability in the future.

No substantial difference in association of OFL and MTFL with the prognosis of 42 IPF patients was observed in multivariable CPH models along age, sex and TLC. When including VC instead of TLC, no parameter was significantly associated with the hazard of the 36 included patients. One might speculate that these insignificant results are due to the lower number of included patients with available VC, due to collinearities between VC and the imaging parameters, or due to FVC being more tightly linked to outcomes than TLC25. Furthermore, Kaplan-Meier analysis indicate association of OFL to patient survival that could not be seen for MTFL.

Previously works demonstrated correlation between densitometric-based analysis in pulmonary fibrosis and forced VC / DLCO26. The current results add results on correlation amount of opacified lung measured by the Pneumonia tool to VC and to MTFL determined by the CALIPER software.

The lower P-values in multivariable CPH analysis might indicate that the local assessment of opacities (i.e. the extent of pulmonary abnormalities in all lung lobes) of the LSS created by the Pneumonia tool could be beneficial for predicting patient prognosis compared to the total amount of OFL. The CALIPER lung texture analysis tool does not allow for calculation of a regional score of fibrosis severity. Overall, there are no significant differences in usability between the tools, except for the fact that the opacity-based Pneumonia tool allows for adjustments. This manual adjustments tend to increase duration of analyses, however the prospect of visual review of segmentation with possible correction leads to a better comprehensibility of results. Although the binary identification of OFL included lung tissue sections with several hyperdense fibrotic features (reticulations, ground-glass, consolidations), some partially hyperlucent alterations (such as honey-combing) were only partially detected in some cases. This resulted in manual adjustment of OFL segmentations in more than half of the cases and could explain the systematic difference revealed in Bland-Altman analysis. In the current study only IPF cases were included; currently, no data is available that indicates whether the results extends to other pulmonary fibrotic diseases.

Conclusion

For the current cohort of IPF patients, no substantial difference in quantification of disease, correlation to spirometry parameters and association with patient prognosis was observed between the dedicated and established CALIPER multi-texture lung analysis tool and the investigated opacity-based Pneumonia tool. Although requiring manual interaction, the opacity-based Pneumonia tool could have potential for more general quantification of pulmonary abnormalities, besides the identification of inflammatory infiltrates it was developed for.