Conditional diagnostic accuracy according to inflammation status and age for diagnosing tuberculous effusion

Abstract

Background

Tuberculous effusion varies from lymphocyte-dominant to neutrophilic effusion according to inflammation status. The criteria of adenosine deaminase (ADA) and lymphocyte/neutrophil (L/N) ratio have yet not been evaluated across different disease conditions.

Methods

Patients who conducted pleural fluid analysis from 2009 to 2019 at Asan Medical Center were included. Criteria (ADA of 50 and L/N ratio of 0.75) were evaluated by quantile subgroups according to age, C-reactive protein (CRP), white blood cell (WBC), and lactate dehydrogenase (LD) by the Monte Carlo simulation method to diagnose tuberculosis. The model for the ADA and L/N ratio was evaluated by AUROC.

Results

Among the 2,918 reviewed cases, 2034 were included with 229 (11.26%) tuberculosis cases. The mean baseline ADA AUROC was 0.88 across all patients. Increased CRP and WBC showed high proportions of neutrophilic tuberculous effusion, with low sensitivity of approximately 45% and 33% in the fifth WBC and CRP groups, respectively. The AUROC of the models decreased with the increase in WBC and CRP groups (ADA model: 0.69 [the top quantile WBC group], 0.74 [the top quantile CRP group]). The AUROC of the models did not show a trend according to the increase in LD and age.

Conclusion

Inflammatory status affects the diagnostic metrics for tuberculous effusion due to the progression of tuberculous effusion. Clinicians should consider the low accuracy of tuberculous effusion criteria in high-inflammatory conditions when diagnosing tuberculosis.

Introduction

Tuberculous pleural effusion is the chronic accumulation of inflammatory fluid in the pleural space caused by the Mycobacterium tuberculosis infection of the pleura [1]. A definitive diagnosis is difficult because of the low sensitivity of bacterial culture methods (< 10%: acid-fast bacilli of pleural fluid, 20–30%: M. tuberculosis culture of the fluid) [2]. In obscure cases, closed needle pleural biopsy is conducted; however, this invasive method yields only a confirmation of 60–80% for tuberculosis pleurisy [3]. To avoid this invasive technique and facilitate diagnosis, pleural fluid analysis with adenosine deaminase (ADA) and lymphocyte-dominant features have been used. A pleural fluid ADA of 50 U/L cut-off has shown a diagnostic sensitivity of 95% and 89% specificity [3, 4]. Furthermore, a study including pleural effusion analysis as a case-control design reported that lymphocyte and ADA criteria had an AUROC of 0.974 and 58% and 99% sensitivity and specificity, respectively [5,6,7].

In a previous study, the deep learning model classified the etiology of pleural effusion based on laboratory results and showed the class probabilities [8]. The visualization map revealed that several tuberculosis pleurisy patients were misclassified as bacterial infections. Pleural effusion caused by a bacterial infection was thought to have a low ADA and a predominance of neutrophils, even though previous research demonstrated similarities between bacterial infection and tuberculous effusion in some cases.

Additionally, the diagnostic metrics differed between the ADA [9] and lymphocyte [4] criteria across several meta-analyses. These differences may be attributed to the random sampling deviation across the studies, but several factors concerning patient characteristics could explain the varying diagnostic performance. For example, patients in the older age group had a lower ADA than those in the younger age group [2]. Moreover, the pleural fluid laboratory data may vary between culture-positive or loculated tuberculous effusion, which is more neutrophilic and less lymphocytic in pleural fluid [10]. Given that neutrophilic tuberculous effusion is associated with severe inflammation, serum biomarkers representing inflammation such as lactate dehydrogenase (LD) [11], C-reactive protein (CRP), and white blood cells (WBCs) have been associated with the different stages of tuberculosis [12]. Although the frequency of neutrophilic tuberculous pleurisy is low [11], inflammatory conditions represented by high inflammatory serum biomarkers may increase the risk of misdiagnosis by using previous ADA and lymphocyte criteria. Nevertheless, the difference in diagnostic metrics according to inflammatory status and age has not been well evaluated in the diagnosis of tuberculous effusion.

This study evaluated the ADA and lymphocyte-based criteria for the diagnosis of tuberculous effusion and the different metrics according to age and inflammatory levels in an intermediate tuberculosis burden country, South Korea.

Methods

Clinical data

We retrospectively extracted the medical records of patients who underwent pleural effusion cell analysis from 2009 to 2019 at Asan Medical Center (Seoul, South Korea). Patient data were extracted from the in-house system and indexed by de-identifying encrypted patient identification numbers to maintain confidentiality [13, 14]. Laboratory data were extracted if their acquisition date was within 2 weeks of pleural cell count, including the following: blood chemistry, complete blood cell count, pleural fluid cell count, and pleural fluid chemistry. In cases with multiple laboratory results, the latest set before pleural fluid analysis was included for analysis. Patient selection from whole cases was randomly assigned to minimize the selection bias affecting the diagnostic metrics, and selected cases underwent chart review to identify the aetiology of effusion. The final diagnosis of pleural effusion was confirmed through manual chart review by two independent clinicians (DSJ, HJP). Patients meeting the following criteria were excluded: (1) multiple causes as judged by two clinicians, (2) no clear aetiology of the pleural effusion, or (3) pleural fluid cell analysis conducted for post-treatment follow-up only, not at initial diagnosis.

This study was approved by the ethics committee of Asan Medical Center and conducted per the Declaration of Helsinki. The requirement for informed consent was waived by the ethics committee of Asan Medical Center (approval number 2022 − 0455), given the retrospective nature of the study.

Definition of tuberculous effusion and neutrophil-dominant tuberculous effusion

Definitive tuberculous effusion was defined when Mycobacterium tuberculosis was cultured from the pleural fluid. As fewer than 50% of cases were culture-positive [15], “clinical tuberculous effusion” was defined by the improvement of pleural effusion on chest X-ray after anti-tuberculosis treatment plus one of the following conditions: (i) pleural effusion with a pulmonary tuberculosis lesion, (ii) pathologic findings of granuloma in the pleural biopsy, or (iii) tuberculosis suspected by imaging tests and tuberculosis drugs initiated according to the judgment of the treating clinician. The neutrophil-dominant tuberculous effusion was defined by neutrophils comprising more than 50% of the nucleated cells in the pleural fluid cell analysis [12].

Non-tuberculous pleural effusion

The alternative pleural effusion aetiology was used as a true negative label. The other aetiologies were categorized as “bacterial infection”, “malignancy”, “volume overload”, and “miscellaneous.” Bacterial infection was defined when bacterial culture was positive for pleural effusion or improved effusion after using adequate antibiotics, excluding other causes. Malignant effusion was defined as a malignant cell identified in the pleural fluid, or if pleural metastasis was suspected in the imaging study without evidence of other causes. Volume overload was defined when definitive causes such as huge ascites or heart failure were identified, and the effusion was resolved through volume control.

Subgroups by inflammatory markers

The inflammatory laboratory markers were selected from the routine laboratory check-up list in our cohort dataset. The blood inflammatory data were used to define the prior probability through inflammatory biomarkers before the ADA and L/N criteria. The WBCs, CRP, and LD were selected to represent the systemic inflammatory markers and used to define the subgroups. The dataset was divided equally into five quantiles (20% each) to evaluate the trend of diagnostic metrics including age and levels of inflammatory markers. Among the quantile groups, the distribution of neutrophil-dominant tuberculous and ADA was evaluated to identify the cause of the variation in the diagnostic results according to the level of inflammation and age.

The cut-off of ADA and lymphocytes/neutrophil ratio for diagnosis of tuberculous effusion

A previous meta-analysis demonstrated the various diagnostic cut-offs of ADA [9] and lymphocyte/neutrophil (L/N) ratio [16]. In this analysis, the major cut-off was defined as an ADA of 50 U/L and L/N ratio of 0.75 [3], which was used to evaluate the diagnostic metrics (sensitivity and specificity) in each quantile group. The lymphocyte ratio was calculated by the proportion of lymphocytes among the whole nucleated cells in the pleural fluid. When the neutrophil and lymphocyte numbers were zero, the ratio was zero.

Statistical analysis

The proportion of neutrophil-dominant tuberculous effusion in each quantile group and the aetiologies of effusion are presented in a bar plot. The group difference in the proportions was analyzed using the chi-square test. The distribution of ADA level across each quantile group according to tuberculous and non-tuberculous effusion is described in a box plot. A one-way ANOVA test was conducted to compare the ADA distribution across the five groups. The diagnostic metrics (sensitivity, specificity, and AUROC) were calculated based on true tuberculosis and others, including malignant effusion, bacterial infections, volume overload, and miscellaneous. Sensitivity and specificity were simulated through a random selection of cases, allowing replacement to have a dataset A of 1000 cases. Sensitivity and specificity were calculated among dataset A, representing each quantile group. Cases with missing data were omitted from the simulation analysis. The dataset was split randomly at 3:7 for training and testing to calculate the AUROC according to the ADA and L/N ratio. A generalized linear model using binomial function was used for model 1 using only the ADA and model 2 using both the ADA and L/N ratio. This process was repeated 1000 times per simulation, and the diagnostic metrics were presented as the median, interquartile box, and 1.5 times standard deviation in a box plot. Paired t-tests were performed to compare models 1 and 2 in every simulated five-quantile group. The statistical analysis was performed using R software version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Data description and baseline diagnostic performance of ADA and L/N ratio

In our clinical data warehouse, a total of 3,799 pleural effusion analyses were conducted between 2009 and 2019. Among these, 2,918 cases were reviewed by manual chart review, through which 884 cases were excluded according to the aforementioned criteria. Finally, 2,034 patients were included in this study, with a total of 229 tuberculosis cases (Fig. 1). The baseline characteristics of the demographic and laboratory values have been described according to the aetiology of pleural effusion (Table 1). Lactate dehydrogenase data were missing in 308 cases (15.1%), and CRP data were missing in 233 cases (11.4%). The remaining laboratory values had a mean missing data percentage of 2.76%. At an ADA cut-off of 50 U/L and L/N ratio of 0.75, the sensitivity and specificity were 72.49% and 95.79%, respectively. The AUROC of models 1(ADA alone) and 2 (ADA + L/N ratio) were 0.88 and 0.77, respectively.

Fig. 1

Flow chart of patient case inclusion. From the data warehouse, patients were randomly selected and a chart review was performed to identify the aetiology of pleural effusions. After excluding 884 patients for the above reasons, 2034 cases were analyzed

Table 1 Baseline characteristics according to the aetiology of pleural effusion

Proportion of neutrophil dominant tuberculous effusions and ADA distribution

All data were categorized according to inflammatory markers (WBC, CRP, and LD) and age. The trend of neutrophil-dominant proportion and ADA distribution by aetiology is presented in Fig. 2. Neutrophilic-dominant tuberculous effusion was more frequently observed as levels of WBC and CRP increased (p: <0.001, p: 0.060, respectively, Fig. 2-A, C). Additionally, the younger age group had a greater proportion of neutrophilic-dominant tuberculous effusion (p: 0.238, Fig. 2-G). However, LD was not significantly associated with the proportion of neutrophil-dominant tuberculous effusion (p: 0.902).

A higher WBC was associated with an increase in mean ADA and distribution of ADA in non-tuberculous effusion (p: <0.001, p: 0.140, respectively non-TB and TB, Fig. 2-B), which was similarly observed in the CRP group (p: <0.001, p: 0.049). The high mean and wider distribution of ADA reduced the diagnostic performance due to the poor discriminative value of ADA in tuberculous and non-tuberculous effusion. Thus, patients with higher WBC and CRP levels are expected to have lower diagnostic metrics. Changes in LD and age did not significantly impact the distribution of ADA (p: LD = 0.138 (non-TB), 0.262 (TB); age = 0.089 (non-TB), 0.176 (TB)).

Fig. 2

Neutrophil-dominant tuberculous effusion and ADA distribution according to quantile group. (A, C, E, G) The neutrophil-dominant tuberculous effusion (pleural neutrophil more than 50%) among total tuberculosis cases in each quantile group are depicted by bar plots. Higher WBC (A) and CRP (C) were associated with neutrophil-dominant tuberculosis, as well as younger age (B, D, F, H). Box plots depict the ADA distribution according to tuberculosis or non-tuberculosis of each quantile group. For clarity, the outliers are not shown in these box plots. With respect to the non-tuberculous ADA distribution, higher WBC (A) and CRP (C) were associated with increased ADA level and distribution

Simulation analysis of diagnostic metrics according to inflammatory markers and age

The diagnostic metrics when using the cut-off of ADA 50 U/L and L/N ratio of 0.75 according to inflammatory markers and age are depicted in Fig. 3. The fourth and fifth WBC groups had a lower sensitivity (median: 59% and 33%, respectively) compared with the other three groups (Fig. 3-A, Table 2). Additionally, the first CRP group (lowest) and fifth group (highest) exhibited low sensitivity (median: 56% and 45%, respectively) (Fig. 3-C). With respect to LD and age, all groups showed a sensitivity of more than 60% and no specific drop in sensitivity by the quantile group (Fig. 3-E, G).

Fig. 3

Diagnostic metrics according to quantile groups by inflammatory marker and age. (A, C, E, G) The sensitivity and specificity of the five quantile groups are depicted by box plots. The distribution of the sensitivity and specificity results from the Monte-Carlo simulation of random sampling in each group are displayed. The higher WBC and CRP were associated with low sensitivity (B, D, F, H). The AUROC distribution of each model using ADA (model 1) and ADA with L/N ratio (model 2) are presented as box plots. The higher WBC and CRP were associated with low AUROCs

Table 2 Summary statistics of sensitivity and specificity according to quantile group

The quantile groups were evaluated by a logistic model using ADA alone and ADA with L/N ratio according to inflammatory markers and age. The AUROCs of the models for diagnosing tuberculosis are depicted in Fig. 3. Overall, the AUROC of model 1 (ADA alone) outperformed that of model 2 (ADA and L/N ratio) (Fig. 3). The highest WBC group had comparatively low AUROCs of 0.69 (model 1) and 0.65 (model 2) (Fig. 3-B, Table 3). In the CRP group, the higher CRP group had lower AUROC in model 1, and the fifth CRP group had 0.74 in model 1 (Fig. 3-D, Table 3). The AUROC of model 2 in the first and fifth CRP groups also showed a comparatively low performance (median: 0.74 and 0.78, respectively) (median: 0.83‒0.84). In particular, the first CRP group exhibited a much lower performance in model 2 than in model 1 (0.74 vs. 0.95, respectively). The AUROC of model 1 did not differ significantly among the quantile groups in the LD and age quantile groups (Fig. 3-F, H). In model 2, the younger age group (first, second) had a lower AUROC than the older age groups (third to fifth).

Table 3 Summary statistics of AUROC according to quantile group by model 1 and model 2

Discussion

In this study, the diagnostic metrics for tuberculosis effusion differed according to inflammatory status. When the WBC was greater than 11.8 × 10³ /L or the CRP was higher than 11.9 mg/dL, the AUROCs of model 1 (ADA alone) and model 2 (ADA + L/N ratio) were lower than the other quantile groups, and the sensitivity was lower than 40% and 30%, respectively. High inflammatory conditions had a greater proportion of neutrophil-dominant tuberculous effusion and wider ADA distributions of non-tuberculous effusion, suggesting that the high inflammatory status of tuberculosis could easily be missed by the current ADA and L/N ratio cut-offs. Moreover, the younger age group had a lower AUROC in model 2 (ADA, L/N ratio) in line with the higher rate of neutrophil-dominant tuberculous effusion. However, there was no clear association with age in model 1 (ADA alone).

Tuberculous effusion is believed to be a delayed hypersensitivity reaction to the tuberculous protein [15], while fluid and inflammatory responses become more prominent in line with the increase in mycobacterial burden [10]. Culture-positive tuberculosis effusion tends to have fewer lymphocytes in the pleural fluid and a high level of CRP [17], suggesting that higher inflammatory effusion is more neutrophilic, affecting the diagnostic cut-off of ADA and L/N ratio. This study assumed that serum WBC and CRP could represent the inflammatory status of effusion. As the inflammation progressed, the neutrophilic-dominant tuberculous effusion increased in this study. Moreover, higher levels of inflammation, such as pleural thickening, were associated with increased ADA in non-tuberculous effusion in a previous study [18]. Our study supports the findings that the non-tuberculous effusion exhibits higher ADA distribution as inflammation progresses. Consequently, the diagnostic accuracy of ADA and the L/N ratio decreased at high inflammatory status, with the sensitivity lowering to 30% at an ADA of 50 and L/N ratio of 0.75. To improve sensitivity, we simulated various cut-offs such as an ADA of 35 to 70 and L/N ratio of 0.3 to 0.75. We observed that an L/N ratio of 0.3 increased sensitivity without decreasing specificity below 95% at high inflammatory status (sensitivity; fifth WBC: 59%, fifth CRP 69%) (Supplement Figs. 1–4).

Previous studies have demonstrated a difference in serum LD levels in culture-positive and culture-negative tuberculous effusion [14]. Although LD can be used to assess the inflammatory status of respiratory disease [19], it is also a well-known biomarker for lymphoma severity [20]. In our analysis, a higher LD level was not associated with neutrophil-dominant tuberculous effusion nor the ADA distribution of non-tuberculous effusion. Moreover, the increase in LD was not associated with a higher proportion of bacterial infection but with malignant effusion (Supplement 5). Representation of inflammation through LD was therefore more likely to be associated with non-infectious than infectious causes. Consequently, the discriminative performance was not associated with an increase in LD level.

A previous study reported that ADA decreased with increasing age in patients with tuberculous effusion [2], which may impact the diagnostic metrics. This study also noted a negative correlation between age and ADA level in tuberculosis (Supplement Fig. 6), which could affect the sensitivity of ADA-based criteria for tuberculous effusion. However, we also demonstrated that the AUROC of model 1 (ADA alone) did not significantly drop in the older age group, which could be attributed to the division of the groups by quantile age of the included patients, which was skewed to older age. Thus, the younger group included patients aged up to 55 years, which could devalue the observed trend in AUROC of model 1 according to age. Additionally, the younger age group exhibited a greater proportion of neutrophilic tuberculous effusion, the AUROC of model 2 (ADA + L/N ratio) was lower in the younger age group. However, in previous studies, the younger age was not associated with progressed tuberculous effusion in culture-positive [11] or loculated effusion [17]. These findings should be further evaluated in future studies.

There are some limitations to this study. First, despite including the details of all pleural fluid analyses conducted over the past 10 years, this only totaled 229 cases, which may not be enough to determine the subtype of tuberculous effusion. However, to our knowledge, this is the largest study of pleural effusion cases comparing the sublevel of patients included by random sampling methods. This method should be subject to the lowest inclusion bias, which could evaluate the conditional probability defined by the laboratory data. Second, this study did not consider the radiologic findings and clinical symptoms that could affect the conditional status. Our data collection method was auto-extraction from the data warehouse, so this type of data collection and annotation was not available. Third, in general, cases of microbiologically confirmed tuberculous pleural effusion are not common [21], Therefore, some argue that there may be a misdiagnosis of tuberculous effusion in this study. We defined tuberculous pleural effusion if there was suspicion based on clinical, laboratory, and radiological evidence, and then observed clinical improvement while maintaining appropriate anti-tuberculosis treatment for at least 3 months. Therefore, we believe the possibility of misdiagnosis is very low.

Conclusions

Inflammatory status defined by WBC and CRP affects the diagnostic performance of ADA and lymphocyte-to-neutrophil ratio criteria for tuberculous effusion. Clinicians should consider the false-negative cases of tuberculous effusion in high-inflammatory conditions and conditional differences in diagnostic performance.