Skip to main content
Download PDF

The PROgnostic ModEl for chronic lung disease (PRO-MEL): development and temporal validation

BMC Pulmonary Medicine volume 24, Article number: 429 (2024) Cite this article

Abstract

Background

Patients with chronic lung diseases (CLDs), defined as progressive and life-limiting respiratory conditions, experience a heavy symptom burden as the conditions become more advanced, but palliative referral rates are low and late. Prognostic tools can help clinicians identify CLD patients at high risk of deterioration for needs assessments and referral to palliative care. As current prognostic tools may not generalize well across all CLD conditions, we aim to develop and validate a general model to predict one-year mortality in patients presenting with any CLD.

Methods

A retrospective cohort study of patients with a CLD diagnosis at a public hospital from July 2016 to October 2017 was conducted. The outcome of interest was all-cause mortality within one-year of diagnosis. Potential prognostic factors were identified from reviews of prognostic studies in CLD, and data was extracted from electronic medical records. Missing data was imputed using multiple imputation by chained equations. Logistic regression models were developed using variable selection methods and validated in patients seen from January 2018 to December 2019. Discriminative ability, calibration and clinical usefulness of the model was assessed. Model coefficients and performance were pooled across all imputed datasets and reported.

Results

Of the 1000 patients, 122 (12.2%) died within one year. Patients had chronic obstructive pulmonary disease or emphysema (55%), bronchiectasis (38%), interstitial lung diseases (12%), or multiple diagnoses (6%). The model selected through forward stepwise variable selection had the highest AUC (0.77 (0.72–0.82)) and consisted of ten prognostic factors. The model AUC for the validation cohort was 0.75 (0.70, 0.81), and the calibration intercept and slope were − 0.14 (-0.54, 0.26) and 0.74 (0.53, 0.95) respectively. Classifying patients with a predicted risk of death exceeding 0.30 as high risk, the model would correctly identify 3 out 10 decedents and 9 of 10 survivors.

Conclusions

We developed and validated a prognostic model for one-year mortality in patients with CLD using routinely available administrative data. The model will support clinicians in identifying patients across various CLD etiologies who are at risk of deterioration for a basic palliative care assessment to identify unmet needs and trigger an early referral to palliative medicine.

Trial registration

Not applicable (retrospective study).

Peer Review reports

Introduction

Chronic lung disease (CLD), defined as a group of progressive and life-limiting respiratory conditions, was the third leading cause of death globally in 2017, accounting for 7% of all deaths [1]. Despite the differences in pathophysiology, clinical presentation and prognosis between specific respiratory conditions, majority of CLD patients near the end-of-life experience similar symptoms of dyspnea, cough, fatigue, and psychosocial distress, with a symptom burden comparable to lung cancer [2]. However, only 17–37% of patients with chronic obstructive pulmonary disease (COPD), and 14% of patients with interstitial lung diseases (ILDs) received palliative care for their symptoms and needs before death [3,4,5,6].

Difficulty in prognosticating a patient’s disease trajectory stands as one of many barriers to providing timely palliative care in CLD [7]. Despite the recent push to move away from a prognosis-centric referral decision process, [8, 9] prognostic tools can serve as surrogate markers to help non-palliative clinicians identify patients at high risk of short-term deterioration, prioritising them for holistic needs assessments and subsequent referral for integrated support [9, 10]. Moreover, embedding such tools within electronic health records to provide automated flagging of high-risk patients enables prognostication to be easily integrated into existing clinical workflows with minimal incremental workload to the clinicians [11].

However, the heterogeneity in patient characteristics and prognoses within CLD makes prognostication and timely referral to palliative care challenging. This is exacerbated by the existing prognostic tools developed for individual conditions, which cannot be generalised across CLD subtypes. Many prognostic tools have been developed and validated for COPD patients for a variety of outcomes [12,13,14,15,16]. The Gender-Age-Physiology (GAP) model for patients with ILDs, [17, 18] as well as the FACED score and Bronchiectasis Severity Index for patients with bronchiectasis have also been validated for mortality prediction in their respective populations [19]. However, to date, there has not been any model developed for, or validated on CLDs as a group. Furthermore, these tools may incorporate information which may not be collected across all care settings or conditions, such as clinician assessments, lung function measurements or diagnostic investigations [13, 14, 20]. As data on these predictors may not be readily available for all patients, the applicability of these tools in routine clinical practice may be limited.

Development of a prognostic tool generalizable to all patients with CLD would potentially allow for expanded use within a hospital and outpatient clinic setting, with clinicians being able to apply the tool to any patient presenting with CLD of any etiology. To ensure the transferability of the model across care settings, the model would have to be constructed based on routinely available information, such as those recorded in administrative databases. Hence, we aimed to develop and validate a single prognostic model for one-year mortality in patients across various CLDs, using routinely collected data from electronic health records. Our model would then support non-palliative clinicians to identify patients at high risk of deterioration in the next one year to prioritize for holistic needs assessment and subsequent referral for supportive care management.

Methods

Study setting

The public healthcare system in Singapore is organized into three geographic healthcare clusters – the National University Health System, the Singapore Health Services, as well as the National Healthcare Group (NHG). Each cluster serves as the regional health manager for approximately 1.5 million residents, [21] and consists of one or more public hospitals providing acute inpatient, specialist outpatient and emergency care, as well as facilities for primary, subacute and long-term care [22]. The public system accounts for almost 80% of all inpatient admissions in Singapore [23]. Tan Tock Seng Hospital (TTSH), where the study population was recruited from, serves a population of a million residents, and is one of the largest public hospitals in the country with more than 1,700 beds [24].

Study design, study population, and source of data

A retrospective cohort study design was adopted to predict one-year mortality among patients with CLD. Patients were recruited into the study if they had a recorded CLD diagnosis between 1st July 2016 and 31st October 2017 in either the inpatient or specialist outpatient settings within TTSH (Additional File 1). The list of ICD-10CM diagnosis codes indicative of CLD consisted of irreversible progressive respiratory conditions associated with substantial symptom burden towards the end of life and were reviewed by respiratory and palliative clinicians for relevance. COPD patients with a ratio of forced expiratory volume to forced vital capacity exceeding 0.70 in the six months prior were excluded from the study as these patients would not have exhibited persistent airflow obstruction [25].

We extracted patients’ clinical information from electronic health records in TTSH. Death dates, comorbidity information and history of healthcare utilization within NHG were taken from the Regional Health System database, a research database comprising of administrative and billing information (Additional File 2) [26].

Sample size

Based on a sample size calculation assuming 20 predictors per model with a minimum of 10 events per variable over a one-year period, and a hospital mortality rate of 20%, we recruited 1000 consecutive patients who satisfied the inclusion criteria [27].

Outcome definition

The date of CLD diagnosis within the study period for each patient was denoted as their index visit date. All-cause mortality within 365 days of the index visit date was the outcome of interest.

Exposure definition

Potential prognostic factors were shortlisted for inclusion in the model based on a prior scoping review of prognostic factors in non-COPD chronic lung diseases, [28] as well as inputs from respiratory and palliative clinicians. We extracted information on the socio-demographics and prior CLD history as of the index visit. For historical data, we extracted information on low oxygen saturation, need for oxygen therapy or ventilation, as well as biomarkers, echocardiogram findings, functional/physiological and pulmonary assessments six months prior and up to the index visit. Information on comorbidities and previous healthcare utilization was captured up to one year prior.

Missing data

Only variables with less than 50% missing data were considered for model building. Additionally, to facilitate model convergence, variables with less than 10 decedents’ data were excluded from model building. Data were assumed to be missing at random and multiple imputation using chained equations were used to impute missing data, using information on the revised list of predictors as input for the imputation model (Additional File 2). Categorical variables with two levels were imputed using logistic regression, while variables with more levels were imputed using a multinomial model. Continuous variables were imputed using predictive mean matching. Fifty imputed datasets were then generated.

Model building

For each imputed dataset, logistic regression models using forward stepwise selection (FW), backward stepwise selection (BW) and least absolute shrinkage and selection operator (LASSO) were built using the list of predictors in Additional File2. Under FW, we started with an intercept model with no variables, and added individual variables sequentially until we obtained a final model of the lowest Akaike Information Criterion (AIC) value. Under BW, the initial model contained the entire list of predictors, and individual variables were removed until the model with the lowest AIC value was obtained. With LASSO, the model containing the entire list of predictors was optimized to fit the data, omitting certain predictors in the process [29]. The variables selected using each method were tabulated, and variables that were selected across all 50 datasets were used to construct the models for each modelling approach. We included variables that were included in all 50 datasets, above the recommended 50% threshold [30]. This ensured the final set of selected variables were consistently associated with mortality risk across the imputed datasets, resulting in a parsimonious model for implementation. Model coefficient estimates for each of the modelling approaches were averaged using Rubin’s rules across the 50 imputed datasets.

Internal validation of model

For each model in each dataset, we assessed discrimination, calibration, and clinical usefulness of the model. Discrimination, the ability of the model to differentiate between decedents from survivors, [31] was assessed using the Area under Curve (AUC), with values closer to one indicating stronger discriminatory ability. Calibration, the extent of agreement between observed outcomes and model predictions, was assessed through the calibration slope and intercept. The calibration intercept quantifies the bias in mean predictions from mean observed data, with values closer to 0 indicating lower bias. The slope, in contrast, provides an assessment of the trend in bias, with values below one suggesting over-estimation by the model, and values above one suggesting under-estimation [32]. Clinical usefulness, the ability of the model to impact decision-making, was assessed through the sensitivity, specificity, as well as positive and negative predictive values at specific risk thresholds, exceeding which patients were defined to be at high risk of death in the next year [31]. A range of thresholds were used to assess the impact of different thresholds on sensitivity of the model [33].

For each imputed dataset, estimates of these performance measures were optimism-corrected over 100 bootstrap iterations. We then applied Rubin’s rule to pool the performance estimates across the 50 imputed datasets [34]. The final model was selected as the model with the best discriminative performance and acceptable model calibration. Additionally, we included stability plots for selected datasets to reflect the stability of our model predictions and performance over 1000 bootstrap iterations [33].

External validation of model

A second cohort of patients were identified for the external validation of the model. Patients from the validation cohort had at least one inpatient or outpatient visit to TTSH within the period of January 2018 to December 2019, and the same inclusion and exclusion criteria based on diagnosis was applied. Missing data was similarly imputed for 50 datasets. Coefficient estimates of the developed model were applied to these datasets, and model performance estimates and calibration were pooled and reported.

Results

A total of 1,000 patients with a CLD diagnosis from July 2016 to October 2017 were included in the study. One hundred and twenty-two (12.2%) patients died within one year of the index visit date, with a median (interquartile range) time to death of 132 (47–224) days for decedents. The cohort comprised mostly of patients who had a diagnosis of COPD/emphysema (55%), bronchiectasis (38%), or ILDs (12%), including those with overlapping CLD diagnoses (6%). Most patients were included based on their inpatient diagnosis (40.2%). Only 14 (1.4%) patients had an inpatient palliative referral at the index visit, and 20 (2%) were referred to support services in the community, such as sub-acute and home-based care.

Socio-demographic information, clinical profile and history of healthcare utilization of the cohort can be found in Tables 1 and 2. When compared to survivors in univariate analyses, decedents were more likely to be older (decedents: median = 77.5 years, survivors: 71.5 years), seen as inpatients in the previous six months (32.0% vs. 22.2%), but were less likely to be treated in the specialist outpatient setting (65.6% vs. 76.5%). They were also more likely to have a diagnosis of ILD (decedents: median = 20.5%, survivors: 10.3%), started on long term oxygen therapy (LTOT) (9.0% vs. 1.7%), or have a higher comorbidity burden (decedents: median Charlson Comorbidity Index score = 6, survivors: 4). Only nine (7.4%) decedents had a referral to the inpatient palliative services within one year before death.

Table 1 Socio-demographic information and healthcare utilization profile
Full size table
Table 2 Clinical profile
Full size table

We did not include spirometry, echocardiography, several biomarkers and dyspnea scores in model building due to > 70% missing data. (Additional File 3). Applying the three variable selection methods to the imputed datasets, LASSO selected the largest model, with 12 variables, followed by FW and BW (Additional File 4). The model selected via FW was chosen as the final model, having reported the highest AUC at 0.77 (95% CI: 0.72, 0.82); models selected via BW and LASSO reported slightly lower AUCs of 0.74. Based on the calibration intercept and slope estimates, the final model was prone to overpredicting the probability of death (intercept= -0.13, 95% CI: -0.51, 0.24; slope = 0.91, 95% CI: 0.72, 1.10). Calibration and stability plots for selected datasets can be found in Additional File 5 and 6.

The final model consisted of 10 prognostic factors (Table 3). Patients who were older, of male sex or of a minority ethnic group were of higher mortality risk. Other risk factors included being on LTOT, having a history of cancer or a diagnosis of ILD or tuberculosis infection. Having a body mass index below 18.5 kg/m2 or requiring assistance with at least one Activity of Daily Living was also associated with a higher mortality risk. In contrast, having specialist outpatient consultations six months prior to diagnosis was associated with lower mortality risk. The equation to compute the risk of death in one year is provided in Additional File 7.

Table 3 Final model estimates
Full size table

Internal validation

Sensitivity and specificity of the model at a range of risk thresholds are reported in Table 4. Classifying patients with a predicted probability of death of 0.30 and above as high-risk, sensitivity and specificity of the final model was 37% and 95% respectively, indicating that the model would identify three out of ten dying patients and nine of ten surviving patients correctly. The negative predictive value of the model was 92%, indicating that nine of ten patients would be accurately identified to be at low risk of death.

Table 4 Performance measures of final model at internal validation
Full size table

External validation

Of the 500 patients in the external validation (EV) cohort, 97 (19.4%) died within one year of the reference visit date. A comparison of the development and EV cohorts can be found in Additional File 8. In brief, compared to the development cohort, there were more patients admitted via the emergency department or inpatient setting (52.4% vs. 40.2%) in the EV cohort. The EV cohort comprised a larger proportion of patients diagnosed with bronchitis, emphysema, or COPD (80.6% vs. 55.0%), and a smaller proportion of patients with bronchiectasis (19.4% vs. 38.3%) or ILDs (5.6% vs. 11.5%). More patients in the EV cohort had an inpatient admission in the one year prior (59.2% vs. 36.8%), as well as a history of CLD in the past five years (81.8% vs. 64.6%). The Charlson Comorbidity Index score was also higher in the EV cohort compared to the development cohort (median (interquartile range): 5 (4–8) vs. 4 (3–7)).

Applying the coefficients of the final model on the EV cohort, the AUC of the model in the EV cohort was similar at 0.75 (0.70, 0.81), while the extent of overprediction increased slightly with calibration intercept and slope of -0.14 (-0.54, 0.26) and 0.74 (0.53, 0.95) respectively. Sensitivity of the model at the threshold of 0.30 as high risk remained similar at 38%, while the negative predictive value reduced slightly to 86% (Table 5).

Table 5 Pooled model performance measures at external validation
Full size table

Discussion

Main findings

We developed and validated the PROgnostic ModEl for Chronic Lung Disease (PRO-MEL), a model consisting of ten prognostic factors to predict one-year mortality in patients with CLDs. The performance of PRO-MEL remained robust in the validation cohort. Patients identified to be at high-risk of death in the next year by PRO-MEL should be prioritized for needs assessments and subsequent palliative referrals.

The prognostic factors in PRO-MEL corroborate with the wider literature on prognostic factors and indices in lung diseases. Our model incorporated four of ten risk factors identified in a recent systematic review of prognostic factors in COPD: older age, male sex, lower body mass index (BMI), and history of LTOT [35, 36]. Older age and male sex are the components of the GAP and ILD-GAP indices for patients with idiopathic pulmonary fibrosis (IPF) and ILDs, [17, 18] with BMI also included in prognostic indices for prediction of mortality in ILD [37] and bronchiectasis patients [38, 39]. The association of an ILD diagnosis with higher risk of death was consistent with observations of poorer outcomes in patients with ILD compared to a similar group with COPD [40, 41]. The association of a prior history of cancer and stroke with higher mortality risks were expected as cancer and stroke are common causes of death in COPD patients [36]. Poorer functional status depicted as requiring assistance with ADLs has also been suggested to be a marker of disease severity with poorer outcomes in COPD [42].

However, there were established prognostic factors that were absent from our model. Overlapping syndromes, or having multiple respiratory conditions, were not associated with higher mortality risk in the final model, despite other studies noting poorer outcomes in patients with co-occurring COPD, bronchiectasis and ILD [43, 44]. Smoking history was not selected as a prognostic factor, despite being associated with poorer outcomes in both COPD and ILD [45]. The lack of association of these two prominent risk factors with mortality risk in our cohort could possibly be attributed to their similarity in frequency amongst survivors and decedents. The same could also be said about having a prior history of diabetes and cardiovascular disease. While having a recent readmission was not predictive of mortality in our study, having prior outpatient visits was associated with lower mortality risk, reflecting the pertinence of prior healthcare utilization as a proxy of disease severity in CLD. Notably, we also did not assess the role of lung function in predicting mortality due to high missingness of data. While lung function is strongly predictive of patient prognosis in CLD, [14, 28] patients with advanced lung diseases may not be able to perform lung function satisfactorily, limiting its utility in our model intended for application on all CLD patients [46].

The performance of PRO-MEL is comparable to published estimates for disease-specific models for CLD. The AUC of PRO-MEL, at 0.77, was on par with estimates reported for validated prognostic indices for COPD and ILD [35, 47]. The calibration intercept and slope suggested slight over-estimation of predicted mortality risks, but previous validation studies on the GAP index have reported similarly poor calibration, [47, 48] and this metric has not been reported for any of the indices for COPD, [35] preventing comparison. While model sensitivity, specificity, PPV and NPV are hardly reported in other studies, PRO-MEL has exhibited consistently high specificity and NPV across most risk thresholds, indicating that the model will be accurate in identifying patients who are at low-risk of mortality. However, as a diagnostic tool, the sensitivity and PPV of the model should be adequately high to ensure most high-risk patients are correctly identified. Lowering the threshold may increase the sensitivity of the model but would also result in a larger volume of patients to be identified, screened and referred for specialist palliative care. Our selected risk threshold at 0.30 balanced both model sensitivity and the impact on resources to ensure that patients in need of palliative care are adequately identified without overburdening the screening and palliative teams. Furthermore, considering the current referral rates of 6% in the validation cohort and 14-37% in studies elsewhere, PRO-MEL, at a risk threshold of 0.30, would nonetheless facilitate the identification, assessment, and subsequent referral of patients at a rate outperforming the status quo [3,4,5,6].

Validation of PRO-MEL on a more recent cohort of patients with a different mix of CLDs showed similar discriminative performance, but poorer calibration. The model was able to perform reasonably well despite the external validation cohort having a higher mortality rate as well as more patients with COPD, bronchitis or emphysema. PRO-MEL will nonetheless require recalibration through updating of the intercept or a complete reweighting of the predictors to reduce the bias in predictions before use [32]. As patient population characteristics inevitably vary over time, monitoring of model performance, as well as regular recalibration and updating will be necessary as part of model implementation [49, 50]. A separate prospective validation study will be conducted to formally establish the transportability of the model to other settings or over time, [49] and evaluate the impact of the model on clinical practice [51].

Adopting a pragmatic approach with a general prognostic model for CLD would provide non-palliative physicians with a tool to identify those with higher mortality risk across all different etiologies. Current disease-specific prognostic models suffer from the limitation of only predicting mortality risk for diseases that were included in their respective derivative cohort. This excludes patients with other forms of CLDs, such as those with overlapping pathologies or rare lung diseases. For instance, up to 67% of the patients with IPF may have co-existing emphysema, and to date, there are no prognostic tool that uses routinely available clinical parameters for these patients [52]. Despite the good discriminative performance of the ILD-GAP index in common ILDs such as IPF and connective tissue disease-related ILDs, [53, 54] physicians often struggle to prognosticate other forms of rare ILDs [18]. A general prognostic model will be helpful in such instances to alert the managing physician to perform a needs assessment and consider early referral to palliative care.

Moreover, PRO-MEL will be useful to support non-palliative clinicians in prospectively identifying patients likely to deteriorate in the short term, allowing for timely access to quality patient care to address their needs along their subsequent disease trajectory [11]. Kaur et al. demonstrated that patients with advanced respiratory disease incur substantial healthcare expenditure and utilization in the last year of life, with a sharp increase in the last months of life, mainly driven by inpatient admissions, [55] suggesting that these patients would benefit from early access to palliative care to address their needs. By identifying patients who are at high risk of death within the next year, those likely to deteriorate and experience exacerbations or complications in the coming months could receive both disease-modifying and palliative treatment concurrently [56]. Compared with patients with lung cancer, patients with COPD have more frequent and longer stays in intensive care, and were less likely to receive medication for symptom palliation [57]. Patients with bronchiectasis or ILD were less likely than patients with COPD to receive palliative support [58]. The early identification of high-risk patients for review and assessment would improve timeliness of referrals and access to palliative care for these patients, [9] and mitigate the escalation of healthcare utilization and expenditure nearing the end of life [59]. Introduction of primary palliative care concurrent with disease-specific treatment will focus on improving their quality of life and wellbeing, while supporting them in managing their symptoms, as well as the emotional, social and spiritual aspects of their disease [10].

Strengths and limitations

Strengths of our study include the adoption of best practices in model development methodology. Prognostic factors included in our study were incorporated from prior systematic reviews on prognostic factors in COPD, and a scoping review on non-COPD prognostic factors the team conducted [13, 14, 28]. Candidate predictors were also screened for availability of data to ensure that the information would be routinely available in most patients and not be exclusive to specific healthcare settings, thereby expanding the generalizability of the model. We also conducted multiple imputations of missing data for 50 datasets, and shortlisted predictors which were applicable to all imputed datasets. This ensured that only predictors that were always associated with mortality across all the datasets were selected, maintaining the parsimony of the final model. In addition to internal validation, we also performed an additional temporal validation to ensure the model performance was reliable in a separate, more recent cohort of patients. Lastly, the model uses routinely available data in administrative billing databases, increasing the transferability of the model to other health systems.

Limitations of our study include the use of data from only one of the three public healthcare clusters in Singapore, which may limit the generalizability of our findings to other health systems throughout the world. However, the model was constructed on an ethnically diverse population, and the role of ethnicity was reflected as the disparity between the ethnic majority and minority groups, increasing its applicability to other health systems with a similar multi-ethnic demographic structure. Furthermore, as most of the predictors in PRO-MEL are related to diagnoses and prior healthcare utilization, we expect the model to maintain reasonable utility in identifying patients at high risk of mortality in other populations. As we used ICD-10CM diagnosis codes to identify patients and did not have access to complete spirometry, dyspnea or exacerbation data, we were unable to stage the disease to identify patients of greater disease severity. Furthermore, we did not incorporate spirometry, history of exacerbations, or other measures of dyspnea in the model, as data was not routinely available for all patients. Lastly, as this was a retrospective database study, the model was developed on historical data, and would require validation in a subsequent prospective study to ensure its utility in clinical practice. Lastly, as we validated the model on a newer cohort of patients from the same health system in Singapore, we were unable to demonstrate if the model will perform similarly well in other health systems. Further external validation studies on independent populations and settings are required.

Conclusion

We developed and validated a one-year mortality prediction model, PRO-MEL, in patients with CLD using routinely available administrative data. PRO-MEL will support non-palliative clinicians across different settings and specialties in identifying patients earlier at their disease trajectory but at high risk of deterioration for a comprehensive basic palliative care assessment to identify unmet needs. This allows for earlier support of their psycho-emotional and social needs, and thereby improving their quality of life. Future work will assess the clinical usefulness of PRO-MEL as a mortality risk prediction algorithm embedded within electronic medical record systems, as part of a clinical workflow for identification, assessment, and management of patients with non-malignant chronic respiratory disease.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to NHG’s data protection policies and restrictions imposed by the ethics committee to ensure data privacy of patients. Aggregate data may be made available from the corresponding author on reasonable request.

Abbreviations

AUC:

Area Under Curve

BMI:

Body mass index

BW:

Backward stepwise

CLD:

Chronic lung disease

COPD:

Chronic obstructive pulmonary disease

EV:

External validation

FW:

Forward stepwise

GAP:

Gender-Age-Physiology

ILD:

Interstitial lung disease

IPF:

Idiopathic pulmonary fibrosis

LASSO:

Least absolute shrinkage and selection operator

LTOT:

Long term oxygen therapy

NHG:

National Healthcare Group

PRO-MEL:

PROgnostic ModEl for Chronic Lung Disease

TTSH:

Tan Tock Seng Hospital

References

  1. GBD Chronic Respiratory Disease Collaborators. Prevalence and attributable health burden of chronic respiratory diseases, 1990–2017: a systematic analysis for the global burden of Disease Study 2017. Lancet Respiratory Med. 2020;8(6):585–96.

    Article  Google Scholar 

  2. Butler SJ, Ellerton L, Gershon AS, Goldstein RS, Brooks D. Comparison of end-of-life care in people with chronic obstructive pulmonary disease or lung cancer: a systematic review. Palliat Med. 2020;34(8):1030–43.

    Article  PubMed  Google Scholar 

  3. Bloom CI, Slaich B, Morales DR, Smeeth L, Stone P, Quint JK. Low uptake of palliative care for COPD patients within primary care in the UK. Eur Respir J. 2018;51(2):1701879.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Scheerens C, Beernaert K, Pype P, Cohen J, Deliens L, Chambaere K. Comparing the use and timing of palliative care services in COPD and lung cancer: a population-based survey. Eur Respir J. 2018;51(5):1702405.

    Article  PubMed  Google Scholar 

  5. Lindell KO, Liang Z, Hoffman LA, Rosenzweig MQ, Saul MI, Pilewski JM, et al. Palliative care and location of death in decedents with idiopathic pulmonary fibrosis. Chest. 2015;147(2):423–9.

    Article  PubMed  Google Scholar 

  6. Zou RH, Nouraie M, Chen X, Saul MI, Kaminski N, Gibson KF, et al. Assessing patterns of palliative care referral and location of death in patients with idiopathic pulmonary fibrosis: a sixteen-year single-center retrospective cohort study. J Palliat Med. 2019;22(5):538–44.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Brown CE, Jecker NS, Curtis JR. Inadequate palliative care in chronic lung disease. An issue of health care inequality. Annals Am Thorac Soc. 2016;13(3):311–6.

    Article  Google Scholar 

  8. Marsaa K, Gundestrup S, Jensen J-U, Lange P, Løkke A, Roberts NB et al. Danish respiratory society position paper: palliative care in patients with chronic progressive non-malignant lung diseases. Eur Clin Respiratory J. 2018;5(1530029).

  9. Janssen DJA, Bajwah S, Boon MH, Coleman C, Currow DC, Devillers A, et al. European Respiratory Society clinical practice guideline: palliative care for people with COPD or interstitial lung disease. Eur Respir J. 2023;62:2202014.

    Article  PubMed  Google Scholar 

  10. Vitorino JV, Duarte BV, Laranjeira C. When to initiate early palliative care? Challenges faced by healthcare providers. Front Med. 2023;10(1220370).

  11. Downar J, Wegier P, Tanuseputro P. Early identification of people who would benefit from a palliative approach—moving from surprise to routine. JAMA Netw Open. 2019;2(9):e1911146.

    Article  PubMed  Google Scholar 

  12. Singanayagam A, Schembri S, Chalmers JD. Predictors of mortality in hospitalized adults with acute exacerbation of chronic obstructive pulmonary disease. Annals Am Thorac Soc. 2013;10(2):81–9.

    Article  Google Scholar 

  13. Smith L-JE, Moore E, Ali I, Smeeth L, Stone P, Quint JK. Prognostic variables and scores identifying the end of life in COPD: a systematic review. Int J COPD. 2017;12:2239–56.

    Article  Google Scholar 

  14. Bellou V, Belbasis L, Konstantinidis AK, Tzoulaki I, Evangelou E. Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal. BMJ. 2019;367:l5358.

    Article  PubMed  PubMed Central  Google Scholar 

  15. van Dijk WD, Lvd B, Svd H-R, Bischoff E, Weel Cv V, JCit et al. Multidimensional prognostic indices for use in COPD patient care. A systematic review. Respir Res. 2011;12(151).

  16. Tsai S-H, Shih C-Y, Kuo C-W, Liao X-M, Lin P-C, Chen C-W et al. Development and validation of a prediction index for recent mortality in advanced COPD patients. NPJ Prim Care Respiratory Med. 2022;32(2).

  17. Ley B, Ryerson CJ, Vittinghoff E, Ryu JH, Tomassetti S, Lee JS, et al. A multidimensional index and staging system for idiopathic pulmonary fibrosis. Ann Intern Med. 2012;156:684–91.

    Article  PubMed  Google Scholar 

  18. Ryerson CJ, Vittinghoff E, Ley B, Lee JS, Mooney JJ, Jones KD, et al. Predicting survival across chronic interstitial lung disease: the ILD-GAP model. Chest. 2014;145(4):723–8.

    Article  PubMed  Google Scholar 

  19. He M, Zhu M, Wang C, Wu Z, Xiong X, Wu H, et al. Prognostic performance of the FACED score and bronchiectasis severity index in bronchiectasis: a systematic review and meta-analysis. Biosci Rep. 2020;40:BSR20194514.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. ElMokhallalati Y, Bradley SH, Chapman E, Ziegler L, Murtagh FE, Johnson MJ, et al. Identification of patients with potential palliative care needs: a systematic review of screening tools in primary care. Palliat Med. 2020;34:989–1005.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ministry of Health Singapore. White paper on Healthier SG. 2022.

  22. Ministry of Health Singapore. Hospital Services 2021 [updated 12 Jun 2023. https://www.moh.gov.sg/home/our-healthcare-system/healthcare-services-and-facilities/hospital-services

  23. Tan CC, Lam CSP, Matchar DB, Zee YK, Wong JEL. Singapore’s health-care system: key features, challenges, and shifts. Lancet. 2021;398:1091–104.

    Article  PubMed  Google Scholar 

  24. Tan Tock Seng Hospital. Corporate Profile 2023. https://www.ttsh.com.sg/About-TTSH/Pages/Corporate-Profile.aspx

  25. Agusti A, Celli BR, Criner GJ, Halpin D, Anzueto A, Barnes P, et al. Global initiative for chronic obstructive lung disease 2023 report: GOLD executive summary. Am J Respir Crit Care Med. 2023;207(7):819–37.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Gunapal PPG, Kannapiran P, Teow KL, Zhu Z, You AX, Saxena N et al. Setting up a regional health system database for seamless population health management in Singapore. Proceedings of Singapore Healthcare. 2016;25(1):27–34.

  27. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9.

    Article  CAS  PubMed  Google Scholar 

  28. Ng SHX, Chai GT, George PP, Kaur P, Yip WF, Chiam ZY et al. Prognostic factors of mortality in non-chronic obstructive pulmonary disease chronic lung disease: a scoping review. J Palliat Med. 2023.

  29. Heinze G, Wallisch C, Dunkler D. Variable selection – a review and recommendations for the practicing statistician. Biom J. 2018;60:431–49.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Thao LTP, Geskus R. A comparison of model selection methods for prediction in the presence of multiply imputed data. Biom J. 2019;61(2):343–56.

    Article  PubMed  Google Scholar 

  31. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35:1925–31.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Van Calster B, McLernon DJ, Smeden Mv, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17(230).

  33. Riley RD, Collins GS. Stability of clinical prediction models developed using statistical or machine learning methods. Biom J. 2023(202200302).

  34. Rubin DB. Multiple imputation for nonresponse in surveys. New York: John Wiley & Sons, Inc; 1987.

    Book  Google Scholar 

  35. Owusuaa C, Dijkland SA, Nieboer D, Rijt CCDvd H, Avd. Predictors of mortality in chronic obstructive pulmonary disease: a systematic review and meta–analysis. BMC Pulm Med. 2022;22(125).

  36. Aisanov Z, Khaltaev N. Management of cardiovascular comorbidities in chronic obstructive pulmonary disease patients. J Thorac Disease. 2020;12(5):2791–802.

    Article  Google Scholar 

  37. Pugashetti JV, Oldham JM. Weighing on our minds: baseline BMI and weight loss as predictors of interstitial lung disease outcome. Chest. 2022;161(5):1134–5.

    Article  PubMed  Google Scholar 

  38. Mateus SP, Ribeiro-Alves M, Salles REB, Costa W, Lopes CHCAJ, Bártholo TP et al. Mortality and comorbidities in patients with bronchiectasis over a 3-year follow-up. 2022;101(52):e32537.

  39. Lee JM, Lee SA, Han CH, Lee SM, Kim CJ, Lee SC, et al. Body mass index as a predictor of mortality in bronchiectasis: a nationwide population-based study. Respir Med. 2021;180:106370.

    Article  PubMed  Google Scholar 

  40. Palm A, Ekström M. Hypoxemia severity and survival in ILD and COPD on long-term oxygen therapy - the population-based DISCOVERY study. Respir Med. 2021;189:106659.

    Article  PubMed  Google Scholar 

  41. Khor YH, Harrison A, Robinson J, Goh NSL, Glaspole I, McDonald CF. Moderate resting hypoxaemia in fibrotic interstitial lung disease. Eur Respir J. 2021;57(1):2001563.

    Article  PubMed  Google Scholar 

  42. Antoniu SA, Boiculese LV, Prunoiu V. Frailty, a dimension of impaired functional status in advanced COPD: utility and clinical applicability. Medicina. 2021;57(474).

  43. Martinez-Garcia MA, Miravitlles M. Bronchiectasis in COPD patients: more than a comorbidity? Int J COPD. 2017;12:1401–11.

    Article  CAS  Google Scholar 

  44. Elhussini MS, Eid AMMHA, Gharib A. Bronchiectasis as Co morbidity with COPD or ILD: complex interactions and severe consequences. Egypt J Bronchol. 2023;17:19.

    Article  Google Scholar 

  45. Beghé B, Cerri S, Fabbri LM, Marchioni A. COPD, pulmonary fibrosis and ILAs in aging smokers: the paradox of striking different responses to the major risk factors. Int J Mol Sci. 2021;22(9292).

  46. Prasad KT, Sehgal IS, Dhooria S, Singh N, Agarwal R, Behera D, et al. Patient characteristics and outcome of end–stage lung diseases referred for lung transplantation in North India. Lung India. 2018;35:290–4.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Kim ES, Choi SM, Lee J, Park YS, Lee C-H, Yim J-J, et al. Validation of the GAP score in Korean patients with idiopathic pulmonary fibrosis. Chest. 2015;147(2):430–7.

    Article  PubMed  Google Scholar 

  48. Zhang X, Ren Y, Xie B, Wang S, Geng J, He X et al. External validation of the GAP model in Chinese patients with idiopathic pulmonary fibrosis. Clin Respiratory J. 2022:1–10.

  49. Van Calster B, Steyerberg EW, Wynants L, Smeden M. There is no such thing as a validated prediction model. BMC Med. 2023;21(70).

  50. Jenkins DA, Martin GP, Sperrin M, Riley RD, Debray TPA, Collins GS et al. Continual updating and monitoring of clinical prediction models: time for dynamic prediction systems? Diagn Prognostic Res. 2021;5(1).

  51. van Royen FS, Moons KGM, Geersing G-J, Smeden Mv. Developing, validating, updating and judging the impact of prognostic models for respiratory diseases. Eur Respir J. 2022;60:2200250.

    Article  PubMed  Google Scholar 

  52. Cottin V, Selman M, Inoue Y, Wong AW, Corte TJ, Flaherty KR, et al. Syndrome of combined pulmonary fibrosis and emphysema: an official ATS/ERS/JRS/ALAT research statement. Am J Respir Crit Care Med. 2022;206(4):e7–41.

    Article  PubMed  Google Scholar 

  53. Kam MLW, Li HH, Tan YH, Low SY. Validation of the ILD-GAP Model and a local Nomogram in a Singaporean cohort. Respiration. 2019;98(5):383–90.

    Article  CAS  PubMed  Google Scholar 

  54. Ryerson CJ, O’Connor D, Dunne JV, Schooley F, Hague CJ, Murphy D, et al. Predicting mortality in systemic sclerosis-associated interstitial lung disease using risk prediction models derived from idiopathic pulmonary fibrosis. Chest. 2015;148(5):1268–75.

    Article  PubMed  Google Scholar 

  55. Kaur P, Wu HY, Hum A, Heng BH, Tan WS. Medical cost of advanced llnesses in the last-year of life—retrospective database study. Age Ageing. 2021:1–8.

  56. Bourke S, Peel E. Palliative care of chronic progressive lung disease. Clin Med. 2014;14(1):79–82.

    Article  CAS  Google Scholar 

  57. Au DH, Udris EM, Fihn SD, McDonell MB, Curtis JR. Differences in health care utilization at the end of life among patients with chronic obstructive pulmonary disease and patients with lung cancer. Arch Intern Med. 2006;166:326–31.

    Article  PubMed  Google Scholar 

  58. Mc Veigh C, Reid J, Larkin P, Porter S, Hudson P. Palliative care for people with non-malignant respiratory disease and their carers: a review of the current evidence. J Res Nurs. 2019;24(6):420–30.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Hum A, Yap CW, Koh MYH. End-stage organ disease–Healthcare utilisation: impact of palliative medicine. BMJ Supportive Palliat Care. 2021;0:1–7.

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge the contributions by Mr Palvannan Kannapiran in guiding the modelling approaches adopted in this study.

Funding

This work was supported by the National Medical Research Council, Singapore [grant number HSRGEoL18may-0003]. The funder had no involvement in any aspect of this study or decision to publish.

Author information

Authors and Affiliations

  1. Health Services and Outcomes Research, National Healthcare Group, Annex @ National Skin Centre, 1 Mandalay Road, Singapore, 308205, Singapore

    Sheryl Hui-Xian Ng, Palvinder Kaur, Wan Fen Yip & Woan Shin Tan

  2. Department of Palliative Medicine, Tan Tock Seng Hospital, 11 Jalan Tan Tock Seng, Singapore, 308433, Singapore

    Zi Yan Chiam, Zhi Jun Low, Jermain Chu, Lee Hung Tey, Han Yee Neo & Allyn Hum

  3. Department of Respiratory and Critical Care Medicine, Tan Tock Seng Hospital, 11 Jalan Tan Tock Seng, Singapore, 308433, Singapore

    Gin Tsen Chai

  4. Lee Kong Chian School of Medicine, Nanyang Technological University, 11 Mandalay Road, Singapore, 308232, Singapore

    Gin Tsen Chai

  5. The Palliative Care Centre for Excellence in Research and Education, Dover Park Hospice, 10 Jalan Tan Tock Seng, Singapore, 308436, Singapore

    Allyn Hum

Authors
  1. Sheryl Hui-Xian Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zi Yan Chiam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gin Tsen Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Palvinder Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wan Fen Yip
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhi Jun Low
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jermain Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lee Hung Tey
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Han Yee Neo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Woan Shin Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Allyn Hum
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TWS and AH conceptualized the study. JC, TLH and LZJ acquired the data and SNHX analysed it. CGT and CZY interpreted the data. SNHX, CZY and CGT drafted the article, with critical revision by YWF, NHY, TWS and AH. All authors approved the current version of the article to be published.

Corresponding author

Correspondence to Sheryl Hui-Xian Ng.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was granted by the review board of the National Healthcare Group as part of a larger study on prognosticating end-stage organ failure (Domain Specific Review Board Study Reference No. 2019/00032). No written consent for participation was obtained for this study as a waiver of informed consent was granted.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Woan Shin Tan and Allyn Hum are joint senior authors.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ng, S.HX., Chiam, Z.Y., Chai, G.T. et al. The PROgnostic ModEl for chronic lung disease (PRO-MEL): development and temporal validation. BMC Pulm Med 24, 429 (2024). https://doi.org/10.1186/s12890-024-03233-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12890-024-03233-0

Keywords

BMC Pulmonary Medicine

ISSN: 1471-2466

Contact us