Comparing new treatments for idiopathic pulmonary fibrosis – a network meta-analysis
BMC Pulmonary Medicine volume 15, Article number: 37 (2015)
The treatment landscape for idiopathic pulmonary fibrosis, a devastating lung disease, is changing. To investigate the effectiveness of treatments for idiopathic pulmonary fibrosis we undertook a systematic review, network meta-analysis and indirect comparison.
We searched MEDLINE, EMBASE and The Cochrane library for relevant studies. Randomised controlled trials of pirfenidone, nintedanib or N-acetylcysteine were eligible. Predefined processes for selecting references, extracting data and assessing study quality were applied. Our network meta-analysis of published data used a fixed effect model. For forced vital capacity measures a standardised mean difference approach was used and converted to odds ratios for interpretation.
Of 1076 references, 67 were retrieved and 11 studies included. Studies were of reasonable size, populations were similar, and the overall quality was good. Only two treatments, pirfenidone (odds ratio 0.62, 95% credible interval 0.52, 0.74) and nintedanib (0.41, 95% credible interval 0.34, 0.51) produced a statistically significant slowing in the rate of forced vital capacity decline compared with placebo. In an indirect comparison, results indicate that nintedanib is statistically significantly better than pirfenidone in slowing forced vital capacity decline (odds ratio 0.67, 95% credible interval 0.51, 0.88). Results were stable in scenario analysis and random effects models. Indirect comparisons of mortality were not statistically significant between nintedanib and pirfenidone.
Two treatments show beneficial effects and when compared indirectly nintedanib appears to have superior benefit on forced vital capacity. Limitations to indirect comparisons should be considered when interpreting these results, however, our findings can be useful to inform treatment decisions.
The emergence of new evidence assessing the effectiveness of therapies for idiopathic pulmonary fibrosis (IPF) has created considerable interest and hope amongst patients, carers and clinicians. IPF is a devastating lung disease characterised by the deposition of excessive scar tissue within the lungs, which leads to breathlessness and ultimately respiratory failure and death. It has been well documented that the incidence of IPF is increasing but the reasons for this are unclear. High mortality rates are also well reported, and based on reported 5-year survival rates these would place IPF seventh on a list of fatal malignancies [1,2]. Until recently few treatment options have been available, so the focus of treatment for many patients has been symptom control and palliation .
The new optimism has stemmed from the fact pirfenidone, licensed in Europe in 2011 on the basis of evidence from 4 randomised controlled trials (RCTs) [4-6], has been shown in a recent multicentre RCT to slow the rate of progression of IPF . Furthermore another agent (nintedanib) also met its primary end-point of reducing the annual rate of pulmonary function decline in two concurrent RCTs . On the strength of these data the Food and Drug Administration (FDA) and the European Medicine Agency have recently approved both drugs for use in IPF. These treatments have the potential for offering new hope for patients and carers  and clinicians will be eager to offer patients the most appropriate treatment. It is therefore imperative to consider the weight of all the evidence for each of these treatments.
The purpose of this study was to systematically review the clinical effectiveness of treatments for IPF and present the findings of a network meta-analysis (NMA) of key outcomes. We followed the principles of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement .
The methods for this systematic review are described in a research protocol which is registered with the International Prospective Register of Systematic Reviews . This is an update of a previous systematic review which also considered cost-effectiveness . We identified articles by searches of MEDLINE, EMBASE and the Cochrane Library from database inception until May 2014 (for search strategies see Additional file 1: Table S1). No language restrictions were applied. Two independent reviewers screened titles and abstracts for inclusion and full text articles were retrieved for further scrutiny. These were reviewed by one reviewer and checked by a second to identify RCTs that included participants with a confirmed diagnosis of IPF. Eligible interventions were N-acetylcysteine (NAC) alone or in combination, pirfenidone or nintedanib, assessed on outcomes measuring indices of lung function/capacity, exercise performance, quality of life and adverse events.
Data extracted from studies included participant and study characteristics, intervention and comparator details and results. An assessment of the methodological quality of each included study was made . Only published data were included. One reviewer undertook data extraction and quality assessment. These were checked by a second reviewer and differences in opinion were resolved through discussion with a third reviewer. We synthesised data in a narrative review and a Bayesian NMA [14,15]. In circumstances where randomised evidence between all relevant comparators is unavailable, network meta-analysis combines evidence from trials comparing different sets of treatments that form a connected evidence network through common comparators, in this case placebo. It retains within trial randomisation, allowing direct and indirect evidence to inform estimates of relative treatment effect in a single analysis. A vague prior distribution (a normal distribution with mean zero and variance 10,000) that contains little information relative to the likelihood was used to ensure that results were based on the data and not the choice of prior .
The NMA assessed five endpoints from 11 studies. An advantage of NMA is that where interventions have not been directly compared in RCTs the results of different trials may be used to estimate the relative treatment effect between two treatments through indirect comparison. In addition, the analysis allows for meta-regression to control for study level covariates which may be a source of heterogeneity between trials .
The decline in FVC may be measured in two ways: as a decline in FVC % predicted; and as an absolute change from baseline (litres). In addition, some trials report vital capacity, which can be assumed to be the same as FVC in IPF. We used the standardised mean difference (SMD) approach to convert these measurements to a common scale using Hedges’ adjusted g method to overcome potential small sample bias . The SMDs were then converted to log odds ratios (logOR) using the method of Chinn  to facilitate interpretation.
Our base case used the FVC % predicted where reported, as by definition this includes adjustment for age and sex. We conducted sensitivity analyses using litres to examine variation in results arising from the different measurement scales, and controlling for baseline FVC through meta-regression.
The NMA was performed in WinBUGS using code adapted from Dias and colleagues , see Additional file 1: Appendix. Two chains were run for 50,000 simulations with a burn-in period of 20,000 and a thinning interval of 2, giving a final sample size of 30,000. Trace, Brooks-Gelman-Rubin, and density plots were examined to establish model convergence . Fixed and random effects models were conducted with best model fit determined by the deviance information criterion .
Searches yielded 1076 unique references and 67 of these were retrieved after initial screening. Of these we included 11 studies (Figure 1). Five RCTs evaluated the use of pirfenidone [4-7], three NAC [22-24], and three nintedanib. [8,25] Two of the NAC studies [22,23] are in fact one three arm RCT and as such there is not complete independence in the placebo arms. Table 1 provides summary descriptions of each study. The participants in all studies would likely be classed as mild to moderate IPF, with baseline FVC  ranging from approximately 68% to 89% of predicted values. Participants were around 64–68 years old, and were predominantly men. In most studies diagnosis was within the previous two years. The ratio of the forced expiratory volume in the first second to the FVC was not reported consistently across the studies. In three recent RCTs of pirfenidone  and nintedanib [7,8] this was required to be at least 0.80 and at least 0.70 for study inclusion respectively. For the RCT of NAC [22,23] this was required to be at least 0.65. Although there were subtle differences in trial inclusion criteria and diagnostic criteria, baseline characteristics were broadly similar and are deemed to be generally reflective of IPF patients with mild-to-moderate disease seen in clinical practice. As such we judged these populations to be similar enough to be combined in meta-analysis. Primary outcomes were based on the FVC in all but one trial . With one exception , the studies had reasonable sample sizes and were adequately powered to test differences in their primary outcomes. Treatment duration ranged between eight and 16 months, with the majority following up until approximately 12 months.
Overall the studies were assessed as having a low or uncertain risk of bias based on the assessment of adequacy of concealment of allocations prior to assignment to groups. All but one trial  were assessed as having adequate randomisation procedures, and blinding was at an unclear or low risk of bias in all but one trial . Most trials described using an intention to treat (ITT) analysis, although not all reported details. Several trials presented adjusted analyses incorporating a range of covariates and interactions [4,5,7,8,22,23]. As the adjustments in the analyses were to take account of within-study variability in the outcome associated with the covariates, it did not negate meta-analysis of studies with and without adjustment.
FVC outcome data used in the NMA can be seen in Table 2 and the evidence network in Figure 2. Trials were relatively homogeneous which validates the use of fixed effect models. When combining the different FVC treatment effects only pirfenidone and nintedanib produce a statistically significant slowing in the rate of FVC decline compared with placebo (ORs < 1 with 95% credible intervals [Crl] all less than 1) (Figure 3). These two treatments were compared indirectly using the placebo as the common comparator, and results indicate that nintedanib is statistically significantly better than pirfenidone in slowing FVC decline (OR 0.67, 95% CrI 0.51, 0.88). Random effects analysis of FVC supports the trends seen although the results were not statistically significant. Sensitivity analysis controlling for baseline FVC had little impact on results. Using FVC decline in litres, where reported, also gave similar results.
A number of trials reported FVC as a dichotomous outcome, using the proportion with a decline in FVC % predicted of 10% or more (see Table 2). In our NMA both treatments are associated with statistically significantly lower odds of a decline in FVC % predicted of ≥10% compared to placebo. The indirect comparison of nintedanib and pirfenidone indicates that pirfenidone is associated with slightly lower odds of a decline in FVC % predicted of ≥10% compared to nintedanib, but that this is non-significant (OR 1.21, 95% CrI 0.86, 1.72), Additional file 1: Figure S1. Although not significant this trend is divergent to the other FVC outcomes and may be related to differences in definitions in the studies. For example in one trial  this was the proportion with a reduction >10% in FVC or death. It is also uncertain whether studies reported this 10% decline as an absolute or a relative decline. Three trials reported results as an annualised decline in FVC but the network for this outcome was not complete for comparison.
The FVC treatment effect versus placebo in one pirfenidone trial  appears to be large when compared with other trials. Two scenario analyses, first to exclude this trial from the FVC % predicted model, and second to adjust the numbers with an FVC decline greater than 10% to remove all cause deaths, did not substantively alter the outputs of the NMA.
Evidence of acute exacerbations were reported in some of the included trials but there were some differences in rates reported. One trial  only reported the incidence per 100 years which we converted using the study sample size to rate per year, and two trials  reported the proportion of participants with at least one event (Additional file 1: Table S2). Acute exacerbations are relatively rare occurrences and studies were not powered to detect a difference in these rates, therefore results should be interpreted cautiously (for the OR for acute exacerbation versus placebo, see Additional file 1: Figure S2). Both nintedanib and pirfenidone have favourable point estimates, however only the OR for nintedanib achieves significance. Owing to a number of uncertainties with these data no indirect comparison was undertaken.
All-cause mortality and respiratory-related mortality were reported in trials of all treatments with the exception of inhaled NAC (Additional file 1: Table S3). The NMA ORs for all-cause mortality compared with placebo showed that both nintedanib and pirfenidone had favourable point estimates, however, only pirfenidone was statistically significant (Additional file 1: Figure S3). In indirect comparison pirfenidone is associated with lower odds of all-cause mortality compared to nintedanib, but this is not significant (OR 1.39, 95% CrI 0.70, 2.82). Similar results were seen for respiratory mortality, see Additional file 1: Table S4 and Figure S4. Studies were not powered to detect a difference on mortality and therefore the results should be considered cautiously.
The included trials reported other outcomes less consistently and are therefore not appropriate for meta-analysis. No studies that reported it showed significant effects of treatment on diffusing capacity of the lung [4-6,22,23], and effects on the six-minute walk test were mixed across the trials with only three showing a positive effect [4,7,22]. Few studies reported dyspnoea and results were not significant [4,22,23]. On the St George’s Respiratory Questionnaire, a measure of quality of life, there were mixed results but overall it would appear that the treatments have little effect on quality of life.
This study presents a comprehensive review of evidence of the effectiveness of three treatments for IPF and is a timely addition to the ongoing appraisal of recently published trials by patients and clinicians. Evidence for FVC shows a significantly slower decline in patients treated with pirfenidone or nintedanib compared to placebo. When compared indirectly, there was a slower decline in FVC for those treated with nintedanib than for pirfenidone. On endpoints thought more important for patients, such as mortality and acute exacerbations, there were mixed results. Nintedanib was significantly better than placebo for acute exacerbations and pirfenidone was significantly better than placebo for mortality and respiratory mortality. Indirect comparisons of mortality outcomes were not statistically significant.
Some aspects of the evidence included in the review should be taken into account when interpreting the findings. The study is limited by inconsistent reporting of the key outcomes in the trial publications and definitions of outcomes differed. It has been questioned whether the participants in one RCT (ASCEND)  differ from other trials . Although our assessment of heterogeneity suggests no statistically significant heterogeneity, the rate of decline in FVC in the placebo group appears to tail off near to the end of the trial. On observation of placebo arms of all other included trials this is at a greater rate, although the slope of decline seen in the more recently published trials are more similar. One possible reason for this reduction may be related to the effects of the imputation of FVC data for patients who died, however, in scenario analyses to test the consistency of results we would suggest that the difference between groups is similar to the other pirfenidone trials. Finally, four of the trials did not report ITT analysis, which is often recommended as the least biased way to estimate intervention effects in RCTs . It is possible therefore that treatment effects may be exaggerated in these trials [5,6,8,24].
On FVC, within individual trials, results have not been entirely consistent in their comparisons with placebo. However, through NMA results show statistically significant effects for both of pirfenidone and nintedanib. Both treatments have received authorisation from the FDA and in Europe. It is anticipated that the findings of our indirect comparisons between nintedanib and pirfenidone will be of use to inform decision making in the clinical setting. This is particularly important given that until recently there has been an unmet need. Few trials reported estimates of quality of life and good quality research to establish the quality of life of patients with IPF should be a priority. This would allow a fuller assessment of the effectiveness of these treatments. Additionally, these treatments do not stabilize or reverse the decline in IPF and more research into treatments with the potential to meet this aim is required.
Our review and NMA has been undertaken following recognised principles for undertaking a systematic review to ensure that our analyses are transparent and as unbiased as possible. Additionally we ensured that only the highest quality studies were included to limit uncertainty in the results. Our group has no direct vested interest in the pivotal trials. Limitations to our study are that the NMA for FVC assumes the different measures of FVC are equivalent and we converted results to odds ratios to provide meaningful results. Although recognised methodologies were used, these factors should be considered when interpreting the results. Assumptions of homogeneity, similarity, and consistency between direct and indirect evidence in our NMA were made , however, uncertainties remain. For example, there could be other sources of observed or unobserved heterogeneity which could impact on relative treatment effects.
Beneficial treatment effects were demonstrated across key outcomes for two included interventions. In indirect comparison nintedanib was associated with significantly better outcome on slowing the decline in FVC than pirfenidone and this finding was robust in sensitivity analyses. Mortality rates showed trends in favour of pirfenidone but these were not statistically significant. Our findings can be used to help inform treatment decisions for this population of patients who following the recent regulator’s approvals will benefit from greater access to these therapies.
Forced vital capacity
Idiopathic pulmonary fibrosis
Randomised controlled trial
Standardised mean difference
Kim DS, Collard HR, King Jr TE. Classification and natural history of the idiopathic interstitial pneumonias. Proc Am Thorac Soc. 2006;3:285–92.
American Cancer Society. Cancer Facts and Figures 2010. 2010. 13-6-2014
Costabel U. Idiopathic pulmonary fibrosis: recent milestones in disease management. Eur Respir Rev. 2012;21:140.
Noble PW, Albera C, Bradford WZ, Costabel U, Glassberg MK, Kardatzke D, et al. Pirfenidone in patients with idiopathic pulmonary fibrosis (CAPACITY): two randomised trials. Lancet. 2011;377:1760–9.
Taniguchi H, Ebina M, Kondoh Y, Ogura T, Azuma A, Suga M, et al. Pirfenidone in idiopathic pulmonary fibrosis. Eur Respir J. 2010;35:821–9.
Azuma A, Nukiwa T, Tsuboi E, Suga M, Abe S, Nakata K, et al. Double-blind, placebo-controlled trial of pirfenidone in patients with idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 2005;171:1040–7.
King Jr TE, Bradford WZ, Castro-Bernardini S, Fagan EA, Glaspole I, Glassberg MK, et al. A phase 3 trial of pirfenidone in patients with idiopathic pulmonary fibrosis. N Engl J Med. 2014;370:2083–92.
Richeldi L, du Bois RM, Raghu G, Azuma A, Brown KK, Costabel U, et al. Efficacy and safety of nintedanib in idiopathic pulmonary fibrosis. N Engl J Med. 2014;370:2071–82.
Hunninghake GM. A new hope for idiopathic pulmonary fibrosis. NEJM. 2014;370:2142–3.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). BMJ. 2009;339:b2535.
Loveman E, Colquitt JL, Copley V, Scott DA, Clegg A, Richeldi L, et al. Treatments for idiopathic pulmonary fibrosis: an updated systematic review and network meta-analysis. PROSPERO: International prospective register of systematic reviews. 2014
Loveman E, Copley V, Colquitt JL, Scott D, Clegg A, Jones J, et al. The clinical and cost effectiveness of treatments for idiopathic pulmonary fibrosis: a systematic review and economic evaluation. Health Technol Assess, 2015;19:20.
Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in health care, Third edition. York, UK: York Publishing Services Ltd., CRD; 2009.
Caldwell D, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331:897–900.
Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making. 2013;33:607–17.
Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005;24:2401–28.
Salanti G, Marinho V, Higgins JPT. A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol. 2009;62:857–64.
Egger M, Davey Smith G, Altman DG. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Books; 2001.
Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000;19:3127–31.
Brooks SP, Gelman A. General methods for monitoring convergence of iterative simulations. J Comput Graph Stat. 1998;7:434–55.
Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Ser B. 2002;64:583–639.
Raghu G, Anstrom KJ, King J, Lasky JA, Martinez FJ. Prednisone, azathioprine, and N-acetylcysteine for pulmonary fibrosis. N Engl J Med. 2012;366:1968–77.
The Idiopathic Pulmonay Fibrosis Clinical Research Network. Randomized trial of acetylcysteine in idiopathic pulmonary fibrosis. N Engl J Med. 2014;370:2093–101.
Homma S, Azuma A, Taniguchi H, Ogura T, Mochiduki Y, Sugiyama Y, et al. Efficacy of inhaled N-acetylcysteine monotherapy in patients with early stage idiopathic pulmonary fibrosis. Respirology. 2012;17:467–77.
Richeldi L, Costabel U, Selman M, Kim DS, Hansell DM, Nicholson AG, et al. Efficacy of a tyrosine kinase inhibitor in idiopathic pulmonary fibrosis. N Engl J Med. 2011;365:1079–87.
Nathan SD, Shlobin OA, Weir N, Ahmad S, Kaldjob JM, Battle E, et al. Long-term Course and Prognosis of Idiopathic Pulmonary Fibrosis in the New Millennium. Chest. 2011;140:221–9.
Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011.
Mills EJ, Thorlund K, Ionnidis JPA. Demystifying trial networks and network meta-analysis. BMJ. 2013;346:f2914.
This research was funded from internal funds from the Southampton Health Technology Assessments Centre (SHTAC) at the University of Southampton, UK. We are grateful to Karen Welch, Information Specialist, SHTAC, University of Southampton, for generating and running the literature searches.
All authors declare no financial support for the submitted work from anyone other than their employer. KOR has been a member of advisory boards for both Intermune and Boehringer Ingelheim and has received consultancy fees and research travel grants from both companies for advisory board work. All other authors have no relationships with commercial entities that might have an interest in the submitted work.
EL developed the research protocol, assisted in the development of the search strategy, assessed studies for inclusion, extracted data from and quality assessed included studies, synthesised evidence, drafted and edited the final manuscript and is the guarantor for the study. VRC synthesised evidence, developed the NMA model and undertook the NMA, drafted the manuscript. DAS synthesised evidence, developed the NMA model and undertook the NMA, drafted the manuscript. JLC contributed to the development of the research protocol, assessed studies for inclusion, extracted data from and quality assessed included studies, synthesised evidence and drafted the manuscript. AJC contributed to developing the research protocol, extracted data from and quality assessed included studies, synthesised evidence, drafted the manuscript. KMAO drafted the manuscript background, assisted with the interpretation of the evidence, drafted the manuscript. All authors read and approved the final manuscript.
MEDLINE search strategy. Appendix. NMA Model code. Figure S1. FVC categorical (>10% decline). Table S2. Acute exacerbations reported in the included trials. Figure S2. Acute exacerbations. Table S3. All-cause mortality. Figure S3. All-cause mortality. Table S4. Respiratory mortality. Figure S4. Respiratory mortality.
About this article
Cite this article
Loveman, E., Copley, V.R., Scott, D.A. et al. Comparing new treatments for idiopathic pulmonary fibrosis – a network meta-analysis. BMC Pulm Med 15, 37 (2015). https://doi.org/10.1186/s12890-015-0034-y