We tried to synthesize the evidence for GINA step 3&4 recommendations for 5 to 18 year-olds with moderate persistent asthma. Our aim was to rank the 21 different GINA treatment options as to their effectiveness using standard systematic review methods extended by network meta-analytic techniques.
In principle, the situation looked favourable for network meta-analysis, with RCTs on six out of seven interventions either against placebo or head-to-head (Figure
2). Lack of direct comparisons, for example GINA 3C versus 4A, could have been compensated by indirect comparisons, for example through GINA 3B and placebo. Only theophylline was disconnected to the network of trials as we found no trials in this age group.
Due to extremely different choices trialists made on outcome reporting methods, we had to abandon attempts at meta-analysis. Apart from embarking on a set of concerted new trials in this area, which may take years to complete, a potentially quicker way to salvage the situation with existing data may be joint action among sponsors and trialists of existing trials to aggregate their raw data to inform an IPD meta-analysis
[42, 43]. The authors of this review would be more than happy to support such an endeavour, thereby achieving this review’s original aim. Such an exercise would depend also on the results of additional trialist-provided information on trial quality, since pooling of very low quality data is unattractive. This brings us to the next point. We assessed the risk of bias in the included trials on a 9-item methodological quality checklist. We scored “?” if the risk of bias seemed hard to determine. We scored 138/207 “?”, and this is largely due to partial, unclear or non-reporting (see Table
2). Adoption and enforcement of the CONSORT statement should become a priority for trialists and journals alike
After criticizing some of the outcome reporting methods, let us consider the strengths and limitations of our own work. We comprehensively searched the literature and tried to minimize the risk of missing RCTs by tracking the references of the GINA-guideline, included RCTs and relevant systematic reviews
[45–47]. However, these efforts yielded no additional relevant references. We performed all major steps, except the extraction of the quantitative data in duplicate. Furthermore, our team had expertise on all aspects of a systematic review: clinical librarian, biostatistician, physician-epidemiologist, two general practitioners, a trainee general practitioner, and a paediatric pulmonologist. Nevertheless, our review is no exception in that it may have been affected by suppression of negative trial results, or publication bias
As far as we are aware, a network meta-analysis on this subject would have been novel. The majority of the meta-analyses performed on these treatment options are combined for paediatric and adult patients. In 2003, Bisgaard analyzed the effect of long–acting β2-adrenoceptor agonists (LABA) on the asthma exacerbation rate in paediatric patients in a review of eight randomized trials
. All trials compared a LABA with a SABA or placebo in children on inhaled corticosteroids and reported on exacerbations or asthma-related hospitalizations in asthmatic children. Bisgaard, while providing the spectrum of relative risks, refrained from formal meta-analysis, because of differences in patient populations, comparators, study design and duration, and definitions of asthma exacerbation. He concluded that there is no evidence in the existing paediatric literature that LABA protects against asthma exacerbations, even when used as an add-on therapy to ICS.
In line with our view that firm evidence to guide step-up therapy is lacking, Lemanske et al. performed the BADGER trial, a three-period-cross-over trial in children eligible for GINA step 3
. The BADGER trial is clearly relevant to the topic of this review. The study addresses the research question which of the three medication options (doubling the dose of the inhaled corticosteroid, adding LABA or LTRA) should be the first choice of treatment in step 3 of the guidelines. Because of its importance to the research question of this review, we will discuss it in some more detail. The BADGER investigators assigned 182 children, from 6 to 17 years of age with uncontrolled asthma, despite receiving a low dose ICS to receive each of three blinded step-up therapies, corresponding with GINA step 3A, 3B and 3C, in random order for a period of 16 weeks each. Several clinical and physical aspects were measured, including the need for oral prednisone, an asthma control test and FEV1. Main outcome was that overall, LABA as add-on (GINA 3A) performed better than increasing ICS dose (GINA 3B) or adding LTRA (GINA 3C). Furthermore, subgroup analyses were performed to predict the direction of the patterns of differential response, primary on baseline values of PC20, Asthma Control Test scores and genotype, and, post hoc, on demographic and physiological characteristics. The only significant (p=0.009) predictor was the baseline Asthma Control Test scores (</≥19) on the probability of the best response to LABA step-up.
Strengths of the BADGER trial are the topical research questions and relevant outcomes measures. Furthermore, sensitivity analyses were performed to assess bias, for example seasonal differences. However, the treatment period-specific results were not reported separately, which was the main reason why we could not use the trial in this review with network meta-analysis. In addition, the study is hampered by the cross-over design with possible carry-over effects of ICS treatment. A wash-out period of four weeks makes using the second and third treatment periods hazardous due to unquantifiable carry-over effects
. Carry-over effect of ICS would have improved the treatment effects of adding LABA or LTRA. Furthermore, post hoc analysis with relatively small subgroups already raised much discussion and suggests hypotheses that need more research in studies with a different design
Although GINA provides us with treatment recommendations, steps 3&4 are still not based on sound evidence. For patients, their parents, and physicians alike, uncertainty about the best treatment remains. New trials should focus on add-on therapy to ICS in children. Ongoing and new RCTs will be part of meta-analysis in a few years. To interpret individual studies, consensus about design and reporting of outcome measurements for RCTs would provide a much better evidence base for the future. In 2009 an official American Thoracic Society/European Respiratory Society statement, about standardizing endpoints for clinical asthma trials and clinical practice was published
. A taskforce formulated recommendations of assessment for the design, conduct and evaluation of asthma trials for clinicians, researchers, and other relevant groups. These recommendations form an excellent starting point for harmonization of outcome measures and accompanying inferential statistical measures in RCTs and other comparative effectiveness research. As far back as 1992, Tugwell and Boers introduced a solution for Rheumatoid Arthritis Clinical Trials, OMERACT (“Outcome Measures in Rheumatoid Arthritis Clinical Trials”)
. OMERACT, an international informal network, strives to improve outcome measurement through a data driven, iterative consensus process involving relevant stakeholder groups. This type of initiative would be welcome in asthma research too.
Another solution may be prospective meta-analysis (PMA)
[17, 42]. PMA meta-analyses RCTs, preferably by using IPD, that were identified, evaluated and determined to be eligible for the meta-analysis before the results of any of those studies become known. PMA was developed to overcome some of the problems of normal (retrospective) meta-analyses, mainly to enable hypotheses to be specified a priori and ignorant of the results of individual trials. Ideally, PMA provides standardization of clinical trial procedures, such as study design and data collection methods, by using, for example, the same instruments and the same time points for measuring outcomes.