Skip to main content

Table 1 Criteria for scoring each domain in the checklist

From: Evidence synthesis in pulmonary arterial hypertension: a systematic review and critical appraisal

Domains Weak Neutral Strong
Relevance At least three of the six checklist items suggested study shortcomings, for example omission of relevant therapies in the analysis, omission of relevant outcomes for evidence synthesis, or inclusion of patients outside the target population. 1–2 checklist items were not addressed satisfactorily; no or insufficient justification for a particular analysis approach was provided (e.g. inclusion of oral therapies only without justification). All checklist items were appropriately addressed.
Credibility Information omitted or insufficient information provided for at least three of the nine checklist items, for examples, omission of key databases in the SLR, omission of a quality assessment of included studies, or lack of identification of imbalances in the distribution of key effect modifiers prior to the analysis. 1–2 checklist items were not addressed satisfactorily, for example, an adequate search strategy but no transparent reporting of the full search strings, or lack of reporting of the results of the quality assessment. All checklist item were addressed appropriately. The checklist domain ‘credibility’ includes one question only applicable to NMA studies; this question was not considered for the domain grading of MA studies.
Analysis At least three of the 10 checklist items suggested study shortcomings, such as lack of subgroup analyses or meta-regression in cases of between-study heterogeneity, pooling of drug classes, treatments or doses without proper justification, or lack of a valid rationale for the use of random effects or fixed effect models. 1–2 checklist items were not addressed satisfactorily, such as insufficient detail on the statistical model. All checklist items were addressed appropriately. The checklist domain ‘analysis’ includes four questions only applicable to NMA studies; these questions were not considered for the domain grading of MA studies.
Reporting quality & transparency At least two of the six checklist items were not addressed satisfactorily, or discussion of the impact of important patient characteristics on treatment effects was not included. Insufficient information for one checklist item or a brief discussion of the impact of the impact of patient characteristics on analysis results was provided. All checklist items were addressed appropriately. The checklist domain ‘reporting quality & transparency’ includes four questions only applicable to NMA studies; these questions were not considered for the domain grading of MA studies.
Interpretation Results were not contextualized with consideration of limitations or specific treatments were endorsed over others despite a lack of discussion of between-study heterogeneity and/or despite pooling of active therapies. Study limitations (e.g. between-study heterogeneity) were provided however without a detailed discussion of the impact these may have had on observed study results. All these aspects were addressed appropriately.
Conflict of interest No information on conflicts of interest was provided, or details of author disclosures and contributions were insufficient. Disclosures as well as author contributions were clearly stated in cases of personal or financial relationships of affiliations that could have biased the work in question. No personal or financial relationships or affiliations (that could have biased the study) were declared.
  1. MA Meta-analysis, NMA Network meta-analysis, SLR Systematic literature review