Effectiveness of computerized clinical decision support systems for asthma and chronic obstructive pulmonary disease in primary care: a systematic review

Background

The use of computerized clinical decision support systems may improve the diagnosis and ongoing management of chronic diseases, which requires recurrent visits to multiple health professionals, disease and medication monitoring and modification of patient behavior. The aim of this review was to systematically review randomized controlled trials evaluating the effectiveness of computerized clinical decision systems (CCDSS) in the care of people with asthma and COPD.

Methods

Randomized controlled trials published between 2003 and 2013 were searched using multiple electronic databases Medline, EMBASE, CINAHL, IPA, Informit, PsychINFO, Compendex, and Cochrane Clinical Controlled Trials Register databases. To be included, RCTs had to evaluate the role of the CCDSSs for asthma and/or COPD in primary care.

Results

Nineteen studies representing 16 RCTs met our inclusion criteria. The majority of the trials were conducted in patients with asthma. Study quality was generally high. Meta-analysis was not conducted because of methodological and clinical heterogeneity. The use of CCDSS improved asthma and COPD care in 14 of the 19 studies reviewed (74%). Nine of the nineteen studies showed statistically significant (p < 0.05) improvement in the primary outcomes measured. The majority of the studies evaluated health care process measures as their primary outcomes (10/19).

Conclusion

Evidence supports the effectiveness of CCDSS in the care of people with asthma. However there is very little information of its use in COPD care. Although there is considerable improvement in the health care process measures and clinical outcomes through the use of CCDSSs, its effects on user workload and efficiency, safety, costs of care, provider and patient satisfaction remain understudied.

Chronic respiratory diseases, particularly asthma and chronic obstructive pulmonary disease (COPD), kill more than four million people every year world-wide and affect hundreds of millions more [1]. Around 300 million people suffer from asthma world-wide, with a projected increase of an additional 100 million people by 2025 [1]. The economic burden of asthma has been estimated to be the highest among chronic diseases [2] and includes both direct (e.g. hospital admissions and costs of medications) and indirect costs (e.g. days away from work) [2, 3]. The Global Burden of Disease Study projected that COPD, which ranked sixth as a cause of death in 1990, will become the third leading cause of death in 2030 [4].

Effective management of chronic diseases requires optimal dissemination and implementation of guidelines, however there is a gap between scientific evidence-based medicine and real clinical practice, especially in primary care [5]. Although effective therapies and guidelines are available, many patients with asthma still have frequent, uncontrolled symptoms and do not receive optimal care. Research demonstrates that only a quarter of patients with persistent asthma symptoms take anti-inflammatory medications as recommended by the guidelines [6]. Much of the cost of asthma care is attributable to poor disease control due to non-adherence to guideline-recommended controller therapies [7, 8], over reliance on reliever medication [9], inadequate monitoring of disease severity and insufficient patient education for effective self-management [10].

Similarly, the care provided for patients with COPD in community settings indicates low level of awareness and implementation of guidelines [11–13], despite the high level of evidence for the efficacy of guideline-based interventions. Medication use is often not in accordance with the guidelines [14] and a high proportion of patients prescribed with inhalers use them incorrectly [15, 16]. Smoking cessation can reduce the rate of decline in lung function, yet many with COPD continue to smoke. Influenza and pneumococcal vaccinations can reduce the rate of exacerbations, hospitalizations and death [12]. However, in Australia, for example, based on the 2004-05 National Health Survey 25% and 59% of those with self-reported COPD had never been administered influenza or pneumococcal vaccinations respectively [3].

Globally, studies evaluating the provision of care by clinicians suggest that evidence-based care was delivered approximately 40-55% of the time [17–19]. The reasons for sub-optimal uptake of guidelines into practice are complex and occur at the patient, provider and system levels [20]. Given, the rising global disease burden from asthma and COPD and intractable health system deficits in providing evidence based care there is a pressing need to identify systems-focused solutions. Computerized Clinical Decision Support System (CCDSS) is well established as one strategic method of improving care for prevention and management of chronic conditions. A CCDSS is “any electronic information system based on a software algorithm designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration” [21]. CCDSS is valuable not only to the clinicians, but can also provide other health care providers, patients, or caregivers with clinical knowledge and patient-specific information to help them make decisions that enhance patient care [22]. Typically CCDSS interventions include forms and templates for entering and documenting patient information, and alerts, reminders, and order sets for providing suggestions and other support.

Importantly, CCDSS interventions can increase adherence to evidence-based medical knowledge, reduce unnecessary variation in clinical practice and improve their clinical decision-making process [21, 23].

CCDS systems that are well designed and implemented have the potential to improve health care quality, increase efficiency by reducing mental workload, improve clinical work-flow and reduce health care costs [23, 24]. CCDSS has been used in the management of various chronic conditions such as diabetes [25, 26], hypertension [27, 28], dyslipidemia [29, 30] and cardiac care [31] across various health care settings.

Although there have been reviews of the effectiveness of CCDSSs in the management of various disease states in different clinical settings [32, 33], there have been no systematic appraisals of their impact on chronic respiratory diseases such as asthma and COPD in primary care. Our systematic review aimed to synthesize evidence for its use in the care of patients with asthma and COPD and to identify the key features of those systems that have the potential to overcome health system barriers and improve outcomes.

CCDSS definition

CCDSS was defined as an automated process for comparing patient-specific characteristics against a computerized knowledge base with resulting recommendations or reminders presented to the provider (or the user) to consider, to help them in clinical decision making.

Search strategy

The electronic databases MEDLINE, EMBASE, CINAHL, IPA, Informit, PsychINFO, Compendex, and Cochrane Clinical Controlled Trials Register databases were reviewed by the primary author. MeSH terms ‘clinical decision support systems/tools/techniques/aids/guidelines’, ‘computer assisted therapy/diagnosis/decision making’, ‘computeris(z)ed decision making’, ‘CCDSS’, ‘medical informatics’, ‘asthma’, ‘COPD’ and combinations thereof were included. The detailed search strategy that was used in MEDLINE is outlined in Table 1. This search strategy was repeated in all other databases. We also systematically searched the reference lists of all the included studies and relevant reviews.

Inclusion criteria

•  Empirical studies published in the English language between 2003 to May 2013;

•  Pediatric or adult CCDSS interventions involving COPD and/or asthma screening, prevention, case detection, and management;

•  Randomized controlled trials comparing CCDSS with explicitly defined clinical or process outcome measures;

•  CCDS system used by any clinicians (physicians, physician assistants, pharmacists, dentists, pulmonary specialists or nurse practitioners) directly involved in patient care;

•  CCDSS targeting patients in improving self-management.

Exclusion criteria

•  Review articles, conference proceedings, meeting abstracts;

•  Paper-based tools (e.g. flow charts and non-electronic clinical pathway tools);

•  CCDSS interventions in people with other conditions (rather than asthma and COPD), including other respiratory diseases;

•  CCDSS for medical education purposes or only providing summaries of information for patients;

•  Group based interventions that did not include individual clinical assessment;

•  Evaluations which focused only on the technical performance of the system as opposed to its effect on clinical practice;

•  In-patient hospital based systems.

Study selection

Two authors (MF and PNP) independently reviewed the titles, index terms, and abstracts of the identified references and rated each paper as “potentially relevant” or “not relevant” based on study design, subjects, setting, and intervention. These two authors then independently reviewed the full texts of the selected potentially relevant articles and again rated each paper as “potentially relevant” or “not relevant”. After application of the full set of inclusion and exclusion criteria to the potentially relevant studies, a further limitation was then applied, and only RCTs were included. Disagreements between reviewers were resolved by discussion with a third author (CA) until consensus was reached.

Data extraction and quality assessment

The primary author (MF) independently extracted data related to Participants, Intervention, Comparator, Outcomes and Study design by utilizing the PICOS strategy for describing trials (Table 2). The second author (PNP) then independently examined the studies and extracted data to confirm accuracy. The data abstracted included the following information: manuscript authors, year of publication, the study design and duration, participant characteristics (health practitioners, patients), the type of CCDSS intervention, the comparator (usual care or another form of CDSS) and the outcomes measured (clinical, process, workload and efficiency, economic and implementation). Bias was assessed using the Cochrane risk of bias tool [34], and was based on the following five dimensions: randomization, allocation concealment, blinding of participants, personnel or outcome assessors, selective outcome reporting and completeness of follow-up [duration of follow-up, intention to treat (ITT) analysis, withdrawals, and reasons for dropouts]. Each of the above attributes was assessed as being high, low or unclear and an overall risk of bias was reached for each of the included studies (Table 3).

Assessment of intervention effects

Type of CCDSS intervention provided

Trials were organized into three categories based on the type of CCDSS intervention provided (Table 4):

1. 1)

   Diagnostic advice only;

2. 2)

   Drug therapy management only; and

3. 3)

   ‘Multi-faceted’ interventions comprising two or more different intervention components.

Effectiveness of CCDSS

Using classifications published in previous reviews pertaining to CCDSS [54], we assessed CCDSS effectiveness based on the following key outcomes:

1. (1)

   Clinical outcomes: Morbidity, health related quality of life, hospitalizations and mortality. [e.g. asthma symptoms (measured using symptom diary), asthma control (Asthma control Questionnaire-ACQ), lung function (Piko-1 device, peak flow meter), health-related quality of life (measured using HRQOL) and adverse events (leading to unscheduled doctors visit or hospitalization)];

2. (2)

   Healthcare process measures: Recommended preventive care services ordered or completed (e.g. influenza vaccination), recommended clinical study ordered or completed (including spirometry), recommended treatment ordered or completed (including rescue medication prescriptions and antibiotic prescriptions)];

3. (3)

   User workload and efficiency outcomes: Effect on user knowledge, number of patients seen per unit time, clinician workload, and efficiency;

4. (4)

   Relationship-centered outcomes: Patient satisfaction surveys;

5. (5)

   Economic outcomes: Cost and cost effectiveness of the CCDSS used; and

6. (6)

   Use and Implementation outcomes: Health care provider acceptance, health care provider satisfaction, and health care provider use and implementation.

A CCDSS was considered effective if it produced a statistically significant (p <0.05) improvement in the primary outcome or improvement in ≥50% of multiple relevant pre-specified outcomes. If the authors did not designate a primary outcome, we considered the outcome used to calculate the trial’s sample size to be primary. Studies that included multiple intervention arms were considered effective if any of the CCDSS based treatment arms was evaluated as effective.

Although we had intended to conduct meta-analyses, this was abandoned owing to the marked heterogeneity in participants, clinical settings, interventions, and the outcomes measured in the included studies. However, effect sizes (Cohen’s d value) of the significant primary outcomes were calculated wherever possible.

The PRISMA guidelines for conducting/reporting systematic reviews were consulted and a completed checklist is attached as Additional file 1. We screened 1042 abstracts, identified 173 full-text potentially relevant articles and included 19 articles representing 16 RCTs in the review (Figure 1).

PRISMA flow diagram of the included and excluded studies[55].

Full size image

Studies were predominantly conducted in the Netherlands (n = 9) and the USA (n = 7) with one study conducted in each of the following countries: Australia, United Kingdom and Denmark.

Eleven studies evaluated asthma care [35–45], 5 studies involved patients with asthma and COPD [46–50], and 3 studies focused on people with medically complex conditions including COPD [51–53]. There were no studies conducted exclusively on people with COPD (Table 2).

Study quality

A summary of the study quality of the included studies is reported in Table 3. Of the 19 trials, 10 studies had a low risk of bias [36–39, 44–46, 51–53]. Eleven studies described an appropriate method of sequence generation [35–38, 44–46, 50–53], 9 studies reported adequate concealment of allocation [36–39, 42, 44, 46, 50, 53], and 13 studies showed either no differences in baseline characteristics between study groups or performed appropriate adjustments [35–37, 39, 40, 42–46, 50, 51, 53]. Eleven studies used objective outcomes or blinding of outcome assessments [36, 39, 40, 43–45, 48, 50–53], and 11 studies achieved a ≥90% follow-up rate for their unit of analysis [35–39, 44–46, 48–50].

CCDSS and study characteristics

Table 2 describes the CCDSS design and implementation characteristics. The majority of interventions (68%) (13/19) were embedded in an existing electronic medical record (EMR) or with the computerized physician order entry (CPOE) systems [35, 39–41, 43, 44, 46–52]. Thirty-one percent (6 studies) had a stand-alone system, of which four were internet based [36–38, 42] and in the other 2 studies CCDSS intervention were administered to practitioners by the study researchers [45, 52]. Five of the six studies with the stand-alone CCDS system showed positive impact. Sixty-three percent (12 studies) automatically pre-populated the EMR data [35–41, 43, 44, 46, 50, 53], 26% (5 studies) relied on practitioners to manually enter the data [42, 47–49, 52], 16% (3 studies) relied on research staff [44, 45, 51] and 21% (4 studies) relied on patients as well for data inputs [36–38, 42]. Forty-seven percent (9 studies) compared a computerized clinical decision support system directly with usual care [35–38, 40, 43, 44, 50, 52].

Advice at the point-of-care was provided in 14 trials [35, 39–41, 43, 44, 46–53] and via the internet in 4 trials [36–38, 42]. Advice in the form of a computer-generated letter recommending changes to the treatment was provided to the practitioners in one study [45]. Advice was provided only to the physicians in 68% (13 trials) [35, 39–41, 43, 44, 46–52], while only one trial involved provision of advice to other healthcare practitioners (pharmacists) in addition to physicians [46]. In 26% (5 studies) patients were directly advised in addition to practitioners [36–38, 42, 53]. Thirteen studies provided explicit training in use of the CCDSS to healthcare practitioners [39, 41–52], while patients were trained to use the internet based CCDSS in 3 trials [36–38]. The CCDSS user interface characteristics were described in only 42% (8) of the trials [35, 38, 39, 43, 48–50, 53].

Studies included a highly varied number of healthcare practitioners, patients and health services.

Since CCDSS is primarily focused on altering provider behavior, the unit of randomization in most CCDSS studies was the provider. Thirty-seven percent (7) of the studies reviewed were randomized at the provider, practice or community level [39, 45, 46, 50–53], while 31% (6) used cluster randomization either between clinics or groups of providers that worked closely together [40, 41, 44, 47–49]. Thirty-one percent (6 studies) were randomized at the patient level [35–38, 42, 43].

In all studies that included patient level randomization there was potential for contamination given a single provider could care for both intervention and control arm patients. The principal summary measures used to compare effects between intervention and control groups varied and included: proportions [35, 39, 41, 45]; difference in medians [36, 38, 43, 50]; difference in means [46, 48]; relative risk [37] and odds ratios [40, 42, 44, 51–53].

Three trials did not clearly report their source of funding [41, 42, 53]. Of the remaining, 9 trials were publicly funded [36, 37, 41, 43, 45, 46, 51–53], 5 trials were conducted with only private funds [40, 44, 47–49], 1 trial was conducted with a combination of private and public funding [42], while another trial did not receive any funding [35]. Five trials declared that at least one author was involved in the development of the CCDS system [37, 38, 47–49], while the remaining trials did not indicate at all if the authors were independent of development.

Type of CCDSS interventions

The included studies utilized CCDSS for a variety of purposes, and were categorized in to three main categories, such as those focusing on screening/diagnosis, drug therapy management, and multifaceted interventions which involved various aspects of disease management along with self-management advice (Table 4).

There was only one study, conducted by Caroll et al. that used the CCDSS (Child Health Improvement through Computer Automation-CHICA system) for the purpose of diagnosing pediatric asthma [35]. Five studies (26%) used the CCDSS for drug therapy management [36, 45, 47–49], and thirteen used a multi-faceted form of CCDSS [37–44, 46, 50–53].

The studies evaluating the CCDSS focusing on drug therapy management included the study by Hashimoto et al. which utilized an Internet based treatment decision support system to guide people with severe asthma in tapering the dose of oral corticosteroids depending on their asthma control [36]. The study by Kattan et al. involved provision of a computer-generated letters to the treating physician summarizing the appropriate treatment recommendations based on the child’s asthma symptoms, health service and medication use [45]. The three studies conducted by Martens et al. also involved a CCDSS in the form of reactive computer reminders (CRS) to improve drug prescribing in general practice [47–49].

The remaining 13 studies (68%) evaluated multifaceted forms of CCDSS [37–44, 46, 50–53]. These CCDSSs ranged from simple activation of electronic alerts to identify people at risk of an asthma exacerbation [44], or prompts to alert the physician to modify treatment in people with medically complex conditions at the point of care [53], to more complex forms of CCDSS interventions involving a series of care suggestions on drug therapy and disease management [46]. Internet-based multifaceted CCDSS interventions were evaluated in three trials which focused on the self-management of asthma [37, 38, 42]. These studies utilized online self-management programs which involved weekly online asthma control monitoring and feedback in the form of treatment advice by a specialized asthma nurse [37, 38], or by the patients’ physician [42]. Another type of multifaceted CCDS interventions evaluated in two other studies were in the form of EHR-based clinical alerts either to improve influenza vaccination in children with asthma [40], or to improve overall asthma care in these children [41]. The multifaceted CCDSS evaluated by Kuilboer et al. was a critiquing system integrated with the general practitioners’ electronic medical records which reviewed physicians’ treatment decisions and generated feedback [50]. Another two studies evaluated the impact of an expert spirometry system on the physician’s decision making during asthma diagnosis [51] and during management [52]. The remaining two trials also tested the effects of another multifaceted CCDSS on the clinician’s performance in the form of an electronic interface system to manage asthma patients in ED [39] and in the form of an automated asthma detection system used to identify and manage people at risk of asthma exacerbation in the emergency department [43].

CCDSS effectiveness

There was marked variability in the outcomes reported. Therefore we assessed the effectiveness of the CCDSS on the primary outcomes measured. In majority of the trials reviewed, the primary outcomes assessed were health care process measures, clinical outcomes, user work load and efficiency, and use and implementation outcomes. Relationship-centered outcomes and economic outcomes were measured by few trials, but only as secondary outcomes. Fourteen trials (74%) showed positive effect from the use of CCDSS on the primary outcome measured, of these 9/19 (47.3%) showed a significantly positive effect [35, 36, 38, 39, 41, 42, 45, 50, 53].

Clinical outcomes

The different clinical outcomes reported in the studies included asthma symptoms [42], asthma/COPD symptoms [46] asthma control (ACQ) [36–38], Health related Quality of life [36, 37, 42, 46], frequency of health care utilizations including hospitalizations [36, 44], admission rate and ED length of stay [43], frequency of exacerbations [36, 37, 44], lung function (FEV1) in asthma patients [36, 37, 42], exhaled nitric oxide [36], symptom free days [37], airway hyper responsiveness [42], number of ED visits [45, 46] number of school days missed [45], medication adherence [46], FEV1 and peak flow measurements in asthma/COPD patients [50].

Five of the nineteen trials assessed clinical outcomes as the primary outcome measure [35, 37, 38, 42, 44], of which three showed clinically significant improvements [35, 38, 42], one showed a positive but modest improvement in the asthma related quality of life [37] and another one did not show any effect on the number of people experiencing an exacerbation from the use of EHR embedded asthma risk alerts [44].

Significant improvement was found in the rate of diagnosis of asthma in children by implementation of a parent survey linked to physician prompts using computer decision support system called the CHICA system [35]. Significant improvement was also found in asthma control measured weekly using the asthma control questionnaire (ACQ) [38], in asthma symptoms using an electronic diary to record symptoms daily and in the asthma quality of life measured using asthma quality of life questionnaire (AQLQ) [42]. The effect sizes (Cohen’s d) calculated for the studies showing significant improvement in the primary clinical outcomes ranged from 0.24 to 0.94, with three studies showing a reasonably large effect size [37, 38, 42].

Health care process outcomes

The different health care process measures that were assessed in the reviewed trials included change in the consumption of oral corticosteroid [36], change in the dose of inhaled corticosteroid [38, 42], change in patients’ asthma knowledge [37], change in inhaler technique [37], change in medication adherence [37], medication changes [37, 38, 44], adherence to the use of ACQ [38], rate of vaccination [40], number of corticosteroid prescription ordered [41], provision of asthma action plan [41], spirometry ordered [41], rate of asthma documentation by ED doctors [43], scheduled physician visits leading to change in medication dose [45, 50], physicians adherence to guidelines [46], change in the number of prescriptions [47, 48, 50], change in the diagnostic ability of the general practitioner [51, 52], diagnostic tests ordered [51, 52] and the rate of referral [52].

Ten trials assessed health care process measures as the primary outcome [36, 40, 41, 45–48, 50–52], of which four showed significant improvement in these outcomes. Significant improvement was seen in process outcomes like cumulative sparing of prednisone dose adjusted weekly according to the internet based CDSS [36], percentage of children given at least one prescription of corticosteroid [41], percentage of visits to the physician leading to medication step up of asthma medication [45] and in the number of contacts with the patients’ physician [50]. The effect size calculated for the two studies [36, 45] with significantly positive improvement was however poor. Three trials showed a positive but modest effect of which one showed a modest improvement in the rate of influenza vaccination by the use of EHR alerts [40]. The other 2 trials showed a modest improvement in the drug prescribing behavior of GPs from the use of the CRS reminders [47, 48]. The remaining 3 studies did not show any effect from the use of CCDSS on the primary health care process outcome assessed.

User workload and efficiency outcomes

Workload and efficiency outcomes assessed in the trials included asthma documentation by emergency department (ED) doctors [39], consultation time [39], time for disposition decision in the ED [43], and user knowledge [49]. These outcomes were assessed as the primary outcome by only two trials [39, 43], of which one trial showed significant improvement in the rate of asthma documentation by the ED doctors in the management of acute asthma [39]. The size of the effect calculated for this trial was relatively large (Cohen’s d =0.78). However the other trial did not show any effect from the use of CCDSS on the time taken by the ED physicians to make a disposition decision [43].

Use and implementation outcomes

The outcomes assessed under this category were physicians attitude to guidelines [46], user friendliness [48, 49], provider satisfaction [49], and the rate of accessing guidelines [53]. Two trials assessed these outcomes as the primary outcome, of which one showed a significant improvement in the rate of use of guidelines by dentists in the management of people with chronic diseases including COPD [53]. The study showed that the use of CCDSS increased the number of times the dentists accessed the guidelines. The other trial also showed a positive but modest effect in the use of CRS (reactive computer reminders) by general practitioners, not to prescribe certain drugs [49].

Other outcomes

Outcomes such as patient satisfaction with CCDSS use was measured as a secondary outcome by two trials and found no difference in patient satisfaction between the intervention and the control group patients [36, 46]. Measures such as health care provider satisfaction were also assessed as secondary outcomes. Of the three studies that measured these outcomes [46, 47, 49], two found that the provider perceived the CCDSS as user friendly. Only one trial measured cost of the intervention and found that the patients in the group receiving the CCDSS intervention had significantly elevated total health care charges [46]. Two other trials measured the cost-effectiveness of the CCDSS used and both found that its use was more cost effective than usual asthma care [44, 45].

This is the first comprehensive review of CCDSS in the care of patients with chronic respiratory diseases, asthma and COPD. The review focused only on studies conducted in primary care as the bulk of the management of these chronic diseases happens in primary care. The review found that the use of CCDSS can have a positive impact on the diagnosis and management of asthma and COPD in primary care. Overall 74% of the studies reviewed showed improvement in the primary outcomes. Although there is literature available on the use of CCDSS in patients with asthma, there is very little literature on its use in the management of people with COPD.

The review also found that 83% (5/6) of the studies that utilized CCDSS with a stand-alone design showed positive outcomes as compared to studies testing CCDSS which were integrated with the EHR or the EMR systems (38%) (5/13). This indicates that systems presenting advice within electronic health records or order entry systems were much less likely to improve care or outcomes than stand-alone programs. It has been found that when integration of alerts within an institution’s electronic health records becomes possible and more alerts are added, practitioners might become overwhelmed and begin to ignore the prompts. This “alert fatigue” phenomenon [56] could be responsible for limiting behavior change. Studies estimate that as many as 96% of alerts are over ridden [57–59] and suggest that the threshold for alerting is too low (that is, alerts are sensitive but not specific). Systems requiring the practitioner to give a reason for over-riding advice were more likely to succeed than systems missing this feature [60].

Four of the five studies evaluating CCDSS with a stand-alone design, were Internet-based interventions targeting both physicians and patients. All the four studies showed that CCDSSs which targeted the patients as well as the physicians were effective in improving outcomes. The findings are consistent with other previous reviews of CCDSS for chronic disease management in primary care [33, 61]. A key feature of these interventions was the active incorporation of a patient self-management component for use outside of the clinical encounter. The CCDSS interventions included in the studies involved regular monitoring and feed-back along with patient education and follow-up. These results confirm the value of collaborative care in chronic respiratory disease management.

Also CCDSS interventions consisting of multiple components such as reminders and education were associated with greater improvement in outcomes than single-target interventions with fewer components. This is also reflected in other reviews evaluating the effectiveness of such multi-component CCDSSs engaging patients in the management of other chronic conditions like diabetes [33] and osteoporosis [62]. Given the advent of personal health records, patient portals, and mobile applications aimed at better engaging patients, the findings suggest that there is a need to consider multiple components and targets in the development of any future interventions.

Of the outcomes measured many of the included studies (53%) often focused on measuring the effectiveness of CCDSSs on the health care process outcomes and the evidence demonstrating the effects of CCDSSs on patient outcomes, user workload and efficiency and economic outcomes remains surprisingly low. This is comparable to other recent CCDSS reviews which also report on the paucity of well-designed studies evaluating the effects of CCDSS on patient related outcomes [32, 63]. This may have occurred owing to under powering, since most of the studies may not have had large enough sample sizes to detect such outcomes. Similarly many of these studies were not conducted over longer time frames. Both the sampling and time issues were possibly due to the relative difficulty of implementing randomized, controlled trials in real clinical settings [54]. Since clinical decision support has a primary function aimed at providing information to the provider at the point of decision making and intervention, outcomes which measure process or provider behavior are often used as a proxy for patient outcomes [61]. Although analysis of process outcomes has a merit as an interim platform to justify the continuing role of CCDSS in clinical care, more research is needed on evaluating the effectiveness of CCDSS on patient outcomes in order to adequately understand the usefulness of CCDSS in clinical setting. Nevertheless, 60% of the 5 studies measuring clinical outcomes showed significantly positive impact on these outcomes as compared to 40% of the 10 studies showing significant improvement in health care process outcomes. This implies that the implementation of CCDSS for asthma/COPD care seems promising in improving clinical outcomes. The most commonly reported clinical outcomes were asthma control and asthma quality of life.

CCDSSs may represent a cost-effective way of improving chronic respiratory disease outcomes in primary care. However, the review found that the economic effects of these systems could not be readily assessed based on the available data. The costs of design, local implementation, ongoing maintenance, and user support can be high, and may be further elevated by the unique nature of chronic respiratory care. This warrants cost-effectiveness analyses, but only two trials reported such data and little cost data of any kind was available across studies. Almost all of the studies discussed the need for more research utilizing cost-effectiveness outcomes to better assess the long term effectiveness of CCDSS.

The review also found that there were no studies that demonstrated a negative finding (patient harm or deterioration related to the intervention). This could be because the studies did not actively collect any data on harm assessment of the CCDSS used. Prospective data on the possible harms of CCDSSs are needed to facilitate informed adoption decisions. Based on the available evidence it is hard to draw conclusions about the potential negative effect of implementing decision-support tools, which is necessary to truly fulfil the goal of evaluating these interventions and to better address implementation challenges [64].

Strengths and limitations

The review has several important strengths. This is the first review evaluating the role of CCDSS in the management of chronic respiratory diseases, asthma and COPD in primary care. We particularly excluded studies regarding in-patient hospital based CCDSSs as we intended to focus its effectiveness in primary and community health care, given that only a small proportion of people with, asthma for example, are managed in the hospital setting. The search strategy of our study was extensive and thorough, and covered a large number (eight) of relevant databases to identify potentially relevant studies. The other strength is we based our review on the strongest studies available, RCTs. Also to reduce the risk of selection bias and incorrect categorization all the included articles were analyzed and critically examined by three reviewers independently.

There are a number of key limitations to this review. We excluded studies regarding in-patient hospital based CCDSSs as we intended to focus its effectiveness in primary and community health care. We included only English language studies conducted in the last 10 years as we wanted to document the recent advances in this area. Our analyses were limited to published reports of randomized controlled trials, so the possibility of publication bias or selective reporting must be acknowledged. The CCDSSs were grouped into categories based on clinical applications rather than on other aspects of CCDSS design. We were unable to conduct meta-analysis, given the substantial heterogeneity in the type of CCDSSs and the outcomes evaluated, however we calculated the effect sizes of the primary outcomes for easier comparison of the study effects. Finally, we summarized only randomized controlled trials which might have resulted in less information about issues related to CCDSS implementation, effect on workflow, and factors affecting usability.

In summary, the review demonstrates that CCDSS can improve chronic disease management processes and clinical outcomes in patients with asthma and COPD, but data showing its effect on use and implementation and economic outcomes were sparse. The review also found that although there are a growing number of RCTs that assessed a wide variety of CCDSSs designed to improve asthma management in primary care, there is very scant evidence of its use in the care of patients with COPD.

The mechanisms behind systems’ success or failure remain understudied, but non-integrated, multifaceted CCDSS providing advice to both practitioners and patients, and those requiring the practitioners to give explanations for over-riding advice might independently improve success.

Future trials with clear descriptions of system design, local context, implementation strategy, costs, adverse outcomes, user satisfaction, and impact on user workflow will better inform CCDSS development and decisions about local implementation.

COPD:

Chronic Obstructive Pulmonary Disease

CCDSS:

Computerized clinical decision support systems

EMR:

Electronic medical records

EHR:

Electronic health records

CRS:

Computer reminder system

CPOE:

Computerized physician order entry

EDR:

Electronic dental records.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Affiliations

1. Sydney Medical School, University of Sydney, Sydney, New South Wales, Australia

   • Mariam Fathima
   • , David Peiris
   •  & Carol Lyn Armour

2. Woolcock Institute of Medical Research, Sydney Medical School, University of Sydney, Sydney, New South Wales, Australia

   • Mariam Fathima
   • , Pradnya Naik-Panvelkar
   •  & Carol Lyn Armour

3. The George Institute for Global Health, Sydney Medical School, University of Sydney, Sydney, New South Wales, Australia

   • David Peiris

4. Faculty of Pharmacy, University of Sydney, Sydney, New South Wales, Australia

   • Bandana Saini

Corresponding author

Correspondence to Mariam Fathima.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MF was responsible for the study conception and design and undertook the searches, selection and quality assessment of the studies, extraction and interpretation of data. PNP duplicated the searches, conducted selection and quality assessment of the studies. MF wrote the initial draft of the paper. DP contributed to the interpretation of the findings. DP, CA and BS critically revised the manuscript. All authors contributed to and have approved the final text.

Reprints and Permissions