Effectiveness of computerized clinical decision support systems for asthma and chronic obstructive pulmonary disease in primary care: a systematic review
The use of computerized clinical decision support systems may improve the diagnosis and ongoing management of chronic diseases, which requires recurrent visits to multiple health professionals, disease and medication monitoring and modification of patient behavior. The aim of this review was to systematically review randomized controlled trials evaluating the effectiveness of computerized clinical decision systems (CCDSS) in the care of people with asthma and COPD.
Methods
Randomized controlled trials published between 2003 and 2013 were searched using multiple electronic databases Medline, EMBASE, CINAHL, IPA, Informit, PsychINFO, Compendex, and Cochrane Clinical Controlled Trials Register databases. To be included, RCTs had to evaluate the role of the CCDSSs for asthma and/or COPD in primary care.
Results
Nineteen studies representing 16 RCTs met our inclusion criteria. The majority of the trials were conducted in patients with asthma. Study quality was generally high. Meta-analysis was not conducted because of methodological and clinical heterogeneity. The use of CCDSS improved asthma and COPD care in 14 of the 19 studies reviewed (74%). Nine of the nineteen studies showed statistically significant (p < 0.05) improvement in the primary outcomes measured. The majority of the studies evaluated health care process measures as their primary outcomes (10/19).
Conclusion
Evidence supports the effectiveness of CCDSS in the care of people with asthma. However there is very little information of its use in COPD care. Although there is considerable improvement in the health care process measures and clinical outcomes through the use of CCDSSs, its effects on user workload and efficiency, safety, costs of care, provider and patient satisfaction remain understudied.
Keywords
CCDSSComputerized clinical decision support systemsAsthmaCOPDComputerized clinical decision makingSystematic review
Background
Chronic respiratory diseases, particularly asthma and chronic obstructive pulmonary disease (COPD), kill more than four million people every year world-wide and affect hundreds of millions more [1]. Around 300 million people suffer from asthma world-wide, with a projected increase of an additional 100 million people by 2025 [1]. The economic burden of asthma has been estimated to be the highest among chronic diseases [2] and includes both direct (e.g. hospital admissions and costs of medications) and indirect costs (e.g. days away from work) [2, 3]. The Global Burden of Disease Study projected that COPD, which ranked sixth as a cause of death in 1990, will become the third leading cause of death in 2030 [4].
Effective management of chronic diseases requires optimal dissemination and implementation of guidelines, however there is a gap between scientific evidence-based medicine and real clinical practice, especially in primary care [5]. Although effective therapies and guidelines are available, many patients with asthma still have frequent, uncontrolled symptoms and do not receive optimal care. Research demonstrates that only a quarter of patients with persistent asthma symptoms take anti-inflammatory medications as recommended by the guidelines [6]. Much of the cost of asthma care is attributable to poor disease control due to non-adherence to guideline-recommended controller therapies [7, 8], over reliance on reliever medication [9], inadequate monitoring of disease severity and insufficient patient education for effective self-management [10].
Similarly, the care provided for patients with COPD in community settings indicates low level of awareness and implementation of guidelines [11–13], despite the high level of evidence for the efficacy of guideline-based interventions. Medication use is often not in accordance with the guidelines [14] and a high proportion of patients prescribed with inhalers use them incorrectly [15, 16]. Smoking cessation can reduce the rate of decline in lung function, yet many with COPD continue to smoke. Influenza and pneumococcal vaccinations can reduce the rate of exacerbations, hospitalizations and death [12]. However, in Australia, for example, based on the 2004-05 National Health Survey 25% and 59% of those with self-reported COPD had never been administered influenza or pneumococcal vaccinations respectively [3].
Globally, studies evaluating the provision of care by clinicians suggest that evidence-based care was delivered approximately 40-55% of the time [17–19]. The reasons for sub-optimal uptake of guidelines into practice are complex and occur at the patient, provider and system levels [20]. Given, the rising global disease burden from asthma and COPD and intractable health system deficits in providing evidence based care there is a pressing need to identify systems-focused solutions. Computerized Clinical Decision Support System (CCDSS) is well established as one strategic method of improving care for prevention and management of chronic conditions. A CCDSS is “any electronic information system based on a software algorithm designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration” [21]. CCDSS is valuable not only to the clinicians, but can also provide other health care providers, patients, or caregivers with clinical knowledge and patient-specific information to help them make decisions that enhance patient care [22]. Typically CCDSS interventions include forms and templates for entering and documenting patient information, and alerts, reminders, and order sets for providing suggestions and other support.
Importantly, CCDSS interventions can increase adherence to evidence-based medical knowledge, reduce unnecessary variation in clinical practice and improve their clinical decision-making process [21, 23].
CCDS systems that are well designed and implemented have the potential to improve health care quality, increase efficiency by reducing mental workload, improve clinical work-flow and reduce health care costs [23, 24]. CCDSS has been used in the management of various chronic conditions such as diabetes [25, 26], hypertension [27, 28], dyslipidemia [29, 30] and cardiac care [31] across various health care settings.
Although there have been reviews of the effectiveness of CCDSSs in the management of various disease states in different clinical settings [32, 33], there have been no systematic appraisals of their impact on chronic respiratory diseases such as asthma and COPD in primary care. Our systematic review aimed to synthesize evidence for its use in the care of patients with asthma and COPD and to identify the key features of those systems that have the potential to overcome health system barriers and improve outcomes.
Methods
CCDSS definition
CCDSS was defined as an automated process for comparing patient-specific characteristics against a computerized knowledge base with resulting recommendations or reminders presented to the provider (or the user) to consider, to help them in clinical decision making.
Search strategy
The electronic databases MEDLINE, EMBASE, CINAHL, IPA, Informit, PsychINFO, Compendex, and Cochrane Clinical Controlled Trials Register databases were reviewed by the primary author. MeSH terms ‘clinical decision support systems/tools/techniques/aids/guidelines’, ‘computer assisted therapy/diagnosis/decision making’, ‘computeris(z)ed decision making’, ‘CCDSS’, ‘medical informatics’, ‘asthma’, ‘COPD’ and combinations thereof were included. The detailed search strategy that was used in MEDLINE is outlined in Table 1. This search strategy was repeated in all other databases. We also systematically searched the reference lists of all the included studies and relevant reviews.
Table 1
Search strategy for Medline
Search terms used
1.
exp Decision Support Systems, Clinical/
2.
clinical decision support systems.tw.
3.
clinical decision support tool*.tw.
4.
clinical decision support system*.tw.
5.
Decision Support Techniques/
6.
Medical Order Entry Systems/
7.
Decision Making, Computer-Assisted/
8.
Diagnosis, Computer-Assisted/
9.
clinical decision support aid*.tw.
10.
clinical decision support guideline*.tw.
11.
computer assisted therap$.tw.
12.
Therapy, Computer-Assisted/
13.
reminder system*.tw.
14.
Reminder Systems/
15.
computeri?ed clinical decision support.tw.
16.
CCDS.tw.
17.
medical informatic*.tw.
18.
Medical Informatics/
19.
1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18
20.
asthma.mp. or exp Asthma/
21.
exp Pulmonary Disease, Chronic Obstructive/
22.
chronic obstructive pulmonary disease.tw.
23.
COPD.tw.
24.
20 or 21 or 22 or 23
19 and 24
25.
19 and 24
26.
limit 25 to (English language and humans)
27.
limit 26 to yr = ”2000 -Current”
The asterisk (*) represents any group of characters, including no character. The question mark (?) represents any single character. The dollar sign ($) represents zero or one character (used when searching for expressions).
Inclusion criteria
Empirical studies published in the English language between 2003 to May 2013;
Pediatric or adult CCDSS interventions involving COPD and/or asthma screening, prevention, case detection, and management;
Randomized controlled trials comparing CCDSS with explicitly defined clinical or process outcome measures;
CCDS system used by any clinicians (physicians, physician assistants, pharmacists, dentists, pulmonary specialists or nurse practitioners) directly involved in patient care;
CCDSS targeting patients in improving self-management.
Paper-based tools (e.g. flow charts and non-electronic clinical pathway tools);
CCDSS interventions in people with other conditions (rather than asthma and COPD), including other respiratory diseases;
CCDSS for medical education purposes or only providing summaries of information for patients;
Group based interventions that did not include individual clinical assessment;
Evaluations which focused only on the technical performance of the system as opposed to its effect on clinical practice;
In-patient hospital based systems.
Study selection
Two authors (MF and PNP) independently reviewed the titles, index terms, and abstracts of the identified references and rated each paper as “potentially relevant” or “not relevant” based on study design, subjects, setting, and intervention. These two authors then independently reviewed the full texts of the selected potentially relevant articles and again rated each paper as “potentially relevant” or “not relevant”. After application of the full set of inclusion and exclusion criteria to the potentially relevant studies, a further limitation was then applied, and only RCTs were included. Disagreements between reviewers were resolved by discussion with a third author (CA) until consensus was reached.
Data extraction and quality assessment
The primary author (MF) independently extracted data related to Participants, Intervention, Comparator, Outcomes and Study design by utilizing the PICOS strategy for describing trials (Table 2). The second author (PNP) then independently examined the studies and extracted data to confirm accuracy. The data abstracted included the following information: manuscript authors, year of publication, the study design and duration, participant characteristics (health practitioners, patients), the type of CCDSS intervention, the comparator (usual care or another form of CDSS) and the outcomes measured (clinical, process, workload and efficiency, economic and implementation). Bias was assessed using the Cochrane risk of bias tool [34], and was based on the following five dimensions: randomization, allocation concealment, blinding of participants, personnel or outcome assessors, selective outcome reporting and completeness of follow-up [duration of follow-up, intention to treat (ITT) analysis, withdrawals, and reasons for dropouts]. Each of the above attributes was assessed as being high, low or unclear and an overall risk of bias was reached for each of the included studies (Table 3).
Table 2
Design and characteristics of the included studies
Citation, year, country
Population: no. of centers/providers/patients (Intervention, I or Control, C)
1/-/2098 (children aged 3-11 years) (I = 1082, C = 1016)
RCT, Community based. Duration: 21 months
Parent survey on the presence of asthma symptoms linked sequentially to physician prompts. Physician prompts mediated by CDSS.
Parents received no screening questions, and physicians received no prompts.
Primary outcome: Physicians’ diagnosis of childhood asthma based on prompts by the CDSS.
(+) effect. The number of children diagnosed with asthma in the intervention group was significantly more compared to the control group (8.6% vs. 5.8%, P <0.02). Effect size Cohen’s d = 0.24, 95% C.I = (0.04-0.43)
6/-/95 (adults with adults diagnosed with severe asthma) (I = 51, C = 38)
Pragmatic Multicentre RCT, Academic and community setting Duration: 6 months
Internet-based management tool involving home monitoring of symptoms (using an electronic diary), treatment decision support for the patients, and monitoring support by a study nurse.
Conventional asthma treatment by pulmonologists
Primary outcome: Cumulative sparing of oral corticosteroids, asthma control using Asthma control Questionnaire (ACQ), asthma-related quality of life (AQLQ), Secondary outcomes: FEV1 (using Piko-1 device), exacerbations, hospitalizations and satisfaction (Global satisfaction scale)
(+) effect. Median cumulative sparing of prednisone was 205 mg in the internet group compared with 0 mg in the conventional group. (P = 0.02) Asthma control, AQLQ, FEV1, exacerbations, hospitalizations and satisfaction with the strategy were not different between groups. Effect size for CCS sparing effect: Cohen’s d = 0.46. 95% C.I = (0.08- 0.93)
Patients were trained in using the electronic self -management support system, recording symptoms, measuring lung function and fraction of exhaled nitric oxide (FENO)
37 general practices and 1 academic outpatient department/69/200 (adults with asthma) (I = 101, C = 99)
Multi-centre, RCT, Community and Academic setting. Duration = 12 months
Internet-based asthma self-management program consisting of weekly asthma control monitoring and treatment advice, online support and group education delivered via remote web communications by a specialized asthma nurse
Usual physician-provided asthma care
Process outcomes: (asthma knowledge, inhaler technique and self-reported medication adherence), health care provider contacts for asthma, use of internet based asthma monitoring tool, and medication changes. Clinical outcomes: Primary: Asthma-related quality of life, (32-item Asthma Quality of Life Questionnaire). Secondary: Asthma control (ACQ), symptom-free days, pre-bronchodilator FEV1 (Piko-1), daily inhaled corticosteroid dose, and exacerbations
(+ but modest effect) Improvement in asthma knowledge, inhaler technique and slightly fewer physician visits in the internet group. Treatment changes occurred more often in the internet group. Modest improvement in asthma control and lung function with the Internet intervention, but no reduction in exacerbations. Improvement in asthma-related quality of life was slightly less than clinically significant. (P ≥ 0.5). Effect size for the primary clinical outcome (AQLQ increase by >0.5): Cohen’s d = 0.6. 95% C.I = (0.3-0.9)
Education and training provided to the participants. Non-blinded nature of the study may have affected the results
37/69/200 (adults with partly controlled or uncontrolled asthma) (I = 101, C = 99)
Prospective RCT, Community and academic. Duration: 1 year
Weekly internet based self-monitoring (using ACQ) and subsequent treatment adjustment (using an online management algorithm)
Usual care by the general practitioner according to the Dutch GP guidelines based on GINA guidelines
Primary: Asthma control using (ACQ), spirometry and ATAQ (asthma therapy assessment questionnaire).
(+) effect. Significant improvements in ACQ score after 12 months in the internet based self-monitoring group. Daily inhaled corticosteroid dose significantly increased in the Internet group compared to usual care in the first 3 months in patients with uncontrolled asthma, but not in patients with well or partly controlled asthma. After one year there were no differences in daily inhaled corticosteroid use or long-acting β2-agonists between the Internet group and usual care.
Patients were trained to measure (FEV1) daily with a hand-held electronic spirometer (PiKo1). Supervision provided by a nurse specialist. So there is heavy initial investment. Study outcomes were self-reported by the patients which may overestimate effect. The effect size for change in asthma control is quite large, esp. in the uncontrolled group making the intervention promising.
Secondary: Mean daily dose of inhaled corticosteroid (ICS), and the proportion of participant’s using long-acting β2-agonists (LABA) or leukotriene receptor antagonists (LTRA).
Effect size primary outcome i.e. change in ACQ in partly controlled asthma group*: Cohen’s d = 0.81 95% C.I = (0.33- 1.35)
Effect size for change in ACQ in uncontrolled asthma group*: Cohen’s d = 0.94. 95% C.I = (0.38-1.5). *Assuming t-test was performed.
3/50/1 (simulated patient) (I = 27, C = 23 ED doctors)
RCT, Community setting, Duration: 4 months
An integrated and dynamic electronic decision support system for management of acute asthma in the emergency department (ED) by ED physicians.
Acute asthma management using paper -based clinical records, treatment order sheets and discharge documentation.
Work load & efficiency outcomes: Primary: Quality of asthma documentation–measured using 10 documentation variables (clinical parameters and discharge documentation). Secondary: consultation time
(+) effect. Significantly higher rates of documentation in 7 out of 10 variables, including provision of written short-term asthma management plans. No significant difference in consultation times. Effect size for documentation of asthma management plan provision: Cohen’s d = 0.78. 95% C.I = (0.18-1.37)
Relevant to the ED setting. A 2 minute introduction to the system, including basic functions of the program provided to physicians, which may not be enough. One simulated patient case may not reflect a spectrum of scenarios faced in the ED setting
20/-/11919 (children with asthma between 5-19 years of age) (I = 6110, C = 5809)
Cluster-RCT, Academic. Duration: 5 months.
Electronic health record (EHR) based influenza vaccine clinical alerts
Routine care
Health care process outcomes- rates of captured opportunities for influenza vaccination (visit-level analysis) and up-to-date influenza vaccination status among patients with asthma
(+) but modest effect. Standardized influenza vaccination rates improved 3.4% more at intervention sites than at control sites. Effect was not significant, Cohen’s d not calculated)
Primary care sites were linked to a teaching hospital. Information on the comparator was unclear-implied usual practice. Training provided to the physicians was quite thorough.
12/-/19450 (children) (I = 6, C = 6) Children with persistent asthma identified by using the pediatric asthma control test (PACT))
RCT, Academic setting. Duration: 1 year
CDSS embedded in an electronic health record (EHR), where it provides support in the management of children with asthma in accordance with the (National Asthma Education Prevention Program guidelines (NAEPP).
Passive asthma management tools available in the electronic health record (EHR).
Health care process outcomes: Proportion of children with at least 1 prescription for controller medication, an up-to-date asthma care plan, and documentation of performed office-based spirometry.
(+) effect. Significant improvement in adherence to NAEPP guidelines. 6% increase in the number of prescriptions for controller medications, (P = 0.006) and 3% increase for spirometry (P = 0.04) in the intervention urban practices. Filing an up-to date asthma care plan improved 14% (P = 0.03) and spirometry improved 6% (P = 0.003) in the suburban practices with the intervention. The effect size could not be calculated, as data provided was insufficient to calculate d values).
Medical practices within the Children’s Hospital of Philadelphia (CHOP) Pediatric Research Consortium-may not be generalizable. Physicians were trained to use the CDSS. The actual number of providers involved in the study is unclear.
Multi-centre RCT with three parallel groups. Community setting Duration: 6 months
Physician-managed online interactive asthma monitoring tool which comprised of (1) an electronic diary, (2) an action plan for the patients and (3) a decision support system for the physician. Patients with persistent asthma received advice on treatment based on their asthma control.
Two other usual care groups: specialist group, where treatment was provided by an asthma specialist in an outpatient clinic; and a general practitioner (GP) group, where treatment was provided by GPs in primary care.
(+) effect. Significant improvement in the Internet group compared to the other 2 groups regarding asthma symptoms, quality of life, lung function, airway responsiveness. Significant improvement in the use of inhaled corticosteroids in the internet and specialist group. Effect size comparing the Internet vs. Specialist group for asthma symptom reduction was Cohen’s d =0.53. 95% C.I = (0.19-0.87). Cohen’s d comparing the Internet vs. GP group for asthma symptom reduction was 0.64. 95% C.I = (0.29-0.99).
The number of practitioners and the number of centers participating were unclear. No training provided to the participants or the GP’s but the laboratory assistants providing spirometry and methacholine test were trained in the required protocol
1/-/704 (Children 2-18 years of age), (I = 358, C = 346)
RCT, Community setting, Duration: 3 months
A fully computerized asthma detection system which printed a paper-based asthma care protocol in the pediatric ED to guide early asthma treatment and reduce time to disposition decision.
Usual care, i.e., no reminders or automatic printout was provided.
Primary outcome: Time from ED triage to disposition (discharge or hospital admission) decision.
No effect. No difference in time to disposition. Length of ED stay and the rate of hospital admission were similar between the two groups. (Effect was not significant, Cohen’s d not calculated)
The number of physicians, respiratory therapists and nurses involved in the study is unclear.
Secondary outcomes: Guideline adherence measures including asthma education ordered, protocol found on chart, any asthma scoring performed.
29/-/911 (patients 5+ years of age with severe asthma) (I = 457, C = 454)
Cluster RCT, Community setting. Duration: 2.5 years
Addition of electronic alerts to computerized records to identify at-risk asthma patients experiencing an exacerbation and modify their care.
Control practices continued usual care.
Primary outcome: number of patients experiencing a moderate-severe exacerbation
No significant difference between groups in number of people experiencing exacerbations. Relative reductions in people experiencing hospitalizations, accident and emergency, out-of hours contacts and increase in prednisolone prescriptions for exacerbations without increasing costs. (Effect was not significant, Cohen’s d not calculated)
Training on using electronic alerts provided to at least one representative from each staff group (GP, nurse, receptionist, manager/administrator, dispenser) of the intervention practices.
Secondary outcome: outpatient attendances for asthma, primary care contacts, ‘did not attends’ (DNAs) at consultations, asthma medications and cost analyses
-/435/937 (5-11year old with moderate to severe asthma) (I = 471, C = 468)
RCT, Community setting. Duration: 1 year
Computer generated letters based on information collected from the child’s carer through bi-monthly telephone calls conducted by the centralized service for all the study sites. The letter to the physician caring for that child summarized the child’s asthma symptoms, health service use, and medication use with a corresponding recommendation to step up or step down medications in accordance with the NAEPP guidelines.
No letters sent to the providers of the children in the control group
Health care process outcomes: scheduled visits and changes in medications. Patient outcomes: maximum number of symptom days, ED visits and hospitalizations for asthma, and school days missed because of asthma.
(+) effect Significant increase in scheduled visits, (17.1% vs12.3%, P = 0.005). Significant increase in medication step up (46% vs 35.6%, P = 0.03). Significantly fewer ED visits in the intervention group compared with controls (0.87 vs 1.14 per year, p = 0.013). No difference in the maximum number of symptom days and number of school days missed. Effect size for % of scheduled visits resulting in step-up of medication: Cohen’s d = 0.23. 95% C.I = (0.02-0.43)
Intervention practitioners were trained. Effect size was low for medication change related outcomes. Key issues also included the design where not all children whose medication change was warranted visited the physician.
4/266/706 (246 physicians and 20 outpatient pharmacists)
RCT, Academic setting, Duration: 3 years
Computerized care suggestions to improve asthma and COPD management. These focused on: (1) pulmonary function tests, (2) influenza and pneumococcal vaccinations, (3) prescribing inhaled steroid preparations in patients with frequent symptoms of dyspnea, (4) prescribing inhaled anticholinergic agents in patients with COPD, (5) escalating doses of inhaled β-agonists for all patients with persistent symptoms, (6) prescribing theophylline for patients with COPD and continued symptoms despite aggressive use of inhaled anticholinergic agents, b-agonists, and steroids, and (7) encouraging smoking cessation.
Four groups: physician intervention only, pharmacist intervention only, both pharmacist and physician interventions, and no intervention (controls).
Primary: Adherence to guideline based care suggestions.
No effect. No differences between groups in adherence to the care suggestions, quality of life, patients satisfaction with physicians’ or pharmacists, medication compliance, emergency department visits, or hospitalizations. Physicians receiving the intervention had significantly higher total health care costs. Physician attitudes toward guidelines were mixed. (Effect was not significant, Cohen’s d not calculated)
Hospital based academic practices. Providers included internal medicine physicians, residents and pharmacists. Training was provided to the providers. Questionnaires were administered via telephone.
Secondary: Quality of life-McMaster Chronic Respiratory Disease Questionnaire (CRQ) for COPD patients or the McMaster Asthma Quality-of-Life. Questionnaire (AQLQ). Patient satisfaction: American Board of Internal Medicine’s patient satisfaction questionnaire. Medication adherence: Inui and Morisky surveys and pharmacy dispensing records
A decision support system with reactive computer reminders (CRS) to improve drug prescribing behaviors. 25 GPs received reminders on antibiotics and asthma/COPD prescriptions.
28 GPs received reactive computer reminders (CRS) to improve prescribing of cholesterol-lowering drugs
Primary outcome: prescription according to the guidelines as a percentage of total prescriptions of a certain drug. Secondary outcome: user friendliness.
(+) but not significant effect. CRS with reactive reminders improved drug prescribing behavior. Preliminary results also indicate reduction in the number of prescriptions according to the advice of the computerized guidelines not to present certain drugs. It was perceived stable and user friendly. (Effect was not significant, Cohen’s d not calculated)
Preliminary study. Both groups served as control to one another. Not specific to asthma/COPD.
14. Martens et al. [48] Netherlands. Follow-up of the above study
14/53/-
Clustered RCT, Community setting. Duration: 1 year
CRS focused on drug-prescribing behavior of GPs. 25 GPs received reminders on antibiotics and asthma/COPD prescriptions
28 GPs received CRS reminders on cholesterol prescriptions
Guideline appropriate prescriptions as a percentage of total prescriptions (of the drug category involved) for the same diagnosis on the individual GP level. Absolute number of prescriptions for a specific diagnosis per GP per 1000 enlisted patients.
No effect. No favorable effects were found for CRS with the message to prescribe certain drugs. On the other hand, CRS with the message not to prescribe certain drugs sometimes positively influenced the prescribing behavior of GPs. (Effect was not significant, Cohen’s d not calculated)
Not specific to asthma/COPD. Both groups served as a control group to one another. Authors report the study to be underpowered due to high inter doctor variation in prescribing behavior (Cluster effect). Training was provided.
15. Martens et al. [49] Netherlands. Follow-up of the above study.
20/48/-
Clustered RCT, Community setting. Duration: 1 year
25 GPs received reactive computer reminders on antibiotics and asthma/COPD prescriptions
28 GPs received (reactive) reminders on cholesterol prescriptions
Number of GPs (competent and willing) with CRS still functioning after 1 year. Number of GPs having technical problems or are unwilling. Number of reminders/GP/month/1000 enlisted patients. GP user satisfaction (satisfaction questionnaire). GP experience (content and extensiveness of CRS). Barriers and facilitators to implementation and use of CRS
(+) learning effect from the CRS. 9% of GPs dropped out after 1 year. A significant learning curve was found (P = 0.03) for the reminders on antibiotics, asthma and COPD. GPs were satisfied with the user-friendliness and the content of the different types of reminders, but less satisfied with certain specific technical performance issues of the system. Cohen’s d = N/A (Effect size not calculable due to insufficient data provided in report)
Not specific to asthma/COPD. Both groups served as a control to one another. GP’s were trained.
32/40/156,772 (study patients (children and adults) either had chronic bronchitis, emphysema, other chronic pulmonary diseases, asthma or COPD) (I = 20, C = 20 General practitioners)
RCT, Community and academic Duration: 10 months
Asthma Critic used for monitoring and treatment of patients with asthma and COPD by Dutch general practitioners in daily practice. The asthma critic was a computer software support program that presented a patient specific comment to the physician based on the current clinical situation.
Usual care
Average number of contacts, FEV1 (force expiratory volume) and peak flow measurements per asthma/COPD patient per practice, and the average number of antihistamine, cromogylate, deptropine, and oral bronchodilator prescriptions per asthma/COPD patient per practice.
(+) effect. Statistically significant increase in contact frequency with the patient (P = 0.034), peak flow measurement, FEV1 measurements in 12-39 years age group (P = 0.02). Significant decrease in cromogylate prescriptions in the age group of 12-39 years, (P = 0.03). Non-significant decrease in deptropine, antihistamines, oral bronchodilators. (Effect size not calculable due to insufficient data provided in report).
The study focused on change in physicians’ behavior. Training was provided to the general practitioners.
Expert support system for the interpretation of spirometry tests to help GPs’ in the diagnosis of chronic respiratory diseases. The expert system provided interpretation in the form of flow volume curve, graphical interpretation and textual interpretative notes of spirometry results to intervention GPs.
GPs in the control group simply received the spirometry test results, and the flow–volume and volume–time curves.
Primary: Difference between the percentage agreement of the cases’ diagnoses between GPs and expert panel judgment before and after interpretation of spirometry
No Effect. There were no differences between the computerized expert support and control groups in the agreement between GPs and expert panel on diagnosis of COPD, asthma and absence of respiratory disease. A higher rate of additional diagnostic tests was observed in the expert support group. (Effect was not significant, Cohen’s d not calculated)
This was a simulated study- no real patients involved. Training was provided.
Secondary: Impact of the expert system intervention on the GPs decision-making processes through six measures: additional diagnostic test rates; width of differential diagnosis; certainty of diagnosis; estimated severity of disease; referral rate; and medication or non-medication changes.
Two interventions: GPs received spirometry interpretation support by either a chest physician (who had standard spirometry software) or expert spirometry support software.
Usual care had standard spirometry software (i.e. no additional interpretation support).
Primary: A change of diagnosis after spirometry interpretation support. Secondary: referral rate, additional diagnostic tests, and disease management changes.
No effect. Differences in proportion of changed diagnoses were not statistically significant. There were no differences in secondary outcomes. (Effect was not significant, Cohen’s d not calculated)
15/102/59,147. (Patients with medically complex conditions like xerostomia, diabetes mellitus, COPD, congestive heart failure).
RCT with three arms (provider activation, patient activation and control group), Community setting. Duration = 2 years
Two CDS approaches. In one group, dentists and hygienists received alerts in the EDRs (electronic dental records) when patients scheduled for dental appointments had one of the targeted medical conditions. In second group, in addition to the above, patients with upcoming dental appointments who had one of the targeted medical conditions received a notification from HPDG (health partners dental group) before the visit, encouraging them to discuss it with his or her dental care provider at the appointment.
Patients in the control group received usual care. Neither the patients nor the provider’s, received alerts about a patient’s medical status or personalized care guidelines.
Primary: Total use-the overall frequency with which providers accessed the guidelines web site via the EDR for any patient. Targeted use—the proportion of providers who accessed the care guidelines in general and for targeted patients at the point of care. Ongoing use—the proportion of providers who continued to access the web-based guidelines through-out the study period.
(+) effect. Participants in the provider and patient activation groups increased their use of the system during the first six months. Provider activation was more effective than was patient activation. (P < 0.05). However, it was not sustainable, and by the end of the study, the rate of use had returned to baseline levels despite participants’ continued receipt of electronic alerts. (Effect expressed as Odds Ratio for web use for provider group in first six months = 4.4 (95% C.I = 1.6-12.1) and 6-12 months after implementation compared to controls = 1.7 (95% C.I = 0.1-2.9). For provider + patient activation group, effect expressed as Odds Ratio for web use in first six months =2.1 (95% CI, 0-9-4.8) and 6-12 months after implementation compared to controls = 1.4 (95% CI 0.5-3.5).
Dental clinic based. Study was not specific to asthma/COPD patients.
Cohen’s d values calculated only for significant primary outcomes. Where primary data were not available, formulae to use P values/frequency tables to estimate d were used based on the Campbell Collaboration free online effect size calculator (available online athttp://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD2.php, accessed 9th September 2014). It must be noted that these effect sizes are estimates only.
Table 3
Quality assessment of the included randomized controlled trials
User workload and Efficiency outcomes (user knowledge, clinician workload, efficiency)
Relationship centered outcomes (patient satisfaction)/Economic outcomes (cost and cost effectiveness)/Use and implementation (health care provider acceptance, satisfaction, use and implementation)
Drug therapy management based CCDSS: Corticosteroid treatment decision support for the patients based on symptoms, lung function and exhaled NO (nitric oxide)
Primary outcome: Significant decrease in corticosteroid consumption in patients with steroid dependent asthma (+ effect)
No difference in asthma control, quality of life, FEV1, exacerbations, hospitalizations between groups. (+ effect)
No difference in patient satisfaction between groups
Multifaceted CCDSS: Weekly asthma monitoring and advice, online and group education and remote web communications
Patients’ asthma knowledge, inhaler technique improved. Medication changes occurred more often. Health care provider contacts were fewer. (+ but modest effect)
Primary outcome: Asthma related quality of life improved. Secondary: Asthma control, lung function improved, symptom-free days increased, exacerbations did not differ between groups. (+ but modest effect)
Multifaceted CCDSS: Weekly asthma monitoring and self-management advice.
Secondary: Significant increase in the corticosteroid dose in patients with uncontrolled asthma, but not in patients with well or partly controlled asthma. (+ effect). Adherence to ACQ monitoring gradually declined in the first month to the seventh month and then remained stable. No difference in dose of corticosteroids or LABA or LRTA after 12 months
Primary outcome: Significant improvement in asthma control in patients with partly and uncontrolled asthma. (+ effect)
Multifaceted CDSS: The system integrated asthma management pathways based on current guidelines into clinical and discharge documentation. Including triage and registration, clinical documentation, treatment orders, order entry and discharge documentation.
Primary outcome: Significantly higher rate of asthma documentation. Secondary outcome: No significant difference in consultation time. (+ effect)
Multifaceted CCDSS: EHR based CDS alerts and reminders based on pediatric asthma management tool (PACT) which captured asthma symptom frequency, asthma severity, facilitated ordering of controller medications, spirometry and ACP (asthma action plan)
Primary outcome: Increase in the number of controller medication prescriptions, and up-to-date asthma action plan (ACP). (+ effect, but not significant). Increase in the use of spirometry in the intervention group (+ effect, but not significant)
Multifaceted CCDSS: EHR based alerts to flag the at-risk status of patients to improve patient access and opportunistic management
Relative increase in LABA usage and decrease in nebulized B-agonists (+ effect)
Primary outcome: No significant difference in the number of people experiencing exacerbations. Relative reduction in people experiencing hospitalizations, accident and emergency attendances, out-of-hour contacts and other health care use. (+ effect, but not significant)
Cost –effectiveness outcome: Adjusted mean health care (NHS) cost lower among intervention practices compared to control practices (+ effect)
Drug therapy management based CCDSS: Computer generated letter recommending change in controller medications based on NAEPP guidelines
Primary outcome: Significant increase in scheduled visits leading to stepping up of asthma medications (+ effect)
Significant decrease in ED visits. No difference in maximum number of symptom days and school days missed, decrease in the number of days with activity limitation.
Intervention-reduced asthma related cost to the health services and was cost-effective. (+ effect)
Drug therapy management based CCDSS: Guideline based reminders when prescribing antibiotics, asthma/COPD and cholesterol prescriptions
Significant learning curve was found (shows improvement in user knowledge) (+ effect)
Primary outcome: Provider use: Only 9% drop-out rate (because of technical problems requiring multiple updates) (+but not significant effect) Provider satisfaction: Positive attitude to the content of the reminders and satisfied with the user friendliness
Multifaceted CCDSS: Asthma critic evaluates whether the patient has asthma or COPD, reviews the physicians treatment, and generates feedback
Primary outcome: Significant increase in the average number of contacts. Significant decrease in the average number of cromogylate prescriptions. No statistically significant change in the antihistamines, deptropine, and oral bronchodilator prescriptions per asthma/COPD patient per practice (+ effect)
Significant increase in FEV1 (forced expiratory volume), and peak-flow measurements per asthma/COPD patient per practice (+ effect)
Multifaceted CCDSS: Presentation of data for diagnosis and management of chronic airway disease
Primary outcome: No difference in between the two groups (Spirometry expert system and sham information) in the diagnosis of COPD, asthma and absence of respiratory disease or in medication changes. Secondary: Slightly more additional diagnostic tests in the expert group (No effect)
Multifaceted CCDSS: Spirometry expert support for change in diagnosis and management
Primary outcome: No differences in the proportion of changed diagnosis between the three groups (spirometry expert system, chest physician and usual care). Also no difference between the groups in referral rate, additional diagnostic tests and medication changes (No effect)
Multifaceted CCDSS: EDR (Electronic dental record) based alerts notifying the dentists of the presence of a medically complex condition in a patient with a link to modify dental care appropriately
Primary outcome: Significant increase in the frequency of dentists accessing guidelines (number of website hits and number of providers using the guideline). Only number of hits sustained after 6 months. After 9 months provider use returned to baseline levels (+ effect)
1)
Diagnostic advice only;
2)
Drug therapy management only; and
3)
‘Multi-faceted’ interventions comprising two or more different intervention components.
Effectiveness of CCDSS
Using classifications published in previous reviews pertaining to CCDSS [54], we assessed CCDSS effectiveness based on the following key outcomes:
(1)
Clinical outcomes: Morbidity, health related quality of life, hospitalizations and mortality. [e.g. asthma symptoms (measured using symptom diary), asthma control (Asthma control Questionnaire-ACQ), lung function (Piko-1 device, peak flow meter), health-related quality of life (measured using HRQOL) and adverse events (leading to unscheduled doctors visit or hospitalization)];
(2)
Healthcare process measures: Recommended preventive care services ordered or completed (e.g. influenza vaccination), recommended clinical study ordered or completed (including spirometry), recommended treatment ordered or completed (including rescue medication prescriptions and antibiotic prescriptions)];
(3)
User workload and efficiency outcomes: Effect on user knowledge, number of patients seen per unit time, clinician workload, and efficiency;
Economic outcomes: Cost and cost effectiveness of the CCDSS used; and
(6)
Use and Implementation outcomes: Health care provider acceptance, health care provider satisfaction, and health care provider use and implementation.
A CCDSS was considered effective if it produced a statistically significant (p <0.05) improvement in the primary outcome or improvement in ≥50% of multiple relevant pre-specified outcomes. If the authors did not designate a primary outcome, we considered the outcome used to calculate the trial’s sample size to be primary. Studies that included multiple intervention arms were considered effective if any of the CCDSS based treatment arms was evaluated as effective.
Although we had intended to conduct meta-analyses, this was abandoned owing to the marked heterogeneity in participants, clinical settings, interventions, and the outcomes measured in the included studies. However, effect sizes (Cohen’s d value) of the significant primary outcomes were calculated wherever possible.
Results
The PRISMA guidelines for conducting/reporting systematic reviews were consulted and a completed checklist is attached as Additional file 1. We screened 1042 abstracts, identified 173 full-text potentially relevant articles and included 19 articles representing 16 RCTs in the review (Figure 1).
Figure 1
PRISMA flow diagram of the included and excluded studies[55].
Studies were predominantly conducted in the Netherlands (n = 9) and the USA (n = 7) with one study conducted in each of the following countries: Australia, United Kingdom and Denmark.
Eleven studies evaluated asthma care [35–45], 5 studies involved patients with asthma and COPD [46–50], and 3 studies focused on people with medically complex conditions including COPD [51–53]. There were no studies conducted exclusively on people with COPD (Table 2).
Study quality
A summary of the study quality of the included studies is reported in Table 3. Of the 19 trials, 10 studies had a low risk of bias [36–39, 44–46, 51–53]. Eleven studies described an appropriate method of sequence generation [35–38, 44–46, 50–53], 9 studies reported adequate concealment of allocation [36–39, 42, 44, 46, 50, 53], and 13 studies showed either no differences in baseline characteristics between study groups or performed appropriate adjustments [35–37, 39, 40, 42–46, 50, 51, 53]. Eleven studies used objective outcomes or blinding of outcome assessments [36, 39, 40, 43–45, 48, 50–53], and 11 studies achieved a ≥90% follow-up rate for their unit of analysis [35–39, 44–46, 48–50].
CCDSS and study characteristics
Table 2 describes the CCDSS design and implementation characteristics. The majority of interventions (68%) (13/19) were embedded in an existing electronic medical record (EMR) or with the computerized physician order entry (CPOE) systems [35, 39–41, 43, 44, 46–52]. Thirty-one percent (6 studies) had a stand-alone system, of which four were internet based [36–38, 42] and in the other 2 studies CCDSS intervention were administered to practitioners by the study researchers [45, 52]. Five of the six studies with the stand-alone CCDS system showed positive impact. Sixty-three percent (12 studies) automatically pre-populated the EMR data [35–41, 43, 44, 46, 50, 53], 26% (5 studies) relied on practitioners to manually enter the data [42, 47–49, 52], 16% (3 studies) relied on research staff [44, 45, 51] and 21% (4 studies) relied on patients as well for data inputs [36–38, 42]. Forty-seven percent (9 studies) compared a computerized clinical decision support system directly with usual care [35–38, 40, 43, 44, 50, 52].
Advice at the point-of-care was provided in 14 trials [35, 39–41, 43, 44, 46–53] and via the internet in 4 trials [36–38, 42]. Advice in the form of a computer-generated letter recommending changes to the treatment was provided to the practitioners in one study [45]. Advice was provided only to the physicians in 68% (13 trials) [35, 39–41, 43, 44, 46–52], while only one trial involved provision of advice to other healthcare practitioners (pharmacists) in addition to physicians [46]. In 26% (5 studies) patients were directly advised in addition to practitioners [36–38, 42, 53]. Thirteen studies provided explicit training in use of the CCDSS to healthcare practitioners [39, 41–52], while patients were trained to use the internet based CCDSS in 3 trials [36–38]. The CCDSS user interface characteristics were described in only 42% (8) of the trials [35, 38, 39, 43, 48–50, 53].
Studies included a highly varied number of healthcare practitioners, patients and health services.
Since CCDSS is primarily focused on altering provider behavior, the unit of randomization in most CCDSS studies was the provider. Thirty-seven percent (7) of the studies reviewed were randomized at the provider, practice or community level [39, 45, 46, 50–53], while 31% (6) used cluster randomization either between clinics or groups of providers that worked closely together [40, 41, 44, 47–49]. Thirty-one percent (6 studies) were randomized at the patient level [35–38, 42, 43].
In all studies that included patient level randomization there was potential for contamination given a single provider could care for both intervention and control arm patients. The principal summary measures used to compare effects between intervention and control groups varied and included: proportions [35, 39, 41, 45]; difference in medians [36, 38, 43, 50]; difference in means [46, 48]; relative risk [37] and odds ratios [40, 42, 44, 51–53].
Three trials did not clearly report their source of funding [41, 42, 53]. Of the remaining, 9 trials were publicly funded [36, 37, 41, 43, 45, 46, 51–53], 5 trials were conducted with only private funds [40, 44, 47–49], 1 trial was conducted with a combination of private and public funding [42], while another trial did not receive any funding [35]. Five trials declared that at least one author was involved in the development of the CCDS system [37, 38, 47–49], while the remaining trials did not indicate at all if the authors were independent of development.
Type of CCDSS interventions
The included studies utilized CCDSS for a variety of purposes, and were categorized in to three main categories, such as those focusing on screening/diagnosis, drug therapy management, and multifaceted interventions which involved various aspects of disease management along with self-management advice (Table 4).
There was only one study, conducted by Caroll et al. that used the CCDSS (Child Health Improvement through Computer Automation-CHICA system) for the purpose of diagnosing pediatric asthma [35]. Five studies (26%) used the CCDSS for drug therapy management [36, 45, 47–49], and thirteen used a multi-faceted form of CCDSS [37–44, 46, 50–53].
The studies evaluating the CCDSS focusing on drug therapy management included the study by Hashimoto et al. which utilized an Internet based treatment decision support system to guide people with severe asthma in tapering the dose of oral corticosteroids depending on their asthma control [36]. The study by Kattan et al. involved provision of a computer-generated letters to the treating physician summarizing the appropriate treatment recommendations based on the child’s asthma symptoms, health service and medication use [45]. The three studies conducted by Martens et al. also involved a CCDSS in the form of reactive computer reminders (CRS) to improve drug prescribing in general practice [47–49].
The remaining 13 studies (68%) evaluated multifaceted forms of CCDSS [37–44, 46, 50–53]. These CCDSSs ranged from simple activation of electronic alerts to identify people at risk of an asthma exacerbation [44], or prompts to alert the physician to modify treatment in people with medically complex conditions at the point of care [53], to more complex forms of CCDSS interventions involving a series of care suggestions on drug therapy and disease management [46]. Internet-based multifaceted CCDSS interventions were evaluated in three trials which focused on the self-management of asthma [37, 38, 42]. These studies utilized online self-management programs which involved weekly online asthma control monitoring and feedback in the form of treatment advice by a specialized asthma nurse [37, 38], or by the patients’ physician [42]. Another type of multifaceted CCDS interventions evaluated in two other studies were in the form of EHR-based clinical alerts either to improve influenza vaccination in children with asthma [40], or to improve overall asthma care in these children [41]. The multifaceted CCDSS evaluated by Kuilboer et al. was a critiquing system integrated with the general practitioners’ electronic medical records which reviewed physicians’ treatment decisions and generated feedback [50]. Another two studies evaluated the impact of an expert spirometry system on the physician’s decision making during asthma diagnosis [51] and during management [52]. The remaining two trials also tested the effects of another multifaceted CCDSS on the clinician’s performance in the form of an electronic interface system to manage asthma patients in ED [39] and in the form of an automated asthma detection system used to identify and manage people at risk of asthma exacerbation in the emergency department [43].
CCDSS effectiveness
There was marked variability in the outcomes reported. Therefore we assessed the effectiveness of the CCDSS on the primary outcomes measured. In majority of the trials reviewed, the primary outcomes assessed were health care process measures, clinical outcomes, user work load and efficiency, and use and implementation outcomes. Relationship-centered outcomes and economic outcomes were measured by few trials, but only as secondary outcomes. Fourteen trials (74%) showed positive effect from the use of CCDSS on the primary outcome measured, of these 9/19 (47.3%) showed a significantly positive effect [35, 36, 38, 39, 41, 42, 45, 50, 53].
Clinical outcomes
The different clinical outcomes reported in the studies included asthma symptoms [42], asthma/COPD symptoms [46] asthma control (ACQ) [36–38], Health related Quality of life [36, 37, 42, 46], frequency of health care utilizations including hospitalizations [36, 44], admission rate and ED length of stay [43], frequency of exacerbations [36, 37, 44], lung function (FEV1) in asthma patients [36, 37, 42], exhaled nitric oxide [36], symptom free days [37], airway hyper responsiveness [42], number of ED visits [45, 46] number of school days missed [45], medication adherence [46], FEV1 and peak flow measurements in asthma/COPD patients [50].
Five of the nineteen trials assessed clinical outcomes as the primary outcome measure [35, 37, 38, 42, 44], of which three showed clinically significant improvements [35, 38, 42], one showed a positive but modest improvement in the asthma related quality of life [37] and another one did not show any effect on the number of people experiencing an exacerbation from the use of EHR embedded asthma risk alerts [44].
Significant improvement was found in the rate of diagnosis of asthma in children by implementation of a parent survey linked to physician prompts using computer decision support system called the CHICA system [35]. Significant improvement was also found in asthma control measured weekly using the asthma control questionnaire (ACQ) [38], in asthma symptoms using an electronic diary to record symptoms daily and in the asthma quality of life measured using asthma quality of life questionnaire (AQLQ) [42]. The effect sizes (Cohen’s d) calculated for the studies showing significant improvement in the primary clinical outcomes ranged from 0.24 to 0.94, with three studies showing a reasonably large effect size [37, 38, 42].
Health care process outcomes
The different health care process measures that were assessed in the reviewed trials included change in the consumption of oral corticosteroid [36], change in the dose of inhaled corticosteroid [38, 42], change in patients’ asthma knowledge [37], change in inhaler technique [37], change in medication adherence [37], medication changes [37, 38, 44], adherence to the use of ACQ [38], rate of vaccination [40], number of corticosteroid prescription ordered [41], provision of asthma action plan [41], spirometry ordered [41], rate of asthma documentation by ED doctors [43], scheduled physician visits leading to change in medication dose [45, 50], physicians adherence to guidelines [46], change in the number of prescriptions [47, 48, 50], change in the diagnostic ability of the general practitioner [51, 52], diagnostic tests ordered [51, 52] and the rate of referral [52].
Ten trials assessed health care process measures as the primary outcome [36, 40, 41, 45–48, 50–52], of which four showed significant improvement in these outcomes. Significant improvement was seen in process outcomes like cumulative sparing of prednisone dose adjusted weekly according to the internet based CDSS [36], percentage of children given at least one prescription of corticosteroid [41], percentage of visits to the physician leading to medication step up of asthma medication [45] and in the number of contacts with the patients’ physician [50]. The effect size calculated for the two studies [36, 45] with significantly positive improvement was however poor. Three trials showed a positive but modest effect of which one showed a modest improvement in the rate of influenza vaccination by the use of EHR alerts [40]. The other 2 trials showed a modest improvement in the drug prescribing behavior of GPs from the use of the CRS reminders [47, 48]. The remaining 3 studies did not show any effect from the use of CCDSS on the primary health care process outcome assessed.
User workload and efficiency outcomes
Workload and efficiency outcomes assessed in the trials included asthma documentation by emergency department (ED) doctors [39], consultation time [39], time for disposition decision in the ED [43], and user knowledge [49]. These outcomes were assessed as the primary outcome by only two trials [39, 43], of which one trial showed significant improvement in the rate of asthma documentation by the ED doctors in the management of acute asthma [39]. The size of the effect calculated for this trial was relatively large (Cohen’s d =0.78). However the other trial did not show any effect from the use of CCDSS on the time taken by the ED physicians to make a disposition decision [43].
Use and implementation outcomes
The outcomes assessed under this category were physicians attitude to guidelines [46], user friendliness [48, 49], provider satisfaction [49], and the rate of accessing guidelines [53]. Two trials assessed these outcomes as the primary outcome, of which one showed a significant improvement in the rate of use of guidelines by dentists in the management of people with chronic diseases including COPD [53]. The study showed that the use of CCDSS increased the number of times the dentists accessed the guidelines. The other trial also showed a positive but modest effect in the use of CRS (reactive computer reminders) by general practitioners, not to prescribe certain drugs [49].
Other outcomes
Outcomes such as patient satisfaction with CCDSS use was measured as a secondary outcome by two trials and found no difference in patient satisfaction between the intervention and the control group patients [36, 46]. Measures such as health care provider satisfaction were also assessed as secondary outcomes. Of the three studies that measured these outcomes [46, 47, 49], two found that the provider perceived the CCDSS as user friendly. Only one trial measured cost of the intervention and found that the patients in the group receiving the CCDSS intervention had significantly elevated total health care charges [46]. Two other trials measured the cost-effectiveness of the CCDSS used and both found that its use was more cost effective than usual asthma care [44, 45].
Discussion
This is the first comprehensive review of CCDSS in the care of patients with chronic respiratory diseases, asthma and COPD. The review focused only on studies conducted in primary care as the bulk of the management of these chronic diseases happens in primary care. The review found that the use of CCDSS can have a positive impact on the diagnosis and management of asthma and COPD in primary care. Overall 74% of the studies reviewed showed improvement in the primary outcomes. Although there is literature available on the use of CCDSS in patients with asthma, there is very little literature on its use in the management of people with COPD.
The review also found that 83% (5/6) of the studies that utilized CCDSS with a stand-alone design showed positive outcomes as compared to studies testing CCDSS which were integrated with the EHR or the EMR systems (38%) (5/13). This indicates that systems presenting advice within electronic health records or order entry systems were much less likely to improve care or outcomes than stand-alone programs. It has been found that when integration of alerts within an institution’s electronic health records becomes possible and more alerts are added, practitioners might become overwhelmed and begin to ignore the prompts. This “alert fatigue” phenomenon [56] could be responsible for limiting behavior change. Studies estimate that as many as 96% of alerts are over ridden [57–59] and suggest that the threshold for alerting is too low (that is, alerts are sensitive but not specific). Systems requiring the practitioner to give a reason for over-riding advice were more likely to succeed than systems missing this feature [60].
Four of the five studies evaluating CCDSS with a stand-alone design, were Internet-based interventions targeting both physicians and patients. All the four studies showed that CCDSSs which targeted the patients as well as the physicians were effective in improving outcomes. The findings are consistent with other previous reviews of CCDSS for chronic disease management in primary care [33, 61]. A key feature of these interventions was the active incorporation of a patient self-management component for use outside of the clinical encounter. The CCDSS interventions included in the studies involved regular monitoring and feed-back along with patient education and follow-up. These results confirm the value of collaborative care in chronic respiratory disease management.
Also CCDSS interventions consisting of multiple components such as reminders and education were associated with greater improvement in outcomes than single-target interventions with fewer components. This is also reflected in other reviews evaluating the effectiveness of such multi-component CCDSSs engaging patients in the management of other chronic conditions like diabetes [33] and osteoporosis [62]. Given the advent of personal health records, patient portals, and mobile applications aimed at better engaging patients, the findings suggest that there is a need to consider multiple components and targets in the development of any future interventions.
Of the outcomes measured many of the included studies (53%) often focused on measuring the effectiveness of CCDSSs on the health care process outcomes and the evidence demonstrating the effects of CCDSSs on patient outcomes, user workload and efficiency and economic outcomes remains surprisingly low. This is comparable to other recent CCDSS reviews which also report on the paucity of well-designed studies evaluating the effects of CCDSS on patient related outcomes [32, 63]. This may have occurred owing to under powering, since most of the studies may not have had large enough sample sizes to detect such outcomes. Similarly many of these studies were not conducted over longer time frames. Both the sampling and time issues were possibly due to the relative difficulty of implementing randomized, controlled trials in real clinical settings [54]. Since clinical decision support has a primary function aimed at providing information to the provider at the point of decision making and intervention, outcomes which measure process or provider behavior are often used as a proxy for patient outcomes [61]. Although analysis of process outcomes has a merit as an interim platform to justify the continuing role of CCDSS in clinical care, more research is needed on evaluating the effectiveness of CCDSS on patient outcomes in order to adequately understand the usefulness of CCDSS in clinical setting. Nevertheless, 60% of the 5 studies measuring clinical outcomes showed significantly positive impact on these outcomes as compared to 40% of the 10 studies showing significant improvement in health care process outcomes. This implies that the implementation of CCDSS for asthma/COPD care seems promising in improving clinical outcomes. The most commonly reported clinical outcomes were asthma control and asthma quality of life.
CCDSSs may represent a cost-effective way of improving chronic respiratory disease outcomes in primary care. However, the review found that the economic effects of these systems could not be readily assessed based on the available data. The costs of design, local implementation, ongoing maintenance, and user support can be high, and may be further elevated by the unique nature of chronic respiratory care. This warrants cost-effectiveness analyses, but only two trials reported such data and little cost data of any kind was available across studies. Almost all of the studies discussed the need for more research utilizing cost-effectiveness outcomes to better assess the long term effectiveness of CCDSS.
The review also found that there were no studies that demonstrated a negative finding (patient harm or deterioration related to the intervention). This could be because the studies did not actively collect any data on harm assessment of the CCDSS used. Prospective data on the possible harms of CCDSSs are needed to facilitate informed adoption decisions. Based on the available evidence it is hard to draw conclusions about the potential negative effect of implementing decision-support tools, which is necessary to truly fulfil the goal of evaluating these interventions and to better address implementation challenges [64].
Strengths and limitations
The review has several important strengths. This is the first review evaluating the role of CCDSS in the management of chronic respiratory diseases, asthma and COPD in primary care. We particularly excluded studies regarding in-patient hospital based CCDSSs as we intended to focus its effectiveness in primary and community health care, given that only a small proportion of people with, asthma for example, are managed in the hospital setting. The search strategy of our study was extensive and thorough, and covered a large number (eight) of relevant databases to identify potentially relevant studies. The other strength is we based our review on the strongest studies available, RCTs. Also to reduce the risk of selection bias and incorrect categorization all the included articles were analyzed and critically examined by three reviewers independently.
There are a number of key limitations to this review. We excluded studies regarding in-patient hospital based CCDSSs as we intended to focus its effectiveness in primary and community health care. We included only English language studies conducted in the last 10 years as we wanted to document the recent advances in this area. Our analyses were limited to published reports of randomized controlled trials, so the possibility of publication bias or selective reporting must be acknowledged. The CCDSSs were grouped into categories based on clinical applications rather than on other aspects of CCDSS design. We were unable to conduct meta-analysis, given the substantial heterogeneity in the type of CCDSSs and the outcomes evaluated, however we calculated the effect sizes of the primary outcomes for easier comparison of the study effects. Finally, we summarized only randomized controlled trials which might have resulted in less information about issues related to CCDSS implementation, effect on workflow, and factors affecting usability.
Conclusion
In summary, the review demonstrates that CCDSS can improve chronic disease management processes and clinical outcomes in patients with asthma and COPD, but data showing its effect on use and implementation and economic outcomes were sparse. The review also found that although there are a growing number of RCTs that assessed a wide variety of CCDSSs designed to improve asthma management in primary care, there is very scant evidence of its use in the care of patients with COPD.
The mechanisms behind systems’ success or failure remain understudied, but non-integrated, multifaceted CCDSS providing advice to both practitioners and patients, and those requiring the practitioners to give explanations for over-riding advice might independently improve success.
Future trials with clear descriptions of system design, local context, implementation strategy, costs, adverse outcomes, user satisfaction, and impact on user workflow will better inform CCDSS development and decisions about local implementation.
Abbreviations
COPD:
Chronic Obstructive Pulmonary Disease
CCDSS:
Computerized clinical decision support systems
EMR:
Electronic medical records
EHR:
Electronic health records
CRS:
Computer reminder system
CPOE:
Computerized physician order entry
EDR:
Electronic dental records.
Declarations
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Authors’ Affiliations
(1)
Sydney Medical School, University of Sydney
(2)
Woolcock Institute of Medical Research, Sydney Medical School, University of Sydney
(3)
The George Institute for Global Health, Sydney Medical School, University of Sydney
Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, Abraham J, Adair T, Aggarwal R, Ahn SY, Alvarado M, Anderson HR, Anderson LM, Andrews KG, Atkinson C, Baddour LM, Barker-Collo S, Bartels DH, Bell ML, Benjamin EJ, Bennett D, Bhalla K, Bikbov B, Bin Abdulhak A, Birbeck G, Blyth F, Bolliger I, Boufous S, Bucello C, Burch M, et al: Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012, 380: 2095-2128. 10.1016/S0140-6736(12)61728-0.View ArticlePubMedGoogle Scholar
Lalloo UG, Walters RD, Adachi M, de Guia T, Emelyanov A, Fritscher CC, Hong J, Jimenez C, King GG, Lin J, Loaiza A, Nadeau G, Neffen H, Sekerel BE, Yorgancıoğlu A, Zar HJ: Asthma programmes in diverse regions of the world: challenges, successes and lessons learnt. Int J Tuberc Lung Dis. 2011, 15: 1574-1587. 10.5588/ijtld.11.0289.View ArticlePubMedGoogle Scholar
Adams RJ, Fuhlbrigge A, Guilbert T, Lozano P, Martinez F: Inadequate use of asthma medication in the United States: results of the asthma in America national population survey. J Allergy Clin Immunol. 2002, 110 (1): 58-64. 10.1067/mai.2002.125489.View ArticlePubMedGoogle Scholar
Finkelstein JA, Lozano P, Farber HJ, Miroshnik I, Lieu TA: Underuse of controller medications among Medicaid-insured children with asthma. Arch Pediatr Adolesc Med. 2002, 156 (6): 562-567. 10.1001/archpedi.156.6.562.View ArticlePubMedGoogle Scholar
Carlton BG, Lucas DO, Ellis EF, Conboy-Ellis K, Shoheiber O, Stempel DA: The status of asthma control and asthma prescribing practices in the United States: results of a large prospective asthma control survey of primary care practices. J Asthma. 2005, 42 (7): 529-535. 10.1081/JAS-67000.View ArticlePubMedGoogle Scholar
Lozano P, Finkelstein JA, Hecht J, Shulruff R, Weiss KB: Asthma medication use and disease burden in children in a primary care population. Arch Pediatr Adolesc Med. 2003, 157 (1): 81-88. 10.1001/archpedi.157.1.81.View ArticlePubMedGoogle Scholar
Krahn MD, Berka C, Langlois P, Detsky AS: Direct and indirect costs of asthma in Canada, 1990. CMAJ. 1996, 154 (6): 821-831.PubMedPubMed CentralGoogle Scholar
The global strategy for the diagnosis, management and prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD). 2014, http://www.goldcopd.org/,
McKenzie DK, Abramson M, Crockett AJ, Glasgow N, Jenkins S, McDonald C, Wood- Baker R, Frith PA, on behalf of The Australian Lung Foundation: The COPD-X Plan: Australian and New Zealand Guidelines for the management of Chronic Obstructive Pulmonary Disease 2011. 2011, Lutwyche, Queensland: Australian Lung Foundation, [http://www.copdx.org.au]Google Scholar
Ta M, George J: Management of chronic obstructive pulmonary disease in Australia after the publication of national guidelines. Intern Med J. 2011, 41: 263-270. 10.1111/j.1445-5994.2009.02133.x.View ArticlePubMedGoogle Scholar
Mularski RA, Asch SM, Shrank WH, Kerr EA, Setodji CM, Adams JL, Keesey J, McGlynn EA: The quality of obstructive lung disease care for adults in the United States as measured by adherence to recommended processes. Chest. 2006, 130: 1844-1850. 10.1378/chest.130.6.1844.View ArticlePubMedGoogle Scholar
Lavorini F, Magnan A, Dubus JC, Voshaar T, Corbetta L, Broeders M, Dekhuijzen R, Sanchis J, Viejo JL, Barnes P, Corrigan C, Levy M, Crompton GK: Effect of incorrect use of dry powder inhalers on management of patients with asthma and COPD. Respir Med. 2008, 102 (4): 593-604. 10.1016/j.rmed.2007.11.003.View ArticlePubMedGoogle Scholar
Melani AS, Bonavia M, Cilenti V, Cinti C, Lodi M, Martucci P, Serra M, Scichilone N, Sestini P, Aliani M, Neiri M: Inhaler mishandling remains common in real life and is associated with reduced disease control. Respir Med. 2011, 105 (6): 930-938. 10.1016/j.rmed.2011.01.005.View ArticlePubMedGoogle Scholar
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, Decristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.View ArticlePubMedGoogle Scholar
Fiks AG: Designing computerized decision support that works for clinicians and families. Curr Probl Pediatr Adolesc Health Care. 2011, 41 (3): 60-88. 10.1016/j.cppeds.2010.10.006.View ArticlePubMedPubMed CentralGoogle Scholar
Perez X, Wisnivesky JP, Lurslurchachai L, Kleinman LC, Kronish IM: Barriers to adherence to COPD guidelines among primary care providers. Respir Med. 2012, 106 (3): 374-381. 10.1016/j.rmed.2011.09.010.View ArticlePubMedGoogle Scholar
Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003, 362 (9391): 1225-1230. 10.1016/S0140-6736(03)14546-1.View ArticlePubMedGoogle Scholar
Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765-10.1136/bmj.38398.500764.8F.View ArticlePubMedPubMed CentralGoogle Scholar
Holbrook A, Keshavjee K, Lee H, Bernstein B, Chan D, Thabane L, Gerstein H, Troyan S, COMPETE II Investigators: Individualized electronic decision support and reminders can improve diabetes care in the community. AMIA Annu Symp Proc. 2005, 2005: 982-PubMed CentralGoogle Scholar
Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H: Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009, 181 (1–2): 37-44.View ArticlePubMedPubMed CentralGoogle Scholar
Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, Bates DW: Impact of computerized decision support on blood pressure management and control: a randomized controlled trial. J Gen Intern Med. 2008, 23 (4): 429-441. 10.1007/s11606-007-0403-1.View ArticlePubMedPubMed CentralGoogle Scholar
Borbolla D, Giunta D, Figar S, Soriano M, Dawidowski A, de Quiros FG: Effectiveness of a chronic disease surveillance systems for blood pressure monitoring. Stud Health Technol Inform. 2007, 129 (1): 223-227.PubMedGoogle Scholar
Bertoni AG, Bonds DE, Chen H, Hogan P, Crago L, Rosenberger E, Barham AH, Clinch CR, Goff DC: Impact of a multifaceted intervention on cholesterol management in primary care practices: guideline adherence for heart health randomized trial. Arch Intern Med. 2009, 169 (7): 678-686. 10.1001/archinternmed.2009.44.View ArticlePubMedPubMed CentralGoogle Scholar
Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, Maislos M, Peleg R, Liss Z, Rabinowitz G, Vardy D, Zahger D, Ilia R, Leibermann N, Porath A: Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Israel Med Assoc J. 2009, 11 (1): 23-29.Google Scholar
Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, Peek N: Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009, 338: b1440-10.1136/bmj.b1440.View ArticlePubMedPubMed CentralGoogle Scholar
Jeffery R, Iserman E, Haynes RB, CDSS Systematic Review Team: Can computerized clinical decision support systems improve diabetes management? A systematic review and meta-analysis. Diabet Med. 2013, 30 (6): 739-745. 10.1111/dme.12087.View ArticlePubMedGoogle Scholar
Roshanov PS, Shika M, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB: Computerized clinical decision support systems for chronic disease management: a decision maker-researcher partnership systematic review. Implement Sci. 2011, 6: 92-10.1186/1748-5908-6-92.View ArticlePubMedPubMed CentralGoogle Scholar
Caroll AE, Anand V, Dugan T, Sheley M, Xu SZ, Down SM: Increased physician diagnosis of asthma with the child health improvement through computer automation decision support system. Pediatr Allergy Immunol. 2012, 25 (3): 168-171. 10.1089/ped.2012.0143.View ArticleGoogle Scholar
Hashimoto S, Brinke AT, Roldaan AC, van Veen IH, Möller GM, Sont JK, Weersink EJ, van der Zee JS, Braunstahl GJ, Zwinderman AH, Sterk PJ, Bel EH: Internet-based tapering of oral corticosteroids in severe asthma: a pragmatic randomised controlled trial. Thorax. 2011, 66 (6): 514-520. 10.1136/thx.2010.153411.View ArticlePubMedGoogle Scholar
Van der Meer V, Bakker MJ, van den Hout WB, Rabe KF, Sterk PJ, Kievit J, Assendelft WJ, Sont JK, SMASHING (Self-Management in Asthma Supported by Hospitals, ICT, Nurses and General Practitioners) Study Group: Internet-based self-management plus education compared with usual care in asthma: a randomized trial. Ann Intern Med. 2009, 151 (2): 110-120. 10.7326/0003-4819-151-2-200907210-00008.View ArticlePubMedGoogle Scholar
Van der Meer V, van Stel HF, Bakker MJ, Roldaan AC, Assendelft WJ, Sterk PJ, Rabe KF, Sont JK, SMASHING (Self-Management of Asthma Supported by Hospitals, ICT, Nurses and General practitioners) Study Group: Weekly self-monitoring and treatment adjustment benefit patients with partly controlled and uncontrolled asthma: an analysis of the SMASHING study. Respir Res. 2010, 11: 74-10.1186/1465-9921-11-74.View ArticlePubMedPubMed CentralGoogle Scholar
Taylor B, Dinh M, Kwok R, Dinh D, Chu M, Tang E: Electronic interface for emergency department management of asthma: a randomized control trial of clinician performance. Emerg Med Australas. 2008, 20 (1): 38-44. 10.1111/j.1742-6723.2007.01040.x.View ArticlePubMedGoogle Scholar
Fiks RW, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, Bell LM, Alessandrini EA: Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009, 124 (1): 159-169. 10.1542/peds.2008-2823.View ArticlePubMedGoogle Scholar
Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, Stephens TB, Swietlik M, Guevara JP: Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010, 125 (4): 770-777. 10.1542/peds.2009-1385.View ArticleGoogle Scholar
Rasmussen LM, Phanareth K, Nolte H, Backer V: Internet-based monitoring of asthma: a long-term, randomized clinical study of 300 asthmatic subjects. J Allergy Clin Immunol. 2005, 115 (6): 1137-1142. 10.1016/j.jaci.2005.03.030.View ArticlePubMedGoogle Scholar
Dexheimer JW, Abramo TJ, Arnold DH, Johnson KB, Shyr Y, Ye F, Fan KH, Patel N, Aronsky D: An asthma management system in a pediatric emergency department. Int J Med Inform. 2013, 82 (4): 230-238. 10.1016/j.ijmedinf.2012.11.006.View ArticlePubMedGoogle Scholar
Smith JR, Noble MJ, Musgrave S, Murdoch J, Price GM, Barton GR, Windley J, Holland R, Harrison BD, Howe A, Price DB, Harvey I, Wilson AM: The at-risk registers in severe asthma (ARRISA) study: a cluster-randomised controlled trial examining effectiveness and costs in primary care. Thorax. 2012, 67 (12): 1052-1060. 10.1136/thoraxjnl-2012-202093.View ArticlePubMedGoogle Scholar
Kattan M, Crain EF, Steinbach S, Visness CM, Walter M, Stout JW, Evans R, Smartt E, Gruchalla RS, Morgan WJ, O’Connor GT, Mitchell HE: A randomized clinical trial of clinician feedback to improve quality of care for inner-city children with asthma. Pediatrics. 2006, 117 (6): 1095-1103. 10.1542/peds.2005-2160.View ArticleGoogle Scholar
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005, 40 (2): 477-497. 10.1111/j.1475-6773.2005.0t369.x.View ArticlePubMedPubMed CentralGoogle Scholar
Martens JD, van der Aa A, Panis B, van der Weijden T, Winkens RA, Severens JL: Design and evaluation of a computer reminder system to improve prescribing behaviour of GPs. Stud Health Technol Inform. 2006, 124: 617-623.PubMedGoogle Scholar
Martens JD, van der Weijden T, Severens JL, de Clercq PA, de Bruijn DP, Kester AD, Winkens RA: The effect of computer reminders on GPs’ prescribing behaviour: a cluster-randomised trial. Int J Med Inform. 2007, 76 (3): 403-416.View ArticleGoogle Scholar
Martens JD, van der Weijden T, Winkens RA, Kester AD, Geerts PJ, Evers SM, Severens JL: Feasibility and acceptability of a computerised system with automated reminders for prescribing behaviour in primary care. Int J Med Inform. 2008, 77 (3): 199-207. 10.1016/j.ijmedinf.2007.05.013.View ArticlePubMedGoogle Scholar
Kuilboer MM, van Wijk MA, Mosseveld M, van der Does E, de Jongste JC, Overbeek SE, Ponsioen B, van der Lei J: Computed critiquing integrated into daily clinical practice affects physicians behaviour- a randomized clinical trial with AsthmaCritic. Methods Inf Med. 2006, 45 (4): 447-454.PubMedGoogle Scholar
Poels PJ, Schermer TR, Schellekens DP, Akkermans RP, de Vries Robbé PF, Kaplan A, Bottema BJ, van Weel C: Impact of a spirometry expert system on general practitioners’ decision making. Eur Respir. 2008, 31 (1): 84-92. 10.1183/09031936.00012007.View ArticleGoogle Scholar
Poels PJ, Schermer TR, Thoonen BP, Jacobs JE, Akkermans RP, de Vries Robbé PF, Quanjer PH, Bottema BJ, van Weel C: Spirometry expert support in family practice: a cluster randomised trial. Prim Care Respir J. 2009, 18 (3): 189-197. 10.4104/pcrj.2009.00047.View ArticlePubMedGoogle Scholar
Frickton J, Rindal DB, Rush W, Flottemesch T, Vazquez G, Thoele MJ, Durand E, Enstad C, Rhodus N: The effect of electronic health records on the use of clinical care guidelines for patients with medically complex conditions. J Am Dent Assoc. 2011, 142 (10): 1133-1142. 10.14219/jada.archive.2011.0082.View ArticleGoogle Scholar
Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D: Effect of clinical decision-support systems a systematic review. Ann of Intern Med. 2012, 157 (1): 29-43. 10.7326/0003-4819-157-1-201207030-00450.View ArticleGoogle Scholar
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-10.1136/bmj.b2535.View ArticlePubMedPubMed CentralGoogle Scholar
Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A: Clinical decision support systems could be modified to reduce “alert fatigue” while still minimizing the risk of litigation. Health Aff (Millwood). 2011, 30 (12): 2310-2317. 10.1377/hlthaff.2010.1111.View ArticleGoogle Scholar
Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS: Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003, 163 (21): 2625-2631. 10.1001/archinte.163.21.2625.View ArticlePubMedGoogle Scholar
Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ, Weingart SN: Overrides of medication alerts in ambulatory care. Arch Intern Med. 2009, 169 (3): 305-311. 10.1001/archinternmed.2008.551.View ArticlePubMedGoogle Scholar
Van Der Sijs H, Aarts J, Vulto A, Berg M: Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006, 13 (2): 138-147. 10.1197/jamia.M1809.View ArticlePubMedPubMed CentralGoogle Scholar
Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HGC, Garg AX, Haynes RB: Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013, 346: 657-10.1136/bmj.f657.View ArticleGoogle Scholar
Bryan C, Boren SA: The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Informatics Prim Care. 2008, 16 (2): 79-91.Google Scholar
Kastner M, Straus SE: Clinical decision support tools for osteoporosis disease management: a systematic review of randomized controlled trials. J Gen Intern Med. 2008, 23 (12): 2095-2105. 10.1007/s11606-008-0812-9.View ArticlePubMedPubMed CentralGoogle Scholar
Anchala R, Pinto MP, Shroufi A, Chowdhury R, Sanderson J, Johnson L, Blanco P, Prabhakaran D, Franco OH: The role of Decision Support System (DSS) in prevention of cardiovascular disease: a systematic review and meta-analysis. PLoS One. 2012, 7 (10): e47064-10.1371/journal.pone.0047064.View ArticlePubMedPubMed CentralGoogle Scholar
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D: The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011, 30 (3): 464-471. 10.1377/hlthaff.2011.0178.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
