- Research article
- Open Access
- Open Peer Review
Effectiveness of a standardized electronic admission order set for acute exacerbation of chronic obstructive pulmonary disease
BMC Pulmonary Medicine volume 18, Article number: 93 (2018)
Variation in hospital management of patients with acute exacerbation of chronic obstructive pulmonary disease (AECOPD) may prolong length of stay, increasing the risk of hospital-acquired complications and worsening quality of life. We sought to determine whether an evidence-based computerized AECOPD admission order set could improve quality and reduce length of stay.
The order set was designed by a provincial COPD working group and implemented voluntarily among three physician groups in a Canadian tertiary-care teaching hospital. The primary outcome was length of stay for patients admitted during order set implementation period, compared to the previous 12 months. Secondary outcomes included length of stay of patients admitted with and without order set after implementation, all-cause readmissions, and emergency department visits.
There were 556 admissions prior to and 857 admissions after order set implementation, for which the order set was used in 47%. There was no difference in overall length of stay after implementation (median 6.37 days (95% confidence interval 5.94, 6.81) pre-implementation vs. 6.02 days (95% confidence interval 5.59, 6.46) post-implementation, p = 0.26). In the post-implementation period, order set use was associated with a 1.15-day reduction in length of stay (95% confidence interval − 0.5, − 1.81, p = 0.001) compared to patients admitted without the order set. There was no difference in readmissions.
Use of a computerized guidelines-based admission order set for COPD exacerbations reduced hospital length of stay without increasing readmissions. Interventions to increase order set use could lead to greater improvements in length of stay and quality of care.
Chronic obstructive pulmonary disease (COPD) is a common and progressive lung disease that is characterized by shortness of breath, activity limitation, and a predisposition to exacerbations. Acute exacerbations of COPD (AECOPD) adversely affect quality of life,  increase the risk of disease progression,  and reduce survival.  Hospitalizations for AECOPD cost approximately USD $3.8 billion  and account for 51% of overall expenditures for COPD.  Prolonged hospital length of stay (LOS) also have a negative impact on patient function and quality of life. 
Evidence-based management guidelines for AECOPD have been developed, [7, 8] and include recommendations regarding pharmacotherapy and post-exacerbation care. Despite these guidelines, hospital care of patients with AECOPD remains highly variable.  This variation may contribute to prolonged LOS  that, in turn, increases the risk of hospital-acquired complications and adversely impacts quality of life. [6, 11]
Order sets are grouped medical orders intended to standardize evidence-based best practice. Computerized Physician Order Entry (CPOE) systems may improve workflow, promote appropriate testing and treatment, reduce errors and improve guideline adherence, [12,13,14,15,16,17] particularly when integrated into general order sets.  Standardized admission order sets have been used in other diseases with variable success at reducing hospital LOS. [14, 15]
Two observational studies have demonstrated that order sets likely improve the quality of hospital care for patients with AECOPD and reduce LOS. [13, 16] However, these studies used pre-post designs that could be influenced by secular trends in AECOPD management, and the studies did not account for the differential effects of the order set among physician groups.
The objective of this study was to determine whether the implementation of an evidence-based computerized admission order set would improve the quality of inpatient AECOPD care. A stepped wedge design was used to account for differential effects among physician groups and to minimize confounding related to the timing of order set implementation. Our hypothesis was that the implementation of a standardized order set would reduce hospital LOS of patients admitted for AECOPD without increasing emergency department (ED) or hospital readmissions. Preliminary study results have previously been reported in abstract form. 
This study is an analysis of administrative health data for a quality improvement project in which an electronic standardized admission order set for patients with AECOPD was implemented at a large, tertiary-care teaching hospital in Calgary, Alberta between March 1, 2013 and March 31, 2015. Since this was a quality improvement project, the University of Calgary Conjoint Health Research Ethics Board waived the requirement for formal ethics approval.
Patients were included if they were: older than 45 years of age; admitted to hospital between March 1, 2013 to March 31, 2015 with an International Classification of Diseases, Tenth Revision (ICD-10-CA) code indicative of AECOPD (J42 [unspecified chronic bronchitis], J43 [emphysema], or J44 [other chronic obstructive pulmonary disease]) in the primary diagnosis field of the hospital discharge abstract database; and admitted to the pulmonary, general internal medicine or hospitalist clinical services. Patients were excluded if they were admitted to the intensive care unit or any other clinical service. Historical controls from the 12 months prior to order set implementation in each group of ordering physicians were identified using similar criteria. Additional details on the methods are provided in an additional file (see Additional file 1).
Order set development
The AECOPD order set was based on published COPD guidelines,  and developed by a provincial COPD working group comprised of physicians, nurses, and respiratory therapists from a variety of clinical settings, in a series of face-to-face and teleconference meetings.
The order set contained recommended testing, medication (including suggested dosing and mode of delivery), consultations, and a priori discharge planning interventions specific to patients with AECOPD. Some interventions were pre-selected to encourage use (e.g., physiotherapy referral). The order set was built into the hospital’s existing CPOE system, Sunrise Clinical Manager (Allscripts Solutions, Chicago IL). Screenshots are provided in an additional file (see Additional file 2).
The order set was implemented using a stepped wedge design  among the three physician groups who admit patients with AECOPD: respirologists, general internists, and family physician hospitalists. It was implemented sequentially within physician groups with each group acting as its own control. Study outcome data were collected at baseline and at each implementation ‘step’.
Implementation among respirologists, general internists and hospitalists occurred in March, May and August 2013, respectively. Prior to each implementation step, the research team met with physicians and allied health staff to introduce the order set. Order set use by each individual physician was voluntary. Monthly statistics on order set use were posted in clinical areas.
Patient demographic, comorbidity and hospitalization data were obtained from provincial administrative data and linked to order set usage data from the CPOE system using the patient’s provincial health number. 
The primary outcome was hospital LOS for patients admitted during the implementation period compared to those admitted during the previous 12 months (pre-post implementation analysis). Secondary outcomes included: hospital LOS of patients admitted with and without the order set after implementation (post-implementation analysis); all-cause readmissions at 7, 30 and 90 days after discharge; ED visits at 7 and 30 days; and in-hospital mortality.
Unadjusted and adjusted median regression models were constructed to assess the impact of the order set on LOS. [22,23,24] Covariates in adjusted models included age, sex, and five clinically relevant comorbidities (heart failure, dementia, liver disease, renal disease, and diabetes) that were strongly associated with the Charlson Comorbidity Index (Somers’ D = 0.94).  Logistic regression was used to adjust 30-day readmission odds ratios for age, sex, comorbidity and admitting physician specialty.
All analyses were performed using SAS version 9.3 (Cary, NC) or R version 3.2.3;  p < 0.05 was considered statistically significant.
Of 1435 AECOPD admissions to one of the three physician groups during the study period, 1413 with a LOS less than 90 days were included in the analysis (Fig. 1). There were 857 admissions after order set implementation, of which 406 patients (47%) were admitted using the order set.
Baseline characteristics of study participants are presented in Tables 1 and 2 for the pre-post and post-implementation analyses, respectively. The hospitalist service admitted most patients with AECOPD, but admitted fewer in the post-implementation period compared to the pre-implementation period (64.5% vs. 74.3%). Patients with co-existing heart failure and diabetes were more commonly admitted under general internists. Over 95% of patients were discharged home.
Order set uptake
In the post-implementation period, 57% of patients admitted to the hospitalist service were admitted using the order set, compared to 30% of patients admitted by general internists or respirologists. Time series analysis revealed that order set use increased gradually after implementation, mostly by general internists and hospitalists (Additional file 3: Figure S1).
Figure 2 shows the unadjusted and adjusted differences in median LOS for patients treated in the pre- and post-implementation periods. Median LOS was 6.37 days (95% confidence interval [CI] 5.94, 6.81; n = 556) for patients admitted before implementation and 6.02 days (95% CI 5.59, 6.46; n = 857) for patients admitted after implementation (p = 0.26).
Unadjusted and adjusted comparisons of median LOS in the post-implementation analysis are presented in Fig. 3. Order set use was associated with a 1.15-day (95% CI -0.50, − 1.81) shorter median LOS, due primarily to a 1.8-day (95% CI -0.95, − 2.61) decrease for the hospitalist group (Fig. 3). Median LOS for patients admitted by general internists or respirologists did not differ by order set use.
Neither order set implementation, nor order set use in the post-implementation period were associated with changes in readmissions or ED visits for all three physician groups (Tables 3 and 4 and Additional file 1 Table S1). Overall in-hospital mortality did not change with order set implementation, but was lower in the hospitalist group with order set use.
This is the largest study to evaluate the impact of a standardized, guideline-based electronic order set for AECOPD on hospital LOS. We performed two analyses: a comparison of LOS before and after the order set was implemented, and a comparison of patients admitted with and without the order set after implementation. The results revealed that order set implementation did not result in an overall LOS reduction, perhaps because only 47% of admitting physicians used it. However, the post-implementation analysis revealed that when it was used, the order set was associated with a LOS reduction of 1.15 days. Use of the order set by hospitalists, who admitted 65% of AECOPD patients in the post-implementation cohort, resulted in the largest LOS reduction of 1.8 days. Importantly, there was no increase in either ED or hospital readmissions, suggesting that earlier discharge resulting from order set use did not occur at the expense of harm to the patient.
Our findings extend observations from two recent studies examining the impact of order sets on AECOPD care. In a pre-post design of 243 patients hospitalized with AECOPD, Kitchlu et al. showed that implementation of an order set improved the quality of admission orders using pre-specified measures of guidelines-based care.  Using a similar design in a study of 275 patients, Brown et al. showed that physician prescribing practices for AECOPD could be improved with an electronic order set.  Secondary analyses in both studies revealed LOS reductions without an increase in readmissions. The current study extends this work by demonstrating an improvement in hospital LOS in a larger study cohort. Unlike the previous studies, which reported on pre-post effects of order set implementation, we specifically examined the effect of actual use of the order set, demonstrating that it could reduce LOS compared to patients admitted without the order set. Furthermore, the stepped wedge design is robust to secular trends in care delivery and LOS of AECOPD patients, and allowed for subgroup analyses by different admitting physician groups. Both our study and the previous studies provide a strong rationale for the standardization of inpatient AECOPD care using computerized order sets.
A standardized order set is appealing due to high variability in inpatient AECOPD management.  Previous studies of order sets for AECOPD demonstrated more consistent use of systemic corticosteroids, appropriate antibiotics, and allied health providers such as physiotherapists, [13, 16] all of which have been shown to reduce hospital LOS. [27,28,29] The current order set similarly prompted admitting physicians to use these therapies, and pre-selection of some items (e.g., bronchodilator delivery, physiotherapy referral) provided additional clinical decision support around guidelines-based care.
The LOS reduction observed after order set implementation was driven by improvements for patients admitted by hospitalists, with no differences observed in the other physician groups; this was an interesting finding with many possible explanations. First, Sandhu et al. demonstrated high variability in AECOPD management among all specialties, with deviation from clinical guidelines occurring more often when care was provided by physicians other than respirologists.  This variability may be due to the diversity of medical problems managed by hospitalists, which could lead to a less uniform approach to inpatient AECOPD management. Thus, it is possible that the opportunity for standardization using an order set was greater for hospitalists than for other specialties. Second, heart failure and diabetes were more prevalent in patients admitted under general internists compared to hospitalist patients. These conditions have both been associated with longer LOS in patients with AECOPD,  and are unlikely to be impacted by order set use. Third, patients presenting with respiratory failure requiring noninvasive ventilation were only admitted by general internists or respirologists; these indicators of more severe AECOPD were not systematically captured, but could have reduced the effectiveness of the order set.  Although the order set’s impact seemed to be isolated to only one admitting group, the 1.8-day reduction in median LOS is an important finding since hospitalists were responsible for providing almost two thirds of inpatient AECOPD care in our study, and this is likely to be similar in other large, tertiary-care urban hospitals in North America.
The order set was used by 47% of admitting physicians during the study period. This low uptake is a consistent finding for voluntary order sets [13, 16, 32, 33] and is a known limitation of their use.  Respirologists and general internists used the order set less frequently than hospitalists, for a number of possible reasons. First, the complexity of the patient’s presentation (e.g., respiratory failure requiring noninvasive ventilation) may have made the order set less applicable at the time of admission. Second, respirologists may have greater perceived self-efficacy with AECOPD management, leading them to admit AECOPD patients without using an order set. Finally, whereas the AECOPD order set was the only respiratory order set embedded in the CPOE system, several admission order sets for medical problems typically admitted under hospitalists (e.g., pneumonia, heart failure) were already embedded. Thus, hospitalists may have more experience in order set use compared to other physicians. Importantly, end users were consulted to ensure the order set was intuitive and minimally disruptive to clinical workflow; these factors have been shown to increase uptake of clinical decision support systems such as standardized order sets. [14, 17, 18] The increase in order set use over time suggested that admitting physicians found it useful.
This study has a number of limitations. First, the non-randomized study design raises the possibility that improvements in LOS were due to other differences between groups admitted with and without the order set. However, when implementing complex healthcare interventions such as the AECOPD order set, traditional randomized controlled trials are impractical due to logistical constraints and the risk of contamination within clinical provider groups. The methodologically robust stepped wedge design minimized contamination by allowing implementation and evaluation of the order set in clusters  while still analyzing the effect of order sets within physician groups.
Second, the use of administrative data prevented analysis of patient characteristics that might have influenced a physician’s decision to use the order set; it is thus possible that the reduced LOS in the post-implementation period is due to order set use in less complex cases. While our findings are consistent with studies performed in different geographic and clinical settings, [13, 16] we acknowledge the importance of future studies to examine whether COPD severity, presentation acuity, or use of specific interventions (e.g., noninvasive ventilation) impact order set use. Such an analysis would help to tailor strategies aimed at increasing order set uptake by specific physician groups.
The lack of clinical data on COPD severity (e.g., spirometry) or baseline performance status also precluded the determination of differential effects of the order set between COPD subgroups; it is also possible that these factors influenced LOS or readmission rates independently of the order set. However, this information is also often not available to clinicians at the time of admission. Thus, we chose to develop an order set that could be used for all patients admitted with AECOPD, consistent with actual clinical practice. Our statistical models did account for age, sex and clinically relevant comorbidities, indicating that our results were robust to these covariates. Future studies could further evaluate how patient characteristics impact order set use and outcomes from the order set.
Finally, we did not analyze individual components of the AECOPD order set, and thus do not know which orders were actually selected or executed (e.g., physiotherapy referral). We also cannot confirm whether there was concordance between pre-checked orders and actual orders selected by admitting physicians. These components may have differential impact on LOS and could help refine the order set. An understanding of how individual components were used may also help to identify areas for focused quality improvement. While the intent of this study was to evaluate the effectiveness of a comprehensive bundle of orders, the improvement in LOS provides compelling evidence to justify a secondary analysis of individual order set components.
In conclusion, this study found that when a standardized electronic order set was used to admit patients with AECOPD, LOS was reduced without increasing readmissions. Innovations such as order sets have the potential to lessen the burden of AECOPD hospitalizations on both patients and the healthcare system, and justify additional studies of clinical decision support tools for AECOPD.
Acute exacerbation of chronic obstructive pulmonary disease
Chronic obstructive pulmonary disease
Computerized physician order entry
Length of stay
Seemungal TA, Donaldson GC, Paul EA, Bestall JC, Jeffries DJ, Wedzicha JA. Effect of exacerbation on quality of life in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 1998;157(5):1418–22.
Kanner RE, Anthonisen NR, Connett JE. Lung health study research group. Lower respiratory illnesses promote FEV(1) decline in current smokers but not ex-smokers with mild chronic obstructive pulmonary disease: results from the lung health study. Am J Respir Crit Care Med. 2001;164(3):358–64.
Connors AF, Dawson NV, Thomas C, et al. Outcomes following acute exacerbation of severe chronic obstructive lung disease. The SUPPORT investigators (study to understand prognoses and preferences for outcomes and risks of treatments). Am J Respir Crit Care Med. 1996;154(4 Pt 1):959–67.
Wier LM, Elixhauser A, Pfuntner A, Au DH. Overview of hospitalizations among patients with COPD. Healthcare utilization project statistical brief #106. Rockville, MD: Agency for Healthcare Research and Quality; 2008. p. 2011.
Waye AE, Jacobs P, Ospina MB, Stickland MK, Mayers I. Economic surveillance for chronic obstructive pulmonary disease (COPD) in Alberta. Edmonton, AB: Institute for Health Economics; 2016.
Wang Q, Bourbeau J. Outcomes and health-related quality of life following hospitalization for an acute exacerbation of COPD. Respirol. 2005;10(3):334–40.
O’Donnell DE, Hernandez P, Kaplan A, et al. Canadian Thoracic Society recommendations for management of chronic obstructive pulmonary disease – 2008 update – highlights for primary care. Can Respir J. 2008;15(Suppl A):1A–8A.
Vestbo J, Hurd SS, Agustí AG, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med. 2013;187(4):347–65.
Sandhu SK, Chu J, Yurkovich M, Harriman D, Taraboanta C, Fitzgerald JM. Variations in the management of acute exacerbations of chronic obstructive pulmonary disease. Can Respir J. 2013;20:175–9.
Noon CE, Hankins CT, Cote MJ. Understanding the impact of variation in the delivery of healthcare services. J Healthc Manag. 2003;48(2):82–97.
Hoogerduijn JG, Schuurmans MJ, Duijnstee MS, de Rooij SE, Grypdonck MF. A systematic review of predictors and screening instruments to identify older hospitalized patients at risk for functional decline. J Clin Nurs. 2007;16(1):46–57.
Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280(15):1311–6.
Brown KE, Johnson KJ, Deronne BM, Parenti CM, Rice KL. Order set to improve the care of patients hospitalized for COPD exacerbations. Ann Am Thorac Soc. 2016;13:811–5.
Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293:1223–38.
Gillaizeau F, Chan E, Trinquart L, et al. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2013;11(11):CD002894.
Kitchlu A, Abdelshaheed T, Tullis E, Gupta S. Gaps in the inpatient management of chronic obstructive pulmonary disease exacerbation and impact of an evidence-based order set. Can Respir J. 2015;22:157–62.
Miller RA, Waitman LR, Chen S, Rosenbloom ST. The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform. 2005;38:469–85.
Munasinghe RL, Arsene C, Abraham TK, Zidan M, Siddique M. Improving the utilization of admission order sets in a computerized physician order entry system by integrating modular disease specific order subsets into a general medicine admission order set. J Am Med Inform Assoc. 2011;18(3):322–6.
Pendharkar S, Hirani N, Faris P, et al. Effectiveness of a standardized inpatient admission order set for acute exacerbation of COPD [abstract]. Am J Respir Crit Care Med. 2015;191:A6179.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomized trial: rationale, design, analysis, and reporting. Br Med J. 2015;350:h391.
Howe GR. Use of computerized record linkage in cohort studies. Epidemiol Rev. 1998;20:112–21.
Koenker R. Quantile Regression [computer program]. 2015. https://cran.r-project.org/web/packages/quantreg/index.html. Accessed 22 July 2016.
Koenker R, Bassett G Jr. Regression quantiles. Econometrica. J Econom Soc. 1978;46:33–50.
Portnoy S, Koenker R. The gaussian hare and the laplacian tortoise: computability of squared-error versus absolute-error estimators. Stat Sci. 1997;12(4):279–300.
Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83.
R. A language and environment for statistical computing [computer program]. Vienna: R Foundation for Statistical Computing; 2015.
Niewoehner DE, Erbland ML, Deupree RH, et al. Effect of systemic glucocorticoids on exacerbations of chronic obstructive pulmonary disease. Department of Veterans Affairs cooperative study group. N Engl J Med. 1999;340:1941–7.
Quon BS, Gan WQ, Sin DD. Contemporary management of acute exacerbations of COPD: a systematic review and metaanalysis. Chest. 2008;133:756–66.
de Morton NA, Keating JL, Jeffs K. Exercise for acutely hospitalised older medical patients. Cochrane Database Syst Rev. 2007;1:CD005955.
Wang Y, Stavem K, Dahl FA, Humerfelt S, Haugen T. Factors associated with a prolonged length of stay after acute exacerbation of chronic obstructive pulmonary disease (AECOPD). Int J Chron Obstruct Pulmon Dis. 2014;9:99–105.
Matkovic Z, Huerta A, Soler N, et al. Predictors of adverse outcome in patients hospitalized for exacerbation of chronic obstructive pulmonary disease. Respiration. 2012;84:17–26.
Wright A, Feblowitz JC, Pang JE, et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform. 2012;81(11):733–45.
Lennox L, Green S, Howe C, Musgrave H, Bell D, Elkin S. Identifying the challenges and facilitators of implementing a COPD care bundle. BMJ Open Respir Res. 2014;1:e000035.
Bobb AM, Payne TH, Gross PA. Viewpoint: controversies surrounding use of order sets for clinical decision support in computerized provider order entry. J Am Med Inform Assoc. 2007;14:41–7.
Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf. 2015;24:325–36.
The authors acknowledge the members and staff of Alberta Health Services’ Respiratory Health Strategic Clinical Network, who helped with design and implementation of the order set.
This project was supported by Alberta Health Services’ Respiratory Health Strategic Clinical Network Seed Grant. Members of the Respiratory Health Strategic Clinical Network were involved in the design of the study and revision of the manuscript. However, the seed grant used to fund this project was obtained through an independent peer review process.
Availability of data and materials
The datasets analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The University of Calgary Conjoint Health Research Ethics Board waived the requirement for formal ethics approval.
Dr. Leigh reports personal fees from AstraZeneca, grants from AstraZeneca, personal fees from Boehringer Ingelheim, grants from GlaxoSmithKline, grants from MedImmune, personal fees from Novartis, personal fees from TEVA Canada, outside the submitted work. The other authors have no disclosures.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplement with additional details on methods and results (DOCX 28 kb)
Screenshot of AECOPD order set (PDF 400 kb)
Figure S1. Monthly percentage of patients admitted using AECOPD order set during study period. Vertical lines represent implementation start dates for each physician specialty. Respirologist represented by hatched line; general internist represented by black solid line; hospitalist represented by grey solid line. Reported probabilities are for linear trends from time series models for each physician specialty. These models showed no evidence of seasonality or auto-regression (JPG 112 kb)