Diagnostic accuracy of spirometry in primary care

Background To evaluate the sensitivity, specificity and predictive values of spirometry for the diagnosis of chronic obstructive pulmonary disease (COPD) and asthma in patients suspected of suffering from an obstructive airway disease (OAD) in primary care. Methods Cross sectional diagnostic study of 219 adult patients attending 10 general practices for the first time with complaints suspicious for OAD. All patients underwent spirometry and structured medical histories were documented. All patients received whole-body plethysmography (WBP) in a lung function laboratory. The reference standard was the Tiffeneau ratio (FEV1/VC) received by the spirometric maneuver during examination with WBP. In the event of inconclusive results, bronchial provocation was performed to determine bronchial hyper-responsiveness (BHR). Asthma was defined as a PC20 fall after inhaling methacholine concentration ≤ 16 mg/ml. Results 90 (41.1%) patients suffered from asthma, 50 (22.8%) suffered from COPD, 79 (36.1%) had no OAD. The sensitivity for diagnosing airway obstruction in COPD was 92% (95%CI 80–97); specificity was 84% (95%CI 77–89). The positive predictive value (PPV) was 63% (95%CI 51–73); negative predictive value (NPV) was 97% (95%CI 93–99). The sensitivity for diagnosing airway obstruction in asthma was 29% (95%CI 21–39); specificity was 90% (95%CI 81–95). PPV was 77% (95%CI 60–88); NPV was 53% (95%CI 45–61). Conclusion COPD can be estimated with high diagnostic accuracy using spirometry. It is also possible to rule in asthma with spirometry. However, asthma can not be ruled out only using spirometry. This diagnostic uncertainty leads to an overestimation of asthma presence. Patients with inconclusive spirometric results should be referred for nitric oxide (NO) – measurement and/or bronchial provocation if possible to guarantee accurate diagnosis.


Background
Asthma is a common chronic disease with a high prevalence of approx. 5% in industrialized nations. It is characterized by a chronic inflammation process which induces bronchial hyper-responsiveness and in most cases, reversible airway obstruction [1]. Another common pulmonary disease is chronic obstructive pulmonary disease (COPD) which shows irreversible airway obstruction, and which is mostly caused by inhaling tobacco smoke [2]. The prevalence of COPD is estimated to be around 10% and expected to be the fourth most important cause of death in 2020 [3]. Due to this high morbidity, general practitioners play a key role in detecting the disease as they see patients during the earlier stages of disease. Spirometric investigation is seen as a gold standard for diagnosing airway obstruction. Therefore, office spirometry is increasingly seen as a quality standard in general practice [4,5].
The efficacy of spirometry in diagnosing COPD was demonstrated by a specialist team, which received referrals for performing spirometry and bronchodilator reversibility testing in patients suspected of having COPD [6]. The DIDASCO Study revealed the difficulty of diagnosing COPD with screening questionnaires only and concluded that spirometry is essential for early diagnosis [7]. These investigations focused on COPD only, which is marked by irreversible airway obstruction. The diagnostic value of spirometry for diagnosing asthma marked by reversible airway obstruction remains unclear. This is of importance, as asthma needs to be diagnosed by bronchial provocation testing when spirometry shows no airway obstruction [8]. One diagnostic study in primary care used spirometry and bronchial provocation testing for identifying patients with asthma and COPD [9]. However, this was only carried out in patients complaining of suffering from a cough; and spirometry was performed by a single specialist. Spirometry and bronchial provocation testing were also used in the DIMCA study [10]. Indeed this was a screening study performed in a specialist center to detect patients in early stadiums of asthma or COPD.
Due to the design of these asthma and COPD trials, there is no evidence of the diagnostic accuracy of spirometry itself. Therefore, the true degree of the associated diagnostic uncertainty for patients with complaints suspected of having an airway obstruction remains unclear. The need for closing this gap of knowledge has been pointed out several times [11,12]. The difficulty is that the pretest probability of a disease and its severity in primary care is lower when compared to a hospital setting, thus hampering the predictive values of diagnostic tests [13,14]. Therefore, test results evaluated in hospital settings can not easily be transferred into general practice [15]. The aim of this study was to investigate the sensitivity, specificity and predictive values of spirometry for diagnosing airway obstruction in asthma and COPD in general practice.

Design and sample
This cross-sectional study was performed between January 2006 and December 2007 with fourteen general practitioners (GPs) working in ten general practices. 219 patients visiting their GP for the first time with complaints suggestive of obstructive airway disease (OAD) were consecutively included in each practice. Patients visited their GPs with symptoms such as dyspnea, coughing or expectoration. Their medical history was taken with a structured questionnaire. The patients had not been diagnosed previously for OAD and they had not received any previous anti-obstructive medicine. Other exclusion criteria related to well known contra-indications for bronchodilator reversibility testing or bronchial provocation, namely untreated hyperthyreosis, unstable coronary artery disease, and cardiac arrhythmia. Pregnancy also led to exclusion. The study was approved by the Medical Ethics Committee of the University of Heidelberg. Patients gave written informed consent.
On the basis of the pilot study [16] we estimated the pretest probability of asthma as being 45% and of COPD as 16%. We estimated the sensitivity for diagnosing asthma to be 30% and the specificity to be 90%. The sensitivity and specificity for finding COPD was each estimated to be 90%. Power calculations showed that we had to include at least 208 patients to determine the sensitivities and specificities with a 95% confidence interval of ± 10% [17]. The diagnostic values of spirometry in general practice were calculated separately for each asthma and COPD group to avoid confusion. The diagnostic value for diagnosing asthma under optimal conditions was investigated by pooling all patients and determining the sensitivity, specificity and predictive values of spirometric maneuvers of the lung function laboratory.

Index test: Spirometry in general practice
Ten general practices were equipped with the same electronic spirometer (Medikro SpiroStar USB ® ) and associated spirometry software. The spirometer was a hand-held instrument for lung function testing that has to be connected via USB device to a computer. Spirometric data, flow-volume and volume-time graphs are displayed in real-time on the personal computer as the patient performs the spirometry test. A calibration file saves the calibration data for internal quality assurance. Instrument performance is regularly monitored and performance deviations are identified by the software. The software also compares the measured values with reference tables. The best of three consecutive spirometry recordings was used in accordance with the guidelines of the European Respiratory Society [18]. The maximal inspiratory and expiratory flow volume curves were generated by forced deep inspiration and expiration with short intervening periods of tidal breathing; patients used a nose clip. The maneuver was performed in a sitting position. Patients with a FEV 1 (forced expiratory volume in one second) < 80% of predicted received a bronchodilation test with an additional performance of spirometry 20 minutes after inhaling salbutamol. Obstructive airway disease was diagnosed if FEV 1 /VC ≤ 70% and/or FEV 1 < 80% [4,5]. Obstruction was considered to be reversible on salbutamol (which indicates a diagnosis of asthma) if the bronchodilation response Δ FEV 1 was ≥12% of the baseline and ≥200 ml [4] and norm values were reached. GPs were asked to make their diagnoses based on the test results.
All of the GPs were appropriately trained in the key aspects of the diagnosis and management of asthma and COPD, as well as in performing and interpreting spirometry during two educational meetings. The practice assistants completed an intensive 6-hour course and were trained in performing and interpreting spirometry. Two outreach visits were also performed with repeated individual education at a direct, practical level, until the optimal performance of the spirometry was secured.

Reference test: Bodyplethysmography and bronchial provocation
After diagnosis by their GP, all patients were referred to the lung function laboratory of the University Medical Hospital at once for investigation with whole-body plethysmography (WBP). If therapy was necessary due to asthma or COPD, it was initiated by the GP. However, patients were instructed not to use any bronchodilator or inhaled steroid twelve hours before visiting the lung function laboratory. Spirometry is normally the routine method for measuring the lung volume required to diagnose airflow obstruction -i.e. (forced) vital capacity ((F)VC) or FEV 1 . However, spirometry is not capable of providing information about intrathoracic residual volume or total airway resistance. A WBP is required to measure residual volume (RV), functional residual capacity (FRC), and total lung capacity (TLC). Therefore, the advantage of WBP over spirometry is that it is able to distinguish between restrictive and obstructive processes. Additionally, the resistance to airflow can be evaluated and the response of airway resistance, airway conductance and thoracic gas volume can be determined in response to bronchodilator reversibility testing and bronchial provocation. In particular circumstances, measurement of these lung volumes are strictly necessary for a correct physiological diagnosis [19,20]. However, as WBP is only common in highly developed health care systems and the added value on top of spirometry remains unclear, it is only recommended in a few guidelines [21][22][23].

Measurement technique of whole-body plethysmography and bronchial provocation
During WBP, the patient sits inside an airtight chamber and makes respiratory efforts against the closed shutter, causing chest volume to expand and decompressing the air in the lungs. The increase in chest volume reduces the box's volume, thus increasing the pressure in the box. The procedures were performed according to standard protocols [21]. Lung function reference values that had been adjusted for sex, age, and height were used [24]. Patients with FEV 1 < 80% of predicted received a bronchodilation test with an additional performance of WBP 20 minutes after inhaling salbutamol. An obstructive airway disease was diagnosed if FEV 1 < 80% and/or FEV 1 /VC ≤ 0.70. The obstruction was classified as reversible on a salbutamol (indicating asthma) when Δ FEV 1 was ≥12% and ≥200 ml from the baseline value [4] and norm values were reached. In all other cases, the obstruction was classified as not reversible ( Figure 1). If there was no bronchial obstruction, bronchial provocation was performed to determine bronchial hyper-responsiveness (BHR). Bronchial provocation is considered to be the best method for diagnosing asthma [25], although there is conflicting evidence [26], probably arising from variations in the population studied, as the diagnostic value increases with pre-test probability of the disease [27]. Trained lung function technicians measured bronchial hyper-responsiveness to methacholine according to the ATS guideline [8]. A diagnosis of 'asthma' was made if there was a 20% fall in FEV1 (PC 20 ) from the baseline value after inhaling methacholine stepwise until the maximum concentration (16 mg/ml). The pneumologist was blinded against the diagnosis of the GP.

Data analysis
The baseline data were presented descriptively. The sensitivity, specificity, and predictive values of the spirometric investigation (FEV 1 and/or FEV 1 /VC) in general practice were calculated with two-by-two contingency tables with the diagnosis of the pneumologist (WBP and bronchial provocation) as 'gold standard'. The data were analyzed with SPSS 15.0 for Windows. 95% confidence intervals were calculated using Wilson's method [28] with the statistical package CIA (Confidence Interval Analysis) [29]. An explanation of how to interpret PPV and NPV is provided in figure 2.

Study population
A total of 293 patients were assessed for eligibility ( Figure  3). 74 patients received spirometry but did not want to receive whole body plethysmography and eventually bronchial provocation. Therefore, altogether 219 patients participated in the study (127 [57.7%] were female) ( Table 1). The average age was 43.8 years. The average body mass index (BMI) was 25.3 (SD 4.4). Of the participating patients, 78 (35.6%) showed airway obstruction in general practice and 138 (63.0%) no abnormal findings in spirometry ( Figure 3). Three spirometric results were lost to follow-up. According to the diagnostic decision making in the lung function laboratory, 90 (41.1%) patients had asthma, 50 (22.8%) of the participating patients had irreversible airway obstruction (COPD), and 79 (36.1%) showed no abnormal findings. A diagnosis of asthma was made in 76 of the cases with bronchial provocation, with only 14 patients identified solely on the basis of bronchodilator reversibility testing. The decision that the bronchial provocation was positive was made in 74 cases by 20% fall of FEV 1 and in two cases by extreme increase of airway resistance accompanied by development of clinical symptoms of asthma during bronchial provocation. There were no significant differences in sex (p = 0.719) or obesity (BMI ≥ 30) (p = 0.272) between the diagnoses (chi-square test).

Performance of spirometry in general practice
Spirometry was performed with full adherence to ERS guidelines in 86 (39.8%) cases (Table 2). There was moderate adherence to ERS in 82 (38.0%) cases. In 48 (22.2%) cases the ERS criteria were not fulfilled. E.g., the flow-volume curves were deformed or not exactly reproduced. Altogether, 78 (36.1%) spirometric maneuvers showed airway obstruction. However, a bronchial reversibility test was only performed in 37 (47.4%) of these 78 cases.

Estimates of diagnostic accuracy of spirometry in general practice
In relation to the COPD diagnosis, 26 patients were diagnosed false positive ( Table 3). 12 of these spirometric maneuvers showed full/moderate adherence, and 14 were not according to guidelines. Four patients were diagnosed as false negative as the forced maneuvers in spirometry were performed weakly, thus resulting in a virtually normal Tiffeneau ratio. In these cases was FEV 1 > 80% of predicted and FEV1/VC < 0.70 in the WBP as reference standard. Sensitivity was 92% and specificity 84%. Thus the pretest probability could be enhanced reasonably from 23% to a posttest probability (PPV) of 63%; and COPD could be ruled out with high certainty (NPV 97%). 63 patients with asthma were diagnosed false negative as they showed no abnormal findings in spirometry (Table  4). It was only possible to identify them through bronchial provocation. Eight patients were diagnosed false positive; two of these spirometric maneuvers showed good adherence, and six were not according to guidelines. The pretest probability was enhanced from 41% up to 77%. However, asthma could not be ruled out, since NPV (53%) was similar to the pretest probability of 'not having asthma' (1-p = 59%); and 1-p was within the confidence interval of NPV (95%CI 45-61). The spirometric results as a part of the WBP investigation in the lung function laboratory are given in Table 5. Only 14 patients were identified by airway obstruction FEV 1 < 80% of predicted and Diagnostic decision making with the reference standard (whole-body plethysmography and bronchial provocation) in the lung function laboratory (COPD = Chronic obstructive pulmonary disease; OAD = obstructive airway disease) Figure 1 Diagnostic decision making with the reference standard (whole-body plethysmography and bronchial provocation) in the lung function laboratory (COPD = Chronic obstructive pulmonary disease; OAD = obstructive airway disease).

Yes No
No OAD positive bronchial reversibility testing. In addition to this, under these optimal conditions with optimal differentiation between asthma and COPD, the sensitivity for diagnosing asthma solely on basis of spirometric maneuvers was only 16%. Again, NPV was similar to the pretest probability of 'not having asthma'.

Diagnostic decision making by the GPs
The comparison of the diagnoses by the general practitioners with the diagnoses of the pneumologists demonstrated a reasonable agreement with respect to COPD (Table 6). Additionally, the GPs suspected asthma correctly in 76.7% of asthma cases despite the diagnostic uncertainty using spirometry. Indeed the prevalence of asthma was overestimated with 58.2% of healthy subjects suspected of having asthma; and 7.8% of patients with asthma were considered to be healthy.

Discussion
To our knowledge, this is the first study evaluating the diagnostic accuracy of spirometry for diagnosing airflow obstruction in patients with asthma or COPD in primary care. We found that the use of spirometry is feasible within general practice after training GPs and practice nurses. Under these conditions, the presence or absence of COPD can be estimated with a comparatively high diagnostic accuracy. It is also possible to rule in asthma. However, it was impossible to rule out asthma as the sensitivity was too low.
The prevalence of COPD is increasing in nearly all countries of the world and a high diagnostic accuracy is a prerequisite of optimal therapeutic management. The important role of spirometry for diagnosing airway obstruction has already been demonstrated [6,7,30,31]. However, the diagnostic accuracy of spirometry for diag-Calculation example for the relation between pretest probability, sensitivitiy, specificity, positive predictive value (PPV) and negative predictive value (NPV) Figure 2 Calculation example for the relation between pretest probability, sensitivitiy, specificity, positive predictive value (PPV) and negative predictive value (NPV).
Pretest probability of having the disease (p): is the prevalence of disease in the investigated population Pretest probability of not having the disease (1-p): is the prevalence of healthy subjects in the investigated population Sensitivity (sens): is the proportion of people with disease who have a positive test Specificity (spec): is the proportion of people free of a disease who have a negative test Positive predictive value (PPV): probability that a patient with a positive test has got really the disease Negative predictive value (NPV): probability that a patient with a negative test is really healthy This example illustrates, that a test is useful when the pretest probability is increased up to a reasonable PPV (rule in the disease) or when NPV is increased reasonably (rule out). The figures illustrate that specificity is more important to rule in (spin = specificity rule in); and that sensitivity is more important to rule out (snout = sensitivity rule out). The relation between p, sens, spez, PPV and NPV is described by the Bayes´Theorem. Indeed a test is only useful, if the pretest probability (p) is out of the range of 95%CI of PPV (in this example 95%CI = 80%-94%); and/or if the pretest probability of not having the disease (1-p) is out of the range of the 95%CI of NPV (in this example 95% CI = 72%-83%). The 95%CI is calculated using Wilson´s method. 28 In this example p is increased up to a meaningful PPV. NPV seems not to be increased reasonably if compared with 1-p.   [11.9] nosing COPD has been unknown up to now, thus leading to diagnostic uncertainty in suspected cases of COPD. Our results demonstrate that the pretest-probability of 22% of patients presenting themselves with complaints suggestive of airway obstruction can be increased up to a posttest probability of 63% for having COPD. This comparatively low PPV might be surprising, as the sensitivity was 84% and specificity was 92%. However, this is explainable by the low pretest probability. Another reason might be due to sub-maximal maneuvers, leading to false positive results by underestimation of FEV 1 [11]. As a consequence, more efforts in terms of continuous education would be necessary for an improvement of performance and an interpretation of spirometry. Nevertheless, COPD can be definitively excluded (NPV 97%) when spirometry is performed optimally. For these reasons, spirometry should be used regularly for diagnosing and managing COPD in primary care.
In contrast to these promising results is the limited value of spirometry in excluding asthma. This might be explained by the reversibility of airway obstruction in asthma. It proved possible to speculate that patients with mild or moderate asthma show no airway obstruction when spirometry is performed. In these cases, it was necessary for the GP to estimate the presence or absence of asthma on the basis of the patient history and inconclusive spirometry. This was misleading in 53 (24.2%) of cases (46 patients false positive and 7 patients false negative). Therefore, alternative methods need to be found for diagnosing asthma in primary care. Guidelines recommend using the measurement of peak-flow-variability to diagnose asthma in case of inconclusive spirometry. However, the low diagnostic value of peak-flow-variability in primary care has already been demonstrated [32]. The SAPALDIA study, which used an epidemiologic approach, has also shown a poor diagnostic value [33]. The measurement of exhaled nitric oxide (NO) which is elevated in eosinophilic airway inflammation [34] has been shown to be more promising [35], although the technology is expensive. Therefore, patients suspected of having asthma might be tested with NO measurement or should be referred for bronchial provocation if possible to guarantee    accurate diagnosis. Nevertheless, spirometry should be used in diagnosing asthma, as the positive predictive value has been comparatively high in general practice.
One important limitation was that 22% of the spirometric maneuvers were not performed correctly in general practice. However, with the analysis of the spirometric maneuvers as part of the WBP investigation in the lung function laboratory, we received accurate diagnostic values of spirometry. Our results revealed that the predictive values of general practice were slightly lower than in the lung function laboratory. In addition to this, it was not possible to include all patients consecutively, as some patients were not willing to travel to the lung function laboratory of the Medical Hospital. This might have led to an overestimation of the diagnostic accuracy of spirometry [36]. However, that would also emphasize the impossibility of excluding asthma solely with spirometry. Another limitation is due to the choice of the cut-off points. Our use of the ratio FEV 1 /VC ≤ 0.70 as is still recommended by GOLD [5] may have led to some overestimation of airway obstruction in older patients [37] and underestimation in younger patients [38]. The ATS/ERS guideline therefore suggests using lower limits of normal, which is statistically defined by the 5 th lower percentile of a reference population, to provide more accurate diagnoses [19]. This diagnostic algorithm was not integrated in the spirometric software at the time of our study. Moreover, we are aware of the limitations of a one-off lung function test to determine a final diagnosis, as a negative bronchodilator response can occur due to fixed airway obstruction in asthma. A trial of steroids might have been necessary to differentiate between asthma and COPD in some patients. Nevertheless, these limitations do not hamper our finding that asthma cannot be excluded solely with spirometry. The WBP showed little added value on top of spirometry. We used it as a reference standard to distinguish between overlapping diseases, COPD and restrictive lung disorder. However, we only experienced two changes in making the diagnosis with the added information of WBP. In two patients suffering from dyspnea attacks, the airway resistance was very high during bronchial provocation, but FEV 1 remained normal. Moreover, we found no patient with restrictive lung disorder, which indicates a low prevalence in primary care settings. Therefore, the added value of WBP for primary care is limited and it should be reserved for patients who are difficult to diagnose and show persistent complaints.
It was not possible to specify the alternate diagnosis of the patients with no OAD, which is a typical problem of diagnostic studies in primary care. It was impossible to perform every investigation (e.g. gastroscopy to determine gastro-oesophageal reflux; x-ray) until a definite diagnosis could be made. This would not have been allowed by the Ethics Committee. However, this limitation does not alter the results of spirometric investigation. Finally, the participating GPs and practice assistants were highly motivated and received intensive training. Nevertheless, 22% of the spirometric maneuvers showed no guideline adherence. In particular bronchodilation testing was not performed regularly which might be due to organisational reasons and time constraints in general practice. The GPs estimated fourteen patients to suffer from COPD. However, the final pneumologists' diagnosis of these patients was asthma due to positive bronchodilator testing. Therefore, this lack of performance led the GPs to over-estimate COPD and under-estimate asthma in patients with airway obstruction. This is of importance as patients with asthma need to be treated preferably with inhaled steroids. However, our results are better than demonstrated by Miravitlles et al. [39], which might be due to the repeated education of the whole practice team. Nevertheless, these results are not satisfying enough. Further efforts are necessary to improve the performance of spirometry, as this could enhance the diagnostic accuracy. It has already been established that GPs are able to perform and interpret spirometry after educational meetings [40] and that performing spirometry has a positive impact on medical decision making [6,30,41]. It therefore seems reasonable and valuable to implement high quality spirometry in primary care.

Conclusion
COPD can be estimated with high diagnostic accuracy using spirometry. It is also possible to rule in asthma with spirometry. However, asthma can not be ruled out only using spirometry. This diagnostic uncertainty leads to an overestimation of asthma presence. Patients with inconclusive spirometric results should be referred for NOmeasurement and/or bronchial provocation if possible to guarantee accurate diagnosis.