Skip to main content

Development and external validation of the DOAT and DOATS scores: simple decision support tools to identify disease progression among nonelderly patients with mild/moderate COVID-19

Abstract

Background

During the fifth wave of the coronavirus disease 2019 (COVID-19) pandemic in Japan, which took place between June and September 2021, a significant number of COVID-19 cases with deterioration occurred in unvaccinated individuals < 65 years old. However, the risk factors for COVID-19 deterioration in this specific population have not yet been determined. This study developed a prediction method to identify COVID-19 patients < 65 years old who are at a high risk of deterioration.

Methods

This retrospective study analyzed data from 1,675 patients < 65 years old who were admitted to acute care institutions in Fukushima with mild-to-moderate-1 COVID-19 based on the Japanese disease severity criteria prior to the fifth wave. For validation, 324 similar patients were enrolled from 3 hospitals in Yamagata. Logistic regression analyses using cluster-robust variance estimation were used to determine predictors of disease deterioration, followed by creation of risk prediction scores. Disease deterioration was defined as the initiation of medication for COVID-19, oxygen inhalation, or mechanical ventilation starting one day or later after admission.

Results

The patients whose condition deteriorated (8.6%) tended to be older, male, have histories of smoking, and have high body temperatures, low oxygen saturation values, and comorbidities, such as diabetes/obesity and hypertension. Stepwise variable selection using logistic regression to predict COVID-19 deterioration retained comorbidities of diabetes/obesity (DO), age (A), body temperature (T), and oxygen saturation (S). Two predictive scores were created based on the optimism-corrected regression coefficients: the DOATS score, including all of the above risk factors, and the DOAT score, which was the DOATS score without oxygen saturation. In the original cohort, the areas under the receiver operating characteristic curve (AUROCs) of the DOATS and DOAT scores were 0.81 (95% confidence interval [CI] 0.77–0.85) and 0.80 (95% CI 0.76–0.84), respectively. In the validation cohort, the AUROCs for each score were both 0.76 (95% CI 0.69–0.83), and the calibration slopes were both 0.80. A decision curve analysis confirmed the clinical practicability of both scores in the validation cohort.

Conclusions

We established two prediction scores that can quickly evaluate the risk of COVID-19 deterioration in mild/moderate patients < 65 years old.

Peer Review reports

Introduction

Due to the global spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the coronavirus disease 2019 (COVID-19) pandemic remains a serious problem worldwide. Approximately 5% of COVID-19 patients develop respiratory failure, and 2% die despite undergoing intensive treatment [1]. During the height of the pandemic, many patients in Japan could not be admitted to hospitals due to the lack of available beds in urban areas such as Tokyo, Kobe, and Osaka. Therefore, it is important to predict which patients will become severely ill as early as possible after the onset, even in those without respiratory failure. In particular, the administration of antiviral drugs and neutralizing antibodies is considered most effective when performed within five to seven days after the onset [2].

Some observational studies have revealed the risk factors for severe COVID-19. Old age, male sex, a smoking habit, and comorbidities such as diabetes, chronic renal disease, malignancies, and chronic respiratory diseases, such as chronic pulmonary disease (COPD) or interstitial lung diseases, have been listed as risk factors for severe COVID-19 [3,4,5,6,7,8]. In addition, some biomarkers have been reported to be associated with disease severity [4, 8,9,10,11,12]. Using these risk factors, several investigators have reported risk stratification methods [9,10,11, 13,14,15,16].

Beginning in February 2021, vaccines for SARS-CoV-2 began to be disseminated in Japan. By the end of July 2021, 30% of residents in Japan had been vaccinated twice. A majority of people who had been vaccinated twice were elderly individuals (≥ 65 years old) or health-care workers. The widespread dissemination of highly effective vaccines has resulted in a reduction in COVID-19 incidence and a decrease in the severity of cases among vaccinated individuals ≥ 65 years old in Japan [17]. Accordingly, the next target population for health care policy has shifted to individuals < 65 years old who have not yet received the vaccine [17]. It is difficult to accurately forecast the risk for disease deterioration by applying existing prediction rules based on risk factors identified before the dissemination of vaccines to these younger target populations under the current conditions. However, few investigations have reported on the risk factors among the general population < 65 years old who have not been vaccinated for SARS-CoV-2 [18].

The present study therefore investigated the risk factors for COVID-19 deterioration among patients < 65 years old who had not received the vaccine and did not require oxygen supplementation, i.e. those with mild-to-moderate-1 stage of COVID-19 according to the Japanese disease severity criteria [19], and developed and externally validated clinical prediction rules to identify cases at risk of deterioration. In particular, we analyzed the factors associated with disease deterioration, including not only death and the initiation of mechanical ventilation but also the start of medications such as remdesivir or dexamethasone and the start of oxygen inhalation during hospitalization.

Methods

Setting and study population

This multicenter retrospective cohort study used the data of all consecutive patients with COVID-19 admitted to the hospitals that participated in a web conference against COVID-19 held weekly by the Department of Pulmonary Medicine, Fukushima Medical University, between March 31, 2020, and May 20, 2021. Of the 43 COVID-19 hospitals in Fukushima, 26 facilities that engaged in medical care during the acute phase of the disease participated in this conference. Of approximately 4,500 patients with COVID-19 in Fukushima, 3,008 (67%) were enrolled in this study. For the external validation study, 3 hospitals in Yamagata treating COVID-19 patients between March 31, 2020, and May 20, 2021, provided data on 324 mild and moderate-1 patients < 65 years old. A diagnosis of COVID-19 was made when the results of a polymerase chain reaction (PCR) test using a nasopharyngeal swab or saliva were positive.

The subjects’ data, such as clinical characteristics including comorbidities, results of examinations, medical course, medications, and outcomes, were obtained from the medical records of each hospital. COVID-19 disease severity was classified according to the definition of the Japanese Ministry of Health, Labor and Welfare as follows: mild, subjects without pneumonia and respiratory failure; moderate-1, subjects with pneumonia but without respiratory failure; moderate-2, subjects with pneumonia and respiratory failure (oxygen saturation < 94% on room air) that does not require mechanical ventilation or extracorporeal membrane oxygenation (ECMO); severe, subjects with pneumonia and respiratory failure that required mechanical ventilation and ECMO [19, 20]. In this article, the term “moderate” refers to moderate-1 of the Japanese COVID-19 severity classification, unless otherwise specified.

Among the 3,008 enrolled patients, 1,675 from 21 facilities had mild and moderate-1 disease and were < 65 years old. These were the eligible subjects in this study (Fig. 1, Supplementary Table 1).

Fig. 1
figure 1

Flowchart of patient recruitment in this study. Among all registered inpatients with COVID-19 in Fukushima, 1,675 were selected for the present study. Disease severity was classified into mild, moderate-1, moderate-2, and severe in accordance with the definition of the Japanese Ministry of Health, Labor and Welfare, as described in the Methods section of this manuscript

Outcomes

The outcomes were (1) any disease deterioration, defined as the initiation of medication for COVID-19 (dexamethasone, methylprednisolone, tocilizumab, baricitinib, or remdesivir) or respiratory therapy (use of inhalation oxygen); or (2) use of a ventilator or the need for ECMO after the first day of hospitalization.

Candidate predictors

The age, sex, body temperature on admission (< 37.0, 37.0-37.9, ≥ 38.0 °C), history of smoking, comorbidities (prevalence ≥ 2%), pregnancy, and blood oxygen saturation (%) measured by a pulse oximeter on admission were evaluated as candidate risk factors for deterioration of the disease [21,22,23]. The collection of information was left to the doctors at each participating institution.

Statistical analyses

Continuous variables are presented as medians with interquartile ranges, and categorical variables are presented as the number of patients with percentages. Patients whose conditions were exacerbated a day or later after admission were assigned to the Deteriorated group, while all others were assigned to the Stable group. Between-group comparisons were performed using the Mann‒Whitney U test for continuous variables and the chi-square test for categorical variables.

The variables with statistically significant differences between the two groups were used to identify possible risk factors for predicting the deterioration of COVID-19 a day or later after admission to the hospital by forward stepwise multivariate logistic regression analyses. In these analyses, cluster-robust variance estimation was used to consider within-hospital correlation [24]. Because information on oxygen saturation may not be available in some clinical sites, such as during patients’ temporary accommodations or at home, we analyzed two models: one including and one excluding oxygen saturation values. Among the comorbidities listed in the database from the participating hospitals in Fukushima, those with a prevalence of ≥ 2% were applied to the analyses. Diabetes and obesity were combined in the same category for the following reasons: first, obesity is an important risk factor for the onset and progression of diabetes, and second, because they both have common clinical features and immune responses to SARS-CoV-2 based on a background of chronic systemic low-grade inflammation, abnormal production of proinflammatory cytokines, and an impaired immune response and host defense [25], both are reported as high-risk factors for COVID-19 mortality [26]. For continuous variables, namely age and oxygen saturation, a restricted cubic spline curve with the logistic regression analysis confirmed the linear relationship with the risk of COVID-19 deterioration (Supplementary Fig. 1a and 1b).

We created the score-based prediction model with an emphasis on simplicity, prioritizing clinical usability with minimal burden on physicians [27]. First, age and oxygen saturation as continuous variables were incorporated into the model for the original cohort using logistic regression with clustering per hospital (original model). Next, a 200-cycle bootstrap process was performed to assess the optimism of the original model, and optimism-corrected regression coefficients for each variable were calculated (bootstrap shrinkage) [28]. The continuous variables were then categorized into 4 or 5 categories (i.e. 0–17, 18–30, 31–40, 41–50, and 51–64 for age [years old]; and 94%, 95%, 96-97%, and 98-100% for oxygen saturation), the midpoints for each category were determined, and “regression units” were calculated using the optimism-corrected regression coefficients, according to a previous study [27, 29]. Finally, points were assigned to each variable category, calculated by dividing the “regression units” by the lowest coefficient and rounding to the nearest integer.

We investigated the predictive accuracy of the models by testing discrimination and calibration in both the original and validation cohorts. Discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUROC). We also presented predictive performance (sensitivity, specificity, and positive and negative likelihood ratios) stratified for each threshold of the scores. A calibration curve was generated to evaluate the agreement between the observed and predicted outcomes with respect to calibration ability. Furthermore, we evaluated the calibration slope and intercept (calibration-in-the-large) in the validation cohort. Finally, we used a decision curve analysis (DCA) to assess the clinical utility of the models [39].

A two-tailed p value of < 0.05 was considered statistically significant. Sample size calculations were not performed because we used all available data in the registry to maximize the power of the results. Cases with missing data were excluded from the analyses, and the details of the missing data are described in the note of Table 1. All statistical analyses were performed using the software programs JMP 13 (SAS Institute Inc, Cary, NC, USA), STATA 16 (Stata Corp LP, College Station, TX, USA), and SPSS27 (IBM Corp., Armonk, NY, USA).

Table 1 Differences in the characteristics between the Stable and Deteriorated groups in the original cohort of the study and external validation cohort

Results

Table 1 shows the characteristics of the mild/moderate COVID-19 patients < 65 years old in the current study. One hundred and forty-four patients (8.6%) experienced deterioration of their disease 1 day after admission or later. These patients were older and more likely to be male with histories of cigarette smoking, a higher body temperature, lower oxygen saturation, and higher rates of comorbidities, such as diabetes/obesity and hypertension, than the Stable group. The rates of pregnancy, as well as comorbidities, such as chronic respiratory diseases, malignancies, dyslipidemia, and cardiac diseases, were not significantly different between the Deteriorated and Stable groups.

Stepwise variable selection using a logistic regression analysis with clustering per hospital to predict COVID-19 deterioration during hospitalization in the original cohort retained comorbidities of diabetes/obesity (DO), age (A), body temperature (T), and oxygen saturation (S). Table 2 shows the results of the multivariable logistic regression analysis including all of the above risk factors (DOATS) and all but oxygen saturation (DOAT) before and after optimism correction with 200 cycles of bootstrapping. Of the original models that included continuous variables as they were, the optimism-corrected AUROCs for the DOATS-based and DOAT-based models predicting deterioration in COVID-19 patients during hospitalization were 0.82 (95% confidence interval [CI] 0.77–0.85) and 0.81 (95% CI 0.77–0.86), respectively (Fig. 2a and b). The calibration curve showed good agreement between the predicted probability of COVID-19 deterioration by the original models and the observed COVID-19 deterioration (Fig. 3a and b).

Table 2 Results of a multivariable logistic regression analysis with all risk factors (DOATS) and with all but oxygen saturation (DOAT) before and after optimism correction with 200 cycles of bootstrapping in the original cohort
Fig. 2
figure 2

Receiver operating characteristic (ROC) curves analyzing the discrimination of (a) the original model with DOATS, (b) the original model with DOAT, (c) the DOATS score, and (d) the DOAT score for COVID-19 deterioration in the original cohort. Of the original models that included continuous variables as they were, the optimism-corrected area under the ROC curves (AUROCs) for (a) the DOATS-based and (b) DOAT-based models predicting the deterioration of COVID-19 patients during hospitalization were 0.82 (95% confidence interval [CI] 0.77–0.85) and 0.81 (95% CI 0.77–0.86), respectively. For (c) the DOATS and (d) DOAT scores, the AUROCs were 0.81 (95% CI 0.77–0.85) and 0.80 (95% CI 0.76–0.84), respectively

Fig. 3
figure 3

Calibration curves of (a) the original model with DOATS, (b) the original model with DOAT, (c) DOATS score and (d) DOAT score for COVID-19 deterioration in the original cohort. The calibration curve analysis showed that the calibration slope and calibration-in-the-large were 0.93 and 0.134 for the original DOATS model (a), 0.93 and 0.124 for the original DOAT model (b), 0.97 and 0.002 for the DOATS score (c), and 0.96 and 0.055 for the DOAT score (d), respectively. The calibration curves of the original models (a, b) and the scores (c, d) demonstrated good agreement between the predicted probability of COVID-19 deterioration and the observed COVID-19 deterioration

The DOATS and DOAT scores constructed by categorizing age and oxygen saturation and assigning score points to each category of final variables (DOATS and DOAT) based on optimism-corrected regression coefficients of the original models are listed in Table 3. Both scores were calculated for each patient by adding the points for risk factors present: the DOATS score ranges from − 2 to 19 points, while the DOAT score ranges from − 2 to 12 points. Table 4 shows the predicted probabilities of COVID-19 deterioration according to the scores along with the actual percentages of patients who had deterioration or required mechanical ventilation and/or ECMO treatment during hospitalization. Overall, the higher the points, the more deteriorated and the greater the need for mechanical ventilation/ECMO.

Table 3 (a) DOATS and (b) DOAT scores for COVID-19 deterioration based on optimism-corrected regression coefficients of the original models
Table 4 Predicted and actual risk of COVID-19 deterioration, as well as the actual risk of using an MV and ECMO in the original cohort according to each (a) DOATS score and (b) DOAT score

The prognostic performance of the DOATS and DOAT scores at each threshold is shown in Supplementary Table 2a and 2b. The AUROCs for the DOATS and DOAT scores predicting deterioration in COVID-19 patients during hospitalization were 0.81 (95% CI 0.77–0.85) and 0.80 (95% CI 0.76–0.84), respectively, resulting in only a small reduction from the AUROCs of the original models (Fig. 2c and d). Both of the predicted risks using these scores showed a generally good fit with the observed COVID-19 deterioration, as evidenced by the calibration curves (Fig. 3c and d). A DCA confirmed the clinical practicability of both scores as well as the original models (Fig. 4).

Fig. 4
figure 4

Decision curve analyses for the original model with DOATS, the original model with DOAT, and the DOATS and DOAT scores in the original cohort. The graph illustrates the net benefit relative to no treatment in any patient (‘Treat none’) using different treatment approaches. The gray line represents the scenario where no patients are treated, resulting in a net benefit of zero (no true-positive and no false-positive classifications). The black line represents the scenario where all patients are treated. The colored lines correspond to different treatment thresholds based on the predictions of the DOATS-based original model (blue line), DOAT-based original model (green line), DOATS score (red line), and DOAT score (orange line) for the risk of deterioration. This graph demonstrates the expected net benefit when treatment decisions are made based on these different approaches. The analysis confirms the clinical practicability and utility of both scores, as well as the original models

The ability of these models was validated in 324 inpatients from 3 hospitals in Yamagata, Japan (Table 1). Forty-one of these patients (12.7%) experienced deterioration of their disease during hospitalization. In the validation cohort, the AUROCs of the DOATS and DOAT scores for COVID-19 deterioration were both 0.76 (95% CI 0.69–0.83) (Fig. 5a and c). Table 5 shows the percentage of patients who experienced exacerbation during hospitalization according to the scores, along with the predicted risk; although the risk was overestimated in some patients with high scores of ≥ 15 on the DOATS or ≥ 11 on the DOAT, overall, the prevalence of a worsening outcome was higher in the group with higher scores than in those with lower scores. The calibration curve analysis showed that the calibration slope and calibration-in-the-large were 0.80 and 0.002 for the DOATS score and 0.80 and 0.04 for the DOAT score, respectively (Fig. 5b and d). A DCA confirmed the clinical practicability of both scores in the validation cohort (Fig. 5e).

Fig. 5
figure 5

Discrimination (a) and calibration (b) of the DOATS score, discrimination (c) and calibration (d) of the DOAT score, and decision curve analyses for both scores (e) in the validation cohort. In the validation cohort, the area under the ROC curves (AUROCs) for (a) the DOATS score and (c) the DOAT score predicting the deterioration of COVID-19 patients during hospitalization were both 0.76 (95% CI 0.69–0.83). The calibration curve analysis revealed that the calibration slope and calibration-in-the-large were 0.80 and 0.002 for the DOATS score (b) and 0.80 and 0.04 for the DOAT score (d), respectively. (e) The graph illustrates the net benefit relative to no treatment in any patient (‘Treat none’) using different treatment approaches. The gray line represents the scenario where no patients are treated, resulting in a net benefit of zero (no true-positive and no false-positive classifications). The black line represents the scenario where all patients are treated. The colored lines correspond to different treatment thresholds based on the predictions of the DOATS score (blue line) and the DOAT score (red line) for the risk of deterioration. This graph demonstrates the expected net benefit when treatment decisions are made based on these different approaches. The decision curve analysis confirmed the clinical practicability of both scores in the validation cohort

Table 5 Predicted and actual risk of COVID-19 deterioration in the validation cohort according to each (a) DOATS score and (b) DOAT score

Discussion

We examined risk factors associated with deterioration of the COVID-19 severity in 1,675 Japanese patients < 65 years old without respiratory failure. Using the identified risk factors, we further established two predictive models; one consisting of four items (comorbidity of diabetes or obesity [DO], age [A], body temperature [T], and oxygen saturation [S], the DOATS score); and another consisting of three items (diabetes or obesity [DO], age [A], and temperature [T], the DOAT score). In cases where a doctor visited patients’ homes, information about oxygen saturation was sometimes unavailable. Therefore, we felt that prediction using three items without this variable, i.e. the DOAT score, would be easier to perform than that using the DOATS score, which consists of four items.

The predictive abilities of these scores were not significantly different in the original or validation cohorts. Because neither the DOATS nor DOAT score requires laboratory testing, physicians, nurses, and public health nurses working in hospitals, clinics, and health centers can use them both to promptly assess a patient’s deterioration risk. It should be noted that the prediction scores are intended to be calculated using data obtained only on the initial visit or first day of hospitalization; the temperature and oxygen saturation after the first encounter were not used in the calculations.

The present study included a relatively large number of subjects from Fukushima Prefecture. COVID-19 is a designated “category 2 infectious disease” in Japan, and in Fukushima Prefecture, inpatient treatment was performed for even mild cases to isolate patients from uninfected persons. Many COVID-19 studies have been conducted primarily in an inpatient setting, skewing toward severe disease. In our database, not only severely ill patients but also mild and moderate cases without respiratory failure were enrolled. As a result, over 1,500 patients who did not have respiratory failure were included in the analysis.

In the present study, clinical deterioration in each patient was identified based on data regarding starting medication against COVID-19, oxygen inhalation including nasal high flow cannula, and mechanical ventilation. This information enabled us to understand the risk factors that predict COVID-19 deterioration among patients in the early phase of the disease. Most previous studies have analyzed either risk factors that predict death or mechanical ventilation used mainly in critically ill COVID-19 patients [3,4,5,6,7,8]. However, whether or not the predictive factors identified in those studies can be used to accurately identify patients with mild cases who may experience deterioration is unclear, as the statistical analyses of those studies focused only on severe outcomes, such as death or use of mechanical ventilation. Even if disease progression does not lead to such severe outcomes, progression to respiratory failure often has a significant effect on the patient. Some patients with respiratory failure may require hospitalization for a long period of time and may require long-term oxygen therapy due to the reduction in their pulmonary function. Therefore, preventing progression to respiratory failure is considered a major medical goal. To this end, it is necessary to identify patients with mild to moderate disease who are at high risk of exacerbation and provide treatment such as neutralizing antibodies or antiviral agents to those individuals as early as possible [30, 31].

Several studies have demonstrated the association between certain clinical characteristics, such as an older age, male sex, symptoms of COVID-19 (a fever, cough, fatigue, and shortness of breath), and comorbidities (hypertension, diabetes, and CKD) and the disease severity or activity [21,22,23]. The addition of blood tests may also increase the accuracy of prediction of disease progression [11]. However, involving blood sampling would make it difficult for medical staff to immediately evaluate patients’ risk while in their presence. In the present study, we demonstrated two simple methods for predicting the development of illness among mild/moderate patients with COVID-19 that do not require blood sampling: DOAT and DOATS scores. Tu et al. developed a model predicting the progression of COVID-19 consisting of nine items: sex, age, and the presence of a fever, hypertension, cardio-cerebrovascular disease, dyspnea, cough, and myalgia [14]. Their model also consisted of clinical characteristics only, with no laboratory data involved. While their model had a similar predictive ability for disease progression to ours in terms of discrimination (AUROC 0.79), ours were simpler and involved fewer variables. During the pandemic, where many patients are visiting clinics, simple and quick methods of identifying patients at risk of disease deterioration are desirable.

At present, many elderly people in Japan have been vaccinated against SARS-CoV-2 [32, 33]. Delayed vaccination for middle-aged and younger people due to a vaccine shortage resulted in a shift in the epidemic statistics from elderly individuals to middle-aged/young people during the fifth pandemic wave. We suspect that this change indicates the presence of different risk factors for COVID-19 deterioration among different groups. Previously reported risk factors may therefore not be effective for the populations who are currently at highest risk because comorbidities, such as cancer and chronic respiratory or renal diseases, are relatively uncommon in young and middle-aged people. In our cohort of patients < 65 years old without respiratory failure, the reported risk factors, such as male sex, cigarette smoking, hypertension, chronic respiratory disease, malignancies, dyslipidemia, and cardiac diseases, were not included as risk factors in the final prediction models for disease deterioration. Therefore, the risk factors for deterioration of COVID-19 may need to be reconsidered and repeatedly updated in accordance with the shift of generations who are most severely suffering from COVID-19.

The strength of the current study is that the results are considered to be generalizable because this study included inpatients from a wide range of hospitals in Fukushima that handle COVID-19 inpatient treatment. In our database, three-fourths of COVID-19 patents in Fukushima were enrolled. In addition, the robustness of the predictive ability of both scores in terms of discrimination, calibration, and clinical practicability is supported by the results from the external validation cohort analyses.

However, several limitations associated with the present study also warrant mention. First, data regarding the exact proportion of vaccinated individuals were not available. COVID-19 vaccinations only began to be administered in Japan in March 2021, such that by the end of May 2021, no vaccinations had yet been administered to people < 65 years old [32, 33]. Therefore, we believe that most of the population in this study had not been vaccinated. Second, precise data regarding the day of the onset and deterioration or treatment before deterioration were not available for any patients. There may be some differences in treatment before deterioration between the Deteriorated and Stable groups. For example, treatments such as inhaled corticosteroid and favipiravir may have affected the clinical course of the patients [34,35,36]. In our database, information about the timing of the prescription of these medicines was not available. Third, some patients with high DOATS or DOAT scores may be overestimated with regard to their predicted risk compared to actual risk, as presented by calibration in external validation, and thus may be overtreated. However, in the face of a potentially life-threatening disease such as severe COVID-19, we believe that missed treatment opportunities due to underestimation of risk are much less desirable than overtreatment. Finally, whether or not the risk factors identified in the present study will still be applicable to risk stratification against recent COVID-19 patients is unclear, as the current situation is markedly different from that at the time of the analyses. The present study was conducted before the dissemination of the SARS-CoV-2 vaccine to nonelderly individuals and before the surge of the omicron variant. The current vaccines are reportedly less effective against omicron variants than against earlier variants [37]. Therefore, even vaccinated individuals may still become infected [38]. However, in situations where many individuals are vaccinated, it is possible that new risk factors for deterioration of COVID-19 will become apparent. Because many people in Fukushima had been vaccinated by the end of the fifth wave of the pandemic, including the younger generation, the ability of both the DOATS and DOAT scores to predict deterioration may be weakened.

In future studies, we need to validate the predictive ability of both scores using the new version of the dataset, which consists of data from many vaccinated patients infected with the delta or omicron variants of coronavirus.

Conclusion

We found that the comorbidities of diabetes or obesity, an older age, higher body temperature, and lower oxygen saturation were risk factors that were linked to disease progression in COVID-19 patients < 65 years old who did not have respiratory failure on admission. We also established two simple prediction models that can quickly and easily evaluate the risk of the patients using the sum of the points that were given according to the presence of these risk factors. We believe that these scoring methods can be used broadly in many clinics and can effectively identify high-risk patients among COVID-19 patients < 65 years old at an early phase, resulting in a reduction of disease progression by enabling treatment to be administered as soon as possible.

Data availability

The data are available from the authors upon reasonable request and with permission from the corresponding authors.

Abbreviations

CI:

Confidence interval

COVID-19:

Coronavirus disease 2019

DOAT:

Comorbidity of diabetes/obesity age and body temperature

DOATS:

Comorbidity of diabetes/obesity age body temperature and oxygen saturation

ECMO:

Extracorporeal membrane oxygenation

OR:

Odds ratio

SARS-CoV-2:

Severe acute respiratory syndrome coronavirus 2

References

  1. Wu Z, McGoogan JM. Characteristics of and important Lessons from the Coronavirus Disease 2019 (COVID-19) outbreak in China: Summary of a report of 72314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323(13):1239–42.

    Article  CAS  PubMed  Google Scholar 

  2. Wourld Health Organization.: Therapeutics and COVID-19: living guideline. [https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2022.4].

  3. Docherty AB, Harrison EM, Green CA, Hardwick HE, Pius R, Norman L, Holden KA, Read JM, Dondelinger F, Carson G, et al. Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ. 2020;369:m1985.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Hu J, Wang Y. The clinical characteristics and risk factors of severe COVID-19. Gerontology. 2021;67(3):255–66.

    Article  CAS  PubMed  Google Scholar 

  5. Kragholm K, Andersen MP, Gerds TA, Butt JH, Ostergaard L, Polcwiartek C, Phelps M, Andersson C, Gislason GH, Torp-Pedersen C, et al. Association between male sex and outcomes of Coronavirus Disease 2019 (Covid-19) - a danish nationwide, register-based study. Clin Infect Dis. 2020;73(11):e4025–30.

    Article  Google Scholar 

  6. Lowe KE, Zein J, Hatipoglu U, Attaway A. Association of Smoking and Cumulative Pack-Year exposure with COVID-19 outcomes in the Cleveland Clinic COVID-19 Registry. JAMA Intern Med. 2021;181(5):709–11.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Peckham H, de Gruijter NM, Raine C, Radziszewska A, Ciurtin C, Wedderburn LR, Rosser EC, Webb K, Deakin CT. Male sex identified by global COVID-19 meta-analysis as a risk factor for death and ITU admission. Nat Commun. 2020;11(1):6317.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Zhao Z, Chen A, Hou W, Graham JM, Li H, Richman PS, Thode HC, Singer AJ, Duong TQ. Prediction model and risk scores of ICU admission and mortality in COVID-19. PLoS ONE. 2020;15(7):e0236618.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Huang J, Cheng A, Lin S, Zhu Y, Chen G. Individualized prediction nomograms for disease progression in mild COVID-19. J Med Virol. 2020;92(10):2074–80.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Ji D, Zhang D, Xu J, Chen Z, Yang T, Zhao P, Chen G, Cheng G, Wang Y, Bi J, et al. Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score. Clin Infect Dis. 2020;71(6):1393–9.

    Article  CAS  PubMed  Google Scholar 

  11. Liu M, Jiang H, Li Y, Li C, Tan Z, Jin F, Zhang T, Nan Y. Independent risk factors for the Dynamic Development of COVID-19: a retrospective study. Int J Gen Med. 2021;14:4349–67.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Sugiyama M, Kinoshita N, Ide S, Nomoto H, Nakamoto T, Saito S, Ishikane M, Kutsuna S, Hayakawa K, Hashimoto M, et al. Serum CCL17 level becomes a predictive marker to distinguish between mild/moderate and severe/critical disease in patients with COVID-19. Gene. 2021;766:145145.

    Article  CAS  PubMed  Google Scholar 

  13. Muto Y, Awano N, Inomata M, Kuse N, Tone M, Takada K, Fujimoto K, Ueda A, Hayashi M, Izumo T. Predictive model for the development of critical coronavirus disease 2019 and its risk factors among patients in Japan. Respir Investig. 2021;59(6):804–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Tu Y, Zhou X, Shao L, Zheng J, Wang J, Wang Y, Tong W, Wang M, Wu J, Zhu J, et al. Predicting Progression of COVID-19 infection to Prioritize Medical Resource Allocation: a Novel Triage Model based on patient characteristics and symptoms at Presentation. Front Public Health. 2021;9:610280.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Wang L, He W, Yu X, Hu D, Bao M, Liu H, Zhou J, Jiang H. Coronavirus disease 2019 in elderly patients: characteristics and prognostic factors based on 4-week follow-up. J Infect. 2020;80(6):639–45.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Zheng Y, Zhang Y, Chi H, Chen S, Peng M, Luo L, Chen L, Li J, Shen B, Wang D. The hemocyte counts as a potential biomarker for predicting disease progression in COVID-19: a retrospective study. Clin Chem Lab Med. 2020;58(7):1106–15.

    Article  CAS  PubMed  Google Scholar 

  17. Lee H, Chubachi S, Namkoong H, Asakura T, Tanaka H, Otake S, Nakagawara K, Morita A, Fukushima T, Watase M, et al. Characteristics of hospitalized patients with COVID-19 during the first to fifth waves of infection: a report from the Japan COVID-19 Task Force. BMC Infect Dis. 2022;22(1):935.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Lopes StSJ, Rocca G, Prasad W, Majerus K, Limper M, Jacobson A, Fan D, Jacobson C, Rutten R et al. L: Factors Associated With Severe COVID-19 Infection Among Persons of Different Ages Living in a Defined Midwestern US Population. Mayo Clinic Proceedings 2021, 96(10):2528–2539.

  19. National Institute of Public Health.: Clinical Management of patients with COVID-19: A guide for front-line healthcare workers [https://www.niph.go.jp/h-crisis/wp-content/uploads/2020/07/20200706103735_content_000646531.pdf].

  20. Yamamoto A, Wada H, Ichikawa Y, Mizuno H, Tomida M, Masuda J, Makino K, Kodama S, Yoshida M, Fukui S, et al. Evaluation of biomarkers of severity in patients with COVID-19 infection. J Clin Med. 2021;10(17):3775.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Geng MJ, Wang LP, Ren X, Yu JX, Chang ZR, Zheng CJ, An ZJ, Li Y, Yang XK, Zhao HT, et al. Risk factors for developing severe COVID-19 in China: an analysis of disease surveillance data. Infect Dis Poverty. 2021;10(1):48.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Lee HW, Yoon SY, Lee JK, Park TY, Kim DK, Chung HS, Heo EY. Clinical implication and risk factor of pneumonia development in mild coronavirus disease 2019 patients. Korean J Intern Med. 2021;36(1):1–10.

    Article  CAS  PubMed  Google Scholar 

  23. Park HY, Lee JH, Lim NK, Lim DS, Hong SO, Park MJ, Lee SY, Kim G, Park JK, Song DS, et al. Presenting characteristics and clinical outcome of patients with COVID-19 in South Korea: a nationwide retrospective observational study. Lancet Reg Health West Pac. 2020;5:100061.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bartoletti M, Giannella M, Scudeller L, Tedeschi S, Rinaldi M, Bussini L, Fornaro G, Pascale R, Pancaldi L, Pasquini Z, et al. Development and validation of a prediction model for severe respiratory failure in hospitalized patients with SARS-CoV-2 infection: a multicentre cohort study (PREDI-CO study). Clin Microbiol infection: official publication Eur Soc Clin Microbiol Infect Dis. 2020;26(11):1545–53.

    Article  CAS  Google Scholar 

  25. Perez-Galarza J, Procel C, Canadas C, Aguirre D, Pibaque R, Bedon R, Sempertegui F, Drexhage H, Baldeon L. Immune Response to SARS-CoV-2 infection in obesity and T2D: literature review. Vaccines (Basel). 2021;9(2):102.

    Article  CAS  PubMed  Google Scholar 

  26. Bello-Chavolla OY, Bahena-Lopez JP, Antonio-Villa NE, Vargas-Vazquez A, Gonzalez-Diaz A, Marquez-Salinas A, Fermin-Martinez CA, Naveja JJ, Aguilar-Salinas CA. Predicting Mortality due to SARS-CoV-2: a mechanistic score relating obesity and diabetes to COVID-19 Outcomes in Mexico. J Clin Endocrinol Metab 2020, 105(8).

  27. Bonnett LJ, Snell KIE, Collins GS, Riley RD. Guide to presenting clinical prediction models for use in clinical settings. BMJ. 2019;365:l737.

    Article  PubMed  Google Scholar 

  28. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–81.

    Article  CAS  PubMed  Google Scholar 

  29. Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–60.

    Article  PubMed  Google Scholar 

  30. Beigel JH, Tomashek KM, Dodd LE, Mehta AK, Zingman BS, Kalil AC, Hohmann E, Chu HY, Luetkemeyer A, Kline S, et al. Remdesivir for the treatment of Covid-19 - final report. N Engl J Med. 2020;383(19):1813–26.

    Article  CAS  PubMed  Google Scholar 

  31. Ganesh R, Philpot LM, Bierle DM, Anderson RJ, Arndt LL, Arndt RF, Culbertson TL, Destro Borgen MJ, Hanson SN, Kennedy BD, et al. Real-world clinical outcomes of Bamlanivimab and Casirivimab-Imdevimab among high-risk patients with mild to Moderate Coronavirus Disease 2019. J Infect Dis. 2021;224(8):1278–86.

    Article  CAS  PubMed  Google Scholar 

  32. Ministry of Health, Labour and Welfare of Japan.: Number of new SARS-CoV-2 PCR positive subjects and the percentage of vaccinated elderlies in Japan (in Japanese) [https://www.mhlw.go.jp/content/10900000/000840238.pdf].

  33. Nippon TV. COVID-19 presented in data and graphs: Vaccinated situation in elderlies (in Japanese) [https://www.news24.jp/archives/corona_vaccine/index3.html].

  34. Shinkai M, Tsushima K, Tanaka S, Hagiwara E, Tarumoto N, Kawada I, Hirai Y, Fujiwara S, Komase Y, Saraya T, et al. Efficacy and safety of Favipiravir in Moderate COVID-19 Pneumonia Patients without Oxygen Therapy: a Randomized, Phase III Clinical Trial. Infect Dis Ther. 2021;10(4):2489–509.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Song JY, Yoon JG, Seo YB, Lee J, Eom JS, Lee JS, Choi WS, Lee EY, Choi YA, Hyun HJ et al. Ciclesonide Inhaler treatment for mild-to-moderate COVID-19: a randomized, Open-Label, phase 2 trial. J Clin Med 2021, 10(16).

  36. Yu LM, Bafadhel M, Dorward J, Hayward G, Saville BR, Gbinigie O, Van Hecke O, Ogburn E, Evans PH, Thomas NPB, et al. Inhaled budesonide for COVID-19 in people at high risk of complications in the community in the UK (PRINCIPLE): a randomised, controlled, open-label, adaptive platform trial. Lancet. 2021;398(10303):843–55.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Lauring AS, Tenforde MW, Chappell JD, Gaglani M, Ginde AA, McNeal T, Ghamande S, Douin DJ, Talbot HK, Casey JD, et al. Clinical severity of, and effectiveness of mRNA vaccines against, covid-19 from omicron, delta, and alpha SARS-CoV-2 variants in the United States: prospective observational study. BMJ. 2022;376:e069761.

    Article  PubMed  Google Scholar 

  38. Flahault A, Touchard J, Pere H, Ulrich L, Sabatier B, Veyer D, Lebeaux D, Thervet E. Breakthrough omicron COVID-19 infections in patients receiving the REGEN-Cov antibody combination. Kidney Int. 2022;101(4):824–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. 2021;21:1643–8.

Download references

Acknowledgements

We thank the Scientific English Editing Section of Fukushima Medical University and Dr. Brian Quinn (Japan Medical Communication; www.japan-mc.co.jp) for their fruitful discussion and linguistic assistance in proofreading the manuscript.

Funding

This study was funded by the Japanese Society for the Promotion of Science (JSPS) KAKENHI grant number 19K08658 and 22K10560. The funder did not play any role in this study.

Author information

Authors and Affiliations

Authors

Contributions

YSh, KO, HMi, YSu, TNi, and YTa had access to all data and contributed to all components of the study and manuscript generation. All other authors meet at least the minimum three criteria (data acquisition, data interpretation, and manuscript drafting) for BMC Pulmonary Medicine authorship. All authors read and approved the final manuscript.

Ethics declarations

Ethics approval and consent to participate

The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Fukushima Medical University (approval number 2020 − 118, approved on August 3, 2020, updated September 01, 2021), which waived the need for written informed consent because of the retrospective nature of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that no competing interest exists.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Figure S1

Restricted cubic spline curves for log odds ratios of COVID-19 deterioration versus a) age and b) oxygen saturation in the original cohort. Table S1 Number of enrolled cases and deteriorated cases at each facility in the original and validation cohort. Table S2 Sensitivity, specificity, and positive and negative likelihood ratios in the original cohort stratified for each threshold of a) DOATS and b) DOAT scores.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shibata, Y., Omae, K., Minemura, H. et al. Development and external validation of the DOAT and DOATS scores: simple decision support tools to identify disease progression among nonelderly patients with mild/moderate COVID-19. BMC Pulm Med 23, 312 (2023). https://doi.org/10.1186/s12890-023-02604-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12890-023-02604-3

Keywords