Hospital Medicine

Febrile infants 60 days of age or younger pose a significant diagnostic challenge for clinicians. Most of these infants are well appearing and do not have localizing signs or symptoms of infection, yet they may have serious bacterial infections (SBI) such as urinary tract infection (UTI), bacteremia, and meningitis. While urinalysis is highly sensitive for predicting UTI,¹ older clinical decision rules and biomarkers such as white blood cell (WBC) count, absolute neutrophil count (ANC), and C-reactive protein (CRP) lack both appropriate sensitivity and specificity for identifying bacteremia and meningitis (ie, invasive bacterial infection [IBI]),^2,3 which affect approximately 2.4% and 0.9% of febrile infants during the first 2 months of life, respectively.⁴ The lack of accurate diagnostic markers can drive overuse of laboratory testing, antibiotics, and hospitalization despite the low rates of these infections. As a result, procalcitonin (PCT) has generated interest because of its potential to serve as a more accurate biomarker for bacterial infections. This review summarizes recent literature on the diagnostic utility of PCT in the identification of IBI in febrile young infants 60 days or younger.

Procalcitonin is undetectable in noninflammatory states but can be detected in the blood within 4 to 6 hours after initial bacterial infection.⁵ Its production is stimulated throughout various tissues of the body by cytokines such as interleukin-6 and tumor necrosis factor, which are produced in response to bacterial infections. Interferon-γ, which is produced in response to viral infections, attenuates PCT production. While these characteristics suggest promise for PCT as a more specific screening test for underlying bacterial infection, there are caveats. PCT levels are physiologically elevated in the first 48 hours of life and vary with gestational age, factors that should be considered when interpreting results.⁶ Additionally, PCT levels can rise in other inflammatory states such as autoimmune conditions and certain malignancies,⁵ though these states are unlikely to confound the evaluation of febrile young infants.

Because of PCT’s potential to be more specific than other commonly used biomarkers, multiple studies have evaluated its performance characteristics in febrile young infants. Gomez et al retrospectively evaluated 1,112 well-appearing infants younger than 3 months with fever without a source in seven European emergency departments (EDs).⁷ Overall, 23 infants (2.1%) had IBI (1 with meningitis). A PCT level of 0.5 ng/mL or greater was the only independent risk factor for IBI (adjusted odds ratio, 21.69; 95% CI, 7.93-59.28). Four infants with IBI had a PCT level less than 0.5 ng/mL, and none of these four had meningitis. PCT was superior to CRP, ANC, and WBC in detecting IBI (area under the curve [AUC], 0.825; 95% CI, 0.698-0.952). PCT was the also the best marker for identifying IBI among 451 infants with a normal urine dipstick and fever detected ≤6 hours before presentation (AUC, 0.819; 95% CI, 0.551-1.087).

In the largest prospective study to date evaluating the diagnostic accuracy of PCT in febrile young infants, Milcent et al studied 2,047 previously healthy infants aged 7-91 days admitted for fever from 15 French EDs.⁸ In total, 21 (1%) had an IBI (8 with meningitis). PCT performed better than CRP, ANC, and WBC for the detection of IBI with an AUC of 0.91 (95% CI, 0.83-0.99). In a multivariable model, a PCT level of 0.3 ng/mL or greater was the only independent risk factor for IBI with an adjusted odds ratio of 40.3 (95% CI, 5.0-332). Only one infant with IBI had a PCT level less than 0.3 ng/mL. This infant was 83 days old, had 4 hours of fever, and became afebrile spontaneously prior to the blood culture revealing Streptococcus pneumoniae. PCT also performed better than CRP in the detection of IBI in infants 7-30 days of age and those with fever for less than 6 hours, though both subgroups had small numbers of infants with IBI. The authors determined that a PCT level of 0.3 ng/mL was the optimal cutoff for ruling out IBI; this cutoff had a sensitivity of 90% and negative likelihood ratio (LR) of 0.1 (Table). In contrast, the more commonly studied PCT cutoff of 0.5 ng/mL increased the negative LR to 0.2. The authors suggested that PCT, when used in the context of history, exam, and tests such as urinalysis, could identify infants at low risk of IBI.

Gomez et al conducted a prospective, single-center study of well-appearing infants with fever without a source and negative urine dipsticks.⁹ They identified IBI in 9 of 196 infants (4.5%) 21 days or younger and 13 of 1,331 infants (1.0%) 22-90 days old. PCT was superior to CRP and ANC for IBI detection in both age groups. However, in infants 21 days or younger, both the positive and negative LRs for PCT levels of 0.5 ng/mL or greater were poor (Table). Differences in results from the prior two studies^7,8 may be related to smaller sample size and differences in patient population because this study included infants younger than 7 days and a higher proportion of infants presenting within 6 hours of fever.

PCT has also been incorporated into clinical decision rules for febrile young infants, primarily to identify those at low risk of either IBI or SBI. The Step-by-Step approach¹⁰ classified well-appearing febrile infants 90 days or younger as having a high risk of IBI if they were ill appearing, younger than 21 days old, had a positive urine dipstick or a PCT level of 0.5 ng/mL or greater, and classified them as intermediate risk if they had a CRP level greater than 20 mg/L or ANC level greater than 10,000/µL. The remaining infants were classified as low risk and could be managed as outpatients without lumbar puncture or empiric antibiotics. Of note, derivation of this rule excluded patients with respiratory signs or symptoms. In a prospective validation study with 2,185 infants from 11 European EDs, 87 (4.0%) had an IBI (10 with bacterial meningitis). Sequentially identifying patients as high risk using general appearance, age, and urine dipstick alone identified 80% of infants with IBI and 90% of those with bacterial meningitis. The remaining case of meningitis would have been detected by an elevated PCT. A total of 7 of 991 infants (0.7%) classified as low risk had an IBI and none had meningitis. Six of these infants had a fever duration of less than 2 hours, which would not be enough time for PCT to rise. The Step-by-Step approach, with a sensitivity of 92% and negative LR of 0.17, performed well in the ability to rule out IBI.

A clinical prediction rule developed by the Pediatric Emergency Care Applied Research Network (PECARN) found that urinalysis, ANC, and PCT performed well in identifying infants 60 days or younger at low risk for SBI and IBI.¹¹ This prospective observational study of 1,821 infants 60 days or younger in 26 US EDs found 170 (9.3%) with SBI and 30 (1.6%) with IBI; 10 had bacterial meningitis. Only one patient with IBI was classified as low risk, a 30-day-old whose blood culture grew Enterobacter cloacae and who had a negative repeat blood culture prior to antibiotic treatment. Together, a negative urinalysis, ANC of 4,090/µL or less, and PCT level of 1.71 ng/mL or less were excellent in predicting infants at low risk for both SBI and IBI, with a sensitivity of 97% and negative LR of 0.05 for the outcome of IBI. When applying these variables with “rounded cutoffs” of PCT levels less than 0.5 ng/mL (chosen by the authors because it is a more commonly used cutoff) and ANC of 4,000/µL or less to identify infants at low risk for SBI, their performance was similar to nonrounded cutoffs. Data for the rule with rounded cutoffs in identifying infants at low risk for IBI were not presented. The PECARN study was limited by the small numbers of infants with IBIs, and the authors recommended caution when applying the rule to infants 28 days or younger.

Older clinical decision rules without PCT, such as the Rochester and modified Philadelphia criteria, use clinical and laboratory features to assess risk of IBI.³ Recent studies have evaluated these criteria in cohorts with larger numbers of infants with IBI since the derivation studies included mostly infants with SBI and small numbers with IBI.³ Gomez et al demonstrated that the Rochester criteria had lower sensitivity and higher negative LR than the Step-by-Step approach in IBI detection.¹⁰ In a case-control study of 135 cases of IBI with 249 matched controls, Aronson et al reported that the modified Philadelphia criteria had higher sensitivity but lower specificity than the Rochester criteria for IBI detection.¹² The ability of the Rochester and modified Philadelphia criteria to rule out IBI, as demonstrated by the negative LR (range 0.2-0.4), was inferior to the negative LRs documented by Milcent et al⁸ (PCT cutoff value of 0.3 ng/mL), the Step-by-Step approach,¹⁰ and the PECARN rule¹¹ (range 0.05-0.17; Table). However, clinical decision rules with and without PCT suffer similar limitations in having poor specificity in identifying infants likely to have IBI.

Several key knowledge gaps around PCT use for diagnosing neonatal infections exist. First, the optimal use of PCT in context with other biomarkers and clinical decision rules remains uncertain. A meta-analysis of 28 studies involving over 2,600 infants that compared PCT level (with and without CRP) with isolated CRP and presepsin levels found that PCT in combination with CRP had greater diagnostic accuracy than either PCT or CRP alone, which highlights a potential opportunity for prospective study.¹³ Second, more data are needed on the use of PCT in the ≤ 28-day age group given the increased risk of both IBI and neonatal herpes simplex virus infection (HSV), compared with that in the second month of life. Neonatal HSV poses diagnostic challenges because half of infants will initially present as afebrile,¹⁴ and delays in initiating antiviral treatment dramatically increase the risk of permanent disability or death.¹⁵ There have been no prospective studies evaluating PCT use as part of neonatal HSV evaluations.

In summary, PCT can play an important adjunctive diagnostic role in the evaluation of febrile young infants, especially during the second month of life when outpatient management is more likely to be considered. PCT is superior to other inflammatory markers in identifying IBI, though the optimal cutoffs to maximize sensitivity and specificity are uncertain. Its performance characteristics, both alone and within clinical decision rules, can help clinicians better identify children at low risk for IBI when compared with clinical decision rules without PCT. PCT measurement can help clinicians miss fewer infants with IBI and identify infants for whom safely doing less is an appropriate option, which can ultimately reduce costs and hospitalizations. PCT may be particularly helpful when the clinical history is difficult to assess or when other diagnostic test results are missing or give conflicting results. Centers that use PCT will need to ensure that results are available within a short turnaround time (a few hours) in order to meaningfully affect care. Future studies of PCT in febrile infant evaluations should focus on identifying optimal strategies for incorporating this biomarker into risk assessments that present information to parents in a way that enables them to understand their child’s risk of a serious infection.

Febrile infants 60 days of age or younger pose a significant diagnostic challenge for clinicians. Most of these infants are well appearing and do not have localizing signs or symptoms of infection, yet they may have serious bacterial infections (SBI) such as urinary tract infection (UTI), bacteremia, and meningitis. While urinalysis is highly sensitive for predicting UTI,¹ older clinical decision rules and biomarkers such as white blood cell (WBC) count, absolute neutrophil count (ANC), and C-reactive protein (CRP) lack both appropriate sensitivity and specificity for identifying bacteremia and meningitis (ie, invasive bacterial infection [IBI]),^2,3 which affect approximately 2.4% and 0.9% of febrile infants during the first 2 months of life, respectively.⁴ The lack of accurate diagnostic markers can drive overuse of laboratory testing, antibiotics, and hospitalization despite the low rates of these infections. As a result, procalcitonin (PCT) has generated interest because of its potential to serve as a more accurate biomarker for bacterial infections. This review summarizes recent literature on the diagnostic utility of PCT in the identification of IBI in febrile young infants 60 days or younger.

Procalcitonin is undetectable in noninflammatory states but can be detected in the blood within 4 to 6 hours after initial bacterial infection.⁵ Its production is stimulated throughout various tissues of the body by cytokines such as interleukin-6 and tumor necrosis factor, which are produced in response to bacterial infections. Interferon-γ, which is produced in response to viral infections, attenuates PCT production. While these characteristics suggest promise for PCT as a more specific screening test for underlying bacterial infection, there are caveats. PCT levels are physiologically elevated in the first 48 hours of life and vary with gestational age, factors that should be considered when interpreting results.⁶ Additionally, PCT levels can rise in other inflammatory states such as autoimmune conditions and certain malignancies,⁵ though these states are unlikely to confound the evaluation of febrile young infants.

Because of PCT’s potential to be more specific than other commonly used biomarkers, multiple studies have evaluated its performance characteristics in febrile young infants. Gomez et al retrospectively evaluated 1,112 well-appearing infants younger than 3 months with fever without a source in seven European emergency departments (EDs).⁷ Overall, 23 infants (2.1%) had IBI (1 with meningitis). A PCT level of 0.5 ng/mL or greater was the only independent risk factor for IBI (adjusted odds ratio, 21.69; 95% CI, 7.93-59.28). Four infants with IBI had a PCT level less than 0.5 ng/mL, and none of these four had meningitis. PCT was superior to CRP, ANC, and WBC in detecting IBI (area under the curve [AUC], 0.825; 95% CI, 0.698-0.952). PCT was the also the best marker for identifying IBI among 451 infants with a normal urine dipstick and fever detected ≤6 hours before presentation (AUC, 0.819; 95% CI, 0.551-1.087).

In the largest prospective study to date evaluating the diagnostic accuracy of PCT in febrile young infants, Milcent et al studied 2,047 previously healthy infants aged 7-91 days admitted for fever from 15 French EDs.⁸ In total, 21 (1%) had an IBI (8 with meningitis). PCT performed better than CRP, ANC, and WBC for the detection of IBI with an AUC of 0.91 (95% CI, 0.83-0.99). In a multivariable model, a PCT level of 0.3 ng/mL or greater was the only independent risk factor for IBI with an adjusted odds ratio of 40.3 (95% CI, 5.0-332). Only one infant with IBI had a PCT level less than 0.3 ng/mL. This infant was 83 days old, had 4 hours of fever, and became afebrile spontaneously prior to the blood culture revealing Streptococcus pneumoniae. PCT also performed better than CRP in the detection of IBI in infants 7-30 days of age and those with fever for less than 6 hours, though both subgroups had small numbers of infants with IBI. The authors determined that a PCT level of 0.3 ng/mL was the optimal cutoff for ruling out IBI; this cutoff had a sensitivity of 90% and negative likelihood ratio (LR) of 0.1 (Table). In contrast, the more commonly studied PCT cutoff of 0.5 ng/mL increased the negative LR to 0.2. The authors suggested that PCT, when used in the context of history, exam, and tests such as urinalysis, could identify infants at low risk of IBI.

Gomez et al conducted a prospective, single-center study of well-appearing infants with fever without a source and negative urine dipsticks.⁹ They identified IBI in 9 of 196 infants (4.5%) 21 days or younger and 13 of 1,331 infants (1.0%) 22-90 days old. PCT was superior to CRP and ANC for IBI detection in both age groups. However, in infants 21 days or younger, both the positive and negative LRs for PCT levels of 0.5 ng/mL or greater were poor (Table). Differences in results from the prior two studies^7,8 may be related to smaller sample size and differences in patient population because this study included infants younger than 7 days and a higher proportion of infants presenting within 6 hours of fever.

PCT has also been incorporated into clinical decision rules for febrile young infants, primarily to identify those at low risk of either IBI or SBI. The Step-by-Step approach¹⁰ classified well-appearing febrile infants 90 days or younger as having a high risk of IBI if they were ill appearing, younger than 21 days old, had a positive urine dipstick or a PCT level of 0.5 ng/mL or greater, and classified them as intermediate risk if they had a CRP level greater than 20 mg/L or ANC level greater than 10,000/µL. The remaining infants were classified as low risk and could be managed as outpatients without lumbar puncture or empiric antibiotics. Of note, derivation of this rule excluded patients with respiratory signs or symptoms. In a prospective validation study with 2,185 infants from 11 European EDs, 87 (4.0%) had an IBI (10 with bacterial meningitis). Sequentially identifying patients as high risk using general appearance, age, and urine dipstick alone identified 80% of infants with IBI and 90% of those with bacterial meningitis. The remaining case of meningitis would have been detected by an elevated PCT. A total of 7 of 991 infants (0.7%) classified as low risk had an IBI and none had meningitis. Six of these infants had a fever duration of less than 2 hours, which would not be enough time for PCT to rise. The Step-by-Step approach, with a sensitivity of 92% and negative LR of 0.17, performed well in the ability to rule out IBI.

A clinical prediction rule developed by the Pediatric Emergency Care Applied Research Network (PECARN) found that urinalysis, ANC, and PCT performed well in identifying infants 60 days or younger at low risk for SBI and IBI.¹¹ This prospective observational study of 1,821 infants 60 days or younger in 26 US EDs found 170 (9.3%) with SBI and 30 (1.6%) with IBI; 10 had bacterial meningitis. Only one patient with IBI was classified as low risk, a 30-day-old whose blood culture grew Enterobacter cloacae and who had a negative repeat blood culture prior to antibiotic treatment. Together, a negative urinalysis, ANC of 4,090/µL or less, and PCT level of 1.71 ng/mL or less were excellent in predicting infants at low risk for both SBI and IBI, with a sensitivity of 97% and negative LR of 0.05 for the outcome of IBI. When applying these variables with “rounded cutoffs” of PCT levels less than 0.5 ng/mL (chosen by the authors because it is a more commonly used cutoff) and ANC of 4,000/µL or less to identify infants at low risk for SBI, their performance was similar to nonrounded cutoffs. Data for the rule with rounded cutoffs in identifying infants at low risk for IBI were not presented. The PECARN study was limited by the small numbers of infants with IBIs, and the authors recommended caution when applying the rule to infants 28 days or younger.

Older clinical decision rules without PCT, such as the Rochester and modified Philadelphia criteria, use clinical and laboratory features to assess risk of IBI.³ Recent studies have evaluated these criteria in cohorts with larger numbers of infants with IBI since the derivation studies included mostly infants with SBI and small numbers with IBI.³ Gomez et al demonstrated that the Rochester criteria had lower sensitivity and higher negative LR than the Step-by-Step approach in IBI detection.¹⁰ In a case-control study of 135 cases of IBI with 249 matched controls, Aronson et al reported that the modified Philadelphia criteria had higher sensitivity but lower specificity than the Rochester criteria for IBI detection.¹² The ability of the Rochester and modified Philadelphia criteria to rule out IBI, as demonstrated by the negative LR (range 0.2-0.4), was inferior to the negative LRs documented by Milcent et al⁸ (PCT cutoff value of 0.3 ng/mL), the Step-by-Step approach,¹⁰ and the PECARN rule¹¹ (range 0.05-0.17; Table). However, clinical decision rules with and without PCT suffer similar limitations in having poor specificity in identifying infants likely to have IBI.

Several key knowledge gaps around PCT use for diagnosing neonatal infections exist. First, the optimal use of PCT in context with other biomarkers and clinical decision rules remains uncertain. A meta-analysis of 28 studies involving over 2,600 infants that compared PCT level (with and without CRP) with isolated CRP and presepsin levels found that PCT in combination with CRP had greater diagnostic accuracy than either PCT or CRP alone, which highlights a potential opportunity for prospective study.¹³ Second, more data are needed on the use of PCT in the ≤ 28-day age group given the increased risk of both IBI and neonatal herpes simplex virus infection (HSV), compared with that in the second month of life. Neonatal HSV poses diagnostic challenges because half of infants will initially present as afebrile,¹⁴ and delays in initiating antiviral treatment dramatically increase the risk of permanent disability or death.¹⁵ There have been no prospective studies evaluating PCT use as part of neonatal HSV evaluations.

In summary, PCT can play an important adjunctive diagnostic role in the evaluation of febrile young infants, especially during the second month of life when outpatient management is more likely to be considered. PCT is superior to other inflammatory markers in identifying IBI, though the optimal cutoffs to maximize sensitivity and specificity are uncertain. Its performance characteristics, both alone and within clinical decision rules, can help clinicians better identify children at low risk for IBI when compared with clinical decision rules without PCT. PCT measurement can help clinicians miss fewer infants with IBI and identify infants for whom safely doing less is an appropriate option, which can ultimately reduce costs and hospitalizations. PCT may be particularly helpful when the clinical history is difficult to assess or when other diagnostic test results are missing or give conflicting results. Centers that use PCT will need to ensure that results are available within a short turnaround time (a few hours) in order to meaningfully affect care. Future studies of PCT in febrile infant evaluations should focus on identifying optimal strategies for incorporating this biomarker into risk assessments that present information to parents in a way that enables them to understand their child’s risk of a serious infection.

Febrile infants 60 days of age or younger pose a significant diagnostic challenge for clinicians. Most of these infants are well appearing and do not have localizing signs or symptoms of infection, yet they may have serious bacterial infections (SBI) such as urinary tract infection (UTI), bacteremia, and meningitis. While urinalysis is highly sensitive for predicting UTI,¹ older clinical decision rules and biomarkers such as white blood cell (WBC) count, absolute neutrophil count (ANC), and C-reactive protein (CRP) lack both appropriate sensitivity and specificity for identifying bacteremia and meningitis (ie, invasive bacterial infection [IBI]),^2,3 which affect approximately 2.4% and 0.9% of febrile infants during the first 2 months of life, respectively.⁴ The lack of accurate diagnostic markers can drive overuse of laboratory testing, antibiotics, and hospitalization despite the low rates of these infections. As a result, procalcitonin (PCT) has generated interest because of its potential to serve as a more accurate biomarker for bacterial infections. This review summarizes recent literature on the diagnostic utility of PCT in the identification of IBI in febrile young infants 60 days or younger.

Procalcitonin is undetectable in noninflammatory states but can be detected in the blood within 4 to 6 hours after initial bacterial infection.⁵ Its production is stimulated throughout various tissues of the body by cytokines such as interleukin-6 and tumor necrosis factor, which are produced in response to bacterial infections. Interferon-γ, which is produced in response to viral infections, attenuates PCT production. While these characteristics suggest promise for PCT as a more specific screening test for underlying bacterial infection, there are caveats. PCT levels are physiologically elevated in the first 48 hours of life and vary with gestational age, factors that should be considered when interpreting results.⁶ Additionally, PCT levels can rise in other inflammatory states such as autoimmune conditions and certain malignancies,⁵ though these states are unlikely to confound the evaluation of febrile young infants.

Because of PCT’s potential to be more specific than other commonly used biomarkers, multiple studies have evaluated its performance characteristics in febrile young infants. Gomez et al retrospectively evaluated 1,112 well-appearing infants younger than 3 months with fever without a source in seven European emergency departments (EDs).⁷ Overall, 23 infants (2.1%) had IBI (1 with meningitis). A PCT level of 0.5 ng/mL or greater was the only independent risk factor for IBI (adjusted odds ratio, 21.69; 95% CI, 7.93-59.28). Four infants with IBI had a PCT level less than 0.5 ng/mL, and none of these four had meningitis. PCT was superior to CRP, ANC, and WBC in detecting IBI (area under the curve [AUC], 0.825; 95% CI, 0.698-0.952). PCT was the also the best marker for identifying IBI among 451 infants with a normal urine dipstick and fever detected ≤6 hours before presentation (AUC, 0.819; 95% CI, 0.551-1.087).

In the largest prospective study to date evaluating the diagnostic accuracy of PCT in febrile young infants, Milcent et al studied 2,047 previously healthy infants aged 7-91 days admitted for fever from 15 French EDs.⁸ In total, 21 (1%) had an IBI (8 with meningitis). PCT performed better than CRP, ANC, and WBC for the detection of IBI with an AUC of 0.91 (95% CI, 0.83-0.99). In a multivariable model, a PCT level of 0.3 ng/mL or greater was the only independent risk factor for IBI with an adjusted odds ratio of 40.3 (95% CI, 5.0-332). Only one infant with IBI had a PCT level less than 0.3 ng/mL. This infant was 83 days old, had 4 hours of fever, and became afebrile spontaneously prior to the blood culture revealing Streptococcus pneumoniae. PCT also performed better than CRP in the detection of IBI in infants 7-30 days of age and those with fever for less than 6 hours, though both subgroups had small numbers of infants with IBI. The authors determined that a PCT level of 0.3 ng/mL was the optimal cutoff for ruling out IBI; this cutoff had a sensitivity of 90% and negative likelihood ratio (LR) of 0.1 (Table). In contrast, the more commonly studied PCT cutoff of 0.5 ng/mL increased the negative LR to 0.2. The authors suggested that PCT, when used in the context of history, exam, and tests such as urinalysis, could identify infants at low risk of IBI.

Gomez et al conducted a prospective, single-center study of well-appearing infants with fever without a source and negative urine dipsticks.⁹ They identified IBI in 9 of 196 infants (4.5%) 21 days or younger and 13 of 1,331 infants (1.0%) 22-90 days old. PCT was superior to CRP and ANC for IBI detection in both age groups. However, in infants 21 days or younger, both the positive and negative LRs for PCT levels of 0.5 ng/mL or greater were poor (Table). Differences in results from the prior two studies^7,8 may be related to smaller sample size and differences in patient population because this study included infants younger than 7 days and a higher proportion of infants presenting within 6 hours of fever.

PCT has also been incorporated into clinical decision rules for febrile young infants, primarily to identify those at low risk of either IBI or SBI. The Step-by-Step approach¹⁰ classified well-appearing febrile infants 90 days or younger as having a high risk of IBI if they were ill appearing, younger than 21 days old, had a positive urine dipstick or a PCT level of 0.5 ng/mL or greater, and classified them as intermediate risk if they had a CRP level greater than 20 mg/L or ANC level greater than 10,000/µL. The remaining infants were classified as low risk and could be managed as outpatients without lumbar puncture or empiric antibiotics. Of note, derivation of this rule excluded patients with respiratory signs or symptoms. In a prospective validation study with 2,185 infants from 11 European EDs, 87 (4.0%) had an IBI (10 with bacterial meningitis). Sequentially identifying patients as high risk using general appearance, age, and urine dipstick alone identified 80% of infants with IBI and 90% of those with bacterial meningitis. The remaining case of meningitis would have been detected by an elevated PCT. A total of 7 of 991 infants (0.7%) classified as low risk had an IBI and none had meningitis. Six of these infants had a fever duration of less than 2 hours, which would not be enough time for PCT to rise. The Step-by-Step approach, with a sensitivity of 92% and negative LR of 0.17, performed well in the ability to rule out IBI.

A clinical prediction rule developed by the Pediatric Emergency Care Applied Research Network (PECARN) found that urinalysis, ANC, and PCT performed well in identifying infants 60 days or younger at low risk for SBI and IBI.¹¹ This prospective observational study of 1,821 infants 60 days or younger in 26 US EDs found 170 (9.3%) with SBI and 30 (1.6%) with IBI; 10 had bacterial meningitis. Only one patient with IBI was classified as low risk, a 30-day-old whose blood culture grew Enterobacter cloacae and who had a negative repeat blood culture prior to antibiotic treatment. Together, a negative urinalysis, ANC of 4,090/µL or less, and PCT level of 1.71 ng/mL or less were excellent in predicting infants at low risk for both SBI and IBI, with a sensitivity of 97% and negative LR of 0.05 for the outcome of IBI. When applying these variables with “rounded cutoffs” of PCT levels less than 0.5 ng/mL (chosen by the authors because it is a more commonly used cutoff) and ANC of 4,000/µL or less to identify infants at low risk for SBI, their performance was similar to nonrounded cutoffs. Data for the rule with rounded cutoffs in identifying infants at low risk for IBI were not presented. The PECARN study was limited by the small numbers of infants with IBIs, and the authors recommended caution when applying the rule to infants 28 days or younger.

Older clinical decision rules without PCT, such as the Rochester and modified Philadelphia criteria, use clinical and laboratory features to assess risk of IBI.³ Recent studies have evaluated these criteria in cohorts with larger numbers of infants with IBI since the derivation studies included mostly infants with SBI and small numbers with IBI.³ Gomez et al demonstrated that the Rochester criteria had lower sensitivity and higher negative LR than the Step-by-Step approach in IBI detection.¹⁰ In a case-control study of 135 cases of IBI with 249 matched controls, Aronson et al reported that the modified Philadelphia criteria had higher sensitivity but lower specificity than the Rochester criteria for IBI detection.¹² The ability of the Rochester and modified Philadelphia criteria to rule out IBI, as demonstrated by the negative LR (range 0.2-0.4), was inferior to the negative LRs documented by Milcent et al⁸ (PCT cutoff value of 0.3 ng/mL), the Step-by-Step approach,¹⁰ and the PECARN rule¹¹ (range 0.05-0.17; Table). However, clinical decision rules with and without PCT suffer similar limitations in having poor specificity in identifying infants likely to have IBI.

Several key knowledge gaps around PCT use for diagnosing neonatal infections exist. First, the optimal use of PCT in context with other biomarkers and clinical decision rules remains uncertain. A meta-analysis of 28 studies involving over 2,600 infants that compared PCT level (with and without CRP) with isolated CRP and presepsin levels found that PCT in combination with CRP had greater diagnostic accuracy than either PCT or CRP alone, which highlights a potential opportunity for prospective study.¹³ Second, more data are needed on the use of PCT in the ≤ 28-day age group given the increased risk of both IBI and neonatal herpes simplex virus infection (HSV), compared with that in the second month of life. Neonatal HSV poses diagnostic challenges because half of infants will initially present as afebrile,¹⁴ and delays in initiating antiviral treatment dramatically increase the risk of permanent disability or death.¹⁵ There have been no prospective studies evaluating PCT use as part of neonatal HSV evaluations.

In summary, PCT can play an important adjunctive diagnostic role in the evaluation of febrile young infants, especially during the second month of life when outpatient management is more likely to be considered. PCT is superior to other inflammatory markers in identifying IBI, though the optimal cutoffs to maximize sensitivity and specificity are uncertain. Its performance characteristics, both alone and within clinical decision rules, can help clinicians better identify children at low risk for IBI when compared with clinical decision rules without PCT. PCT measurement can help clinicians miss fewer infants with IBI and identify infants for whom safely doing less is an appropriate option, which can ultimately reduce costs and hospitalizations. PCT may be particularly helpful when the clinical history is difficult to assess or when other diagnostic test results are missing or give conflicting results. Centers that use PCT will need to ensure that results are available within a short turnaround time (a few hours) in order to meaningfully affect care. Future studies of PCT in febrile infant evaluations should focus on identifying optimal strategies for incorporating this biomarker into risk assessments that present information to parents in a way that enables them to understand their child’s risk of a serious infection.

Continuous pulse oximetry monitoring in children with bronchiolitis who don’t require supplemental oxygen is discouraged by practice guidelines and is recognized as a form of medical overuse.^1-3 This practice can be associated with negative outcomes, including prolonged length of stay,^4-6 increased cost of hospitalization,⁷ and alarm fatigue among nurses.⁸ Despite initiatives to reduce continuous pulse oximetry monitoring in stable patients with bronchiolitis,^1,2 wide practice variation exists between hospitals.^9,10 Previous studies have shown that higher prevalence of inpatient bronchiolitis admissions is associated with decreased utilization of unnecessary interventions.¹¹ However, the relationship between pulse oximetry use and bronchiolitis prevalence has not been studied. The objective of this study is to test the hypothesis that hospital units with lower proportions of patients admitted for bronchiolitis and those with fewer general pediatrics patients relative to subspecialty patients would have higher rates of pulse oximetry overuse.

Study Design

We conducted a substudy of the Pediatric Research in Inpatient Settings (PRIS) Network’s Eliminating Monitoring Overuse (EMO) pulse oximetry study,^10,12 a 56-hospital cross-sectional study that used direct observation to measure the prevalence of continuous pulse oximetry monitoring in hospitalized infants with bronchiolitis who did not require supplemental oxygen between December 1, 2018, through March 31, 2019. This substudy was not included as part of the original aims of the project and was proposed as a separate analysis during data collection. For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study and served as the central IRB. The Research Ethics Board at University of Calgary also approved the study.

Site Selection

Hospitals with at least 60 observations were eligible for inclusion. Of the 32 hospitals that conducted the minimum observations, 25 agreed to participate (21 free-standing children’s hospitals, 3 children’s hospitals within general hospitals, and 1 community hospital).

Patient Population

The parent study included patients aged 8 weeks through 23 months with a primary diagnosis of bronchiolitis. Patients were included only if they were not receiving supplemental oxygen or nasal cannula flow at the time of data collection. The inclusion and exclusion criteria were for both the parent study and the substudy. Further inclusion and exclusion criteria have been described previously.^10,12

Data Collection

In order to ascertain continuous pulse oximetry monitoring status, staff at each hospital performed observational rounds by walking to the bedside of each patient who met inclusion criteria. Additional methodology for the parent study has been published elsewhere.^10,12

Bronchiolitis Admission Volume by Unit

Collaborators at each hospital gathered bronchiolitis census data from each unit that admitted patients with bronchiolitis. Units were identified prior to data collection and were characterized at the institution level based on previous local definitions. Each site was responsible for using institution-specific data collection methods for determining bronchiolitis and total admissions on each unit (eg, departmental reports or directly querying admissions data using International Classification of Diseases, Tenth Revision, diagnosis codes for bronchiolitis) over the same period as the parent study. Following data analysis, bronchiolitis admission burden was classified into five categories, based on less than 10%, 10% to less than 20%, 20% to less than 30%, 30% to less than 40%, or 40% or more of total admissions having a primary discharge diagnosis of bronchiolitis during the study period. This categorization allowed investigators to determine whether there was a dose-dependent response among categories.

Unit Composition

Site investigators also completed a survey identifying which patients were admitted to each unit (eg, general pediatrics only, medical subspecialty, surgical). Based on these results, units were further classified into seven types (Appendix Table). For the final analysis, units caring exclusively for general pediatrics patients were compared to all other unit types.

Analysis

Bronchiolitis admission burden and unit composition data were combined with observations of pulse oximetry monitoring use of patients not requiring supplemental oxygen from the parent study. We determined unadjusted observed monitoring proportions for each unit’s bronchiolitis admission burden category across all 25 hospitals. This was calculated as a simple proportion of the total number of observations during which patients were continuously monitored divided by the total number of observations performed within each unit’s admission category. We then calculated unadjusted odds ratios using the 40% and higher bronchiolitis admission burden category as a reference. We calculated similar proportions and odds ratios for the dichotomous unit composition variable. Next, we used mixed-effects logistic regression with a random intercept for each hospital to allow for differences in baseline monitoring rates, which varied widely between hospitals (2% to 92%),¹⁰ to calculate adjusted odds ratios for the unit’s admission category and unit’s composition. We also adjusted for the same covariates used in the primary study’s analysis (Table).¹⁰

We analyzed 2,366 observations of bronchiolitis patients from 25 hospitals. Most observations were concentrated in freestanding children’s hospitals (89%), and 50% were from hospitals with more than 250 pediatric beds. Observations were well distributed among the five categories of admit burden (Table).

In unadjusted regression, the relationship between admission burden and rate of pulse oximetry use did not appear to be dose-dependent, and 95% CIs were wide. We then analyzed the data accounting for baseline differences in hospital monitoring rates and adjusted for the covariates significantly associated with continuous pulse oximetry monitoring in the primary study’s analysis with use of a mixed-effects model. As shown in the Table, low-burden units in which bronchiolitis constituted less than 10% of total admissions had a 2.16-fold increased odds of unnecessary pulse oximetry monitoring compared to high-burden units in which bronchiolitis constituted 40% or more of total admissions (95% CI, 1.27-3.69; P = .01).

In examining the subspecialty unit composition, 596 observations (25.2%) were conducted on units exclusively caring for general pediatrics patients. In the mixed-effects model adjusted for bronchiolitis admission burden and the covariates used in the study’s primary analysis, units exclusively caring for general pediatrics patients did not have significantly different independent odds of pulse oximetry monitoring use compared to units with a mixed patient population (OR 1.01; 95% CI, 0.71-1.45; P = .95) (Appendix Table).

In this multicenter observational study of children hospitalized with bronchiolitis not concurrently receiving supplemental oxygen, units that only occasionally cared for bronchiolitis patients appeared to be more likely to overuse continuous pulse oximetry during bronchiolitis hospitalizations.

This finding was not immediately apparent when examining the raw data because of wide hospital-level variation in continuous pulse oximetry monitoring use. However, when the high degree of hospital-level variation in baseline overuse was accounted for with use of a random intercept for each hospital in the mixed-effects model, units that cared for higher proportions of bronchiolitis patients had significantly lower odds of continuous pulse oximetry monitoring use compared to units that cared for these infants infrequently.

As many institutions have subspecialized units to cultivate nursing expertise for care of certain diseases and patient populations, we hypothesized that units caring primarily for children on general pediatrics units would also have lower rates of monitoring overuse compared to mixed units. Interestingly, these units did not perform better, likely because potential cultural factors that might contribute to differences in monitoring are accounted for by bronchiolitis admission burden.

Our findings build on prior literature by demonstrating that unit-level, as well as hospital-level, factors appear to drive overuse in healthcare. A prior single-site retrospective cohort study demonstrated an association between higher prevalence of inpatient bronchiolitis and decreased use of unnecessary interventions such as laboratory and radiographic testing, as well as steroid and antibiotic administration.¹¹ Although study of the relationship between volume and quality is not new to healthcare, to our knowledge, this study is the first to examine the relationship between pulse oximetry overuse in bronchiolitis and unit-level factors like admission burden and subspecialty composition.

There are several limitations. First, because the study population included only children not receiving supplemental oxygen, both the parent study and this substudy assumed that all observed use of pulse oximetry monitoring was overuse. In some cases, however, there may have been other compelling clinical reasons, institutional policies, or differences in pulse oximetry availability that were not captured during data collection or in our adjusted model. Second, hospitals used convenience sampling. It is possible this resulted in samples that were not representative of each unit’s underlying patient population or monitoring practice. In addition, not all of the 32 eligible sites were able to provide data related to hospital admissions at the unit level and thus are not included in our analysis. This remains a potential source of hospital-level selection bias.

These findings demonstrate that high bronchiolitis admission burden correlates with lower rates of unnecessary pulse oximetry monitoring in bronchiolitis. We speculate that these outcomes might reflect differing degrees of nursing comfort, expertise, and unit-level norms in caring for bronchiolitis patients, although our study was not designed to establish underlying causes. Identification of operating principles that underpin low pulse oximetry monitoring on high-burden units will provide guidance for decreasing unnecessary monitoring and will inform future studies seeking ways to discourage continuous pulse oximetry monitoring in low-risk infants. Given the institutional variation in monitoring rates, future studies examining both institution-wide and unit-level interventions will be necessary to decrease unnecessary pulse oximetry monitoring in bronchiolitis. Furthermore, these findings may be relevant to studying care quality in other disease processes, with bronchiolitis serving as a model illness for overuse.

The authors acknowledge the National Heart, Lung, and Blood Institute of the National Institutes of Health scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. The authors thank the executive council of the Pediatric Research in Inpatient Settings Network for their contributions to the early scientific development of this project. The network assessed a Collaborative Support Fee for access to the hospitals and support of this project.

The authors thank the PRIS Network collaborators for their major contributions to data collection (see Appendix).

Continuous pulse oximetry monitoring in children with bronchiolitis who don’t require supplemental oxygen is discouraged by practice guidelines and is recognized as a form of medical overuse.^1-3 This practice can be associated with negative outcomes, including prolonged length of stay,^4-6 increased cost of hospitalization,⁷ and alarm fatigue among nurses.⁸ Despite initiatives to reduce continuous pulse oximetry monitoring in stable patients with bronchiolitis,^1,2 wide practice variation exists between hospitals.^9,10 Previous studies have shown that higher prevalence of inpatient bronchiolitis admissions is associated with decreased utilization of unnecessary interventions.¹¹ However, the relationship between pulse oximetry use and bronchiolitis prevalence has not been studied. The objective of this study is to test the hypothesis that hospital units with lower proportions of patients admitted for bronchiolitis and those with fewer general pediatrics patients relative to subspecialty patients would have higher rates of pulse oximetry overuse.

Study Design

We conducted a substudy of the Pediatric Research in Inpatient Settings (PRIS) Network’s Eliminating Monitoring Overuse (EMO) pulse oximetry study,^10,12 a 56-hospital cross-sectional study that used direct observation to measure the prevalence of continuous pulse oximetry monitoring in hospitalized infants with bronchiolitis who did not require supplemental oxygen between December 1, 2018, through March 31, 2019. This substudy was not included as part of the original aims of the project and was proposed as a separate analysis during data collection. For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study and served as the central IRB. The Research Ethics Board at University of Calgary also approved the study.

Site Selection

Hospitals with at least 60 observations were eligible for inclusion. Of the 32 hospitals that conducted the minimum observations, 25 agreed to participate (21 free-standing children’s hospitals, 3 children’s hospitals within general hospitals, and 1 community hospital).

Patient Population

The parent study included patients aged 8 weeks through 23 months with a primary diagnosis of bronchiolitis. Patients were included only if they were not receiving supplemental oxygen or nasal cannula flow at the time of data collection. The inclusion and exclusion criteria were for both the parent study and the substudy. Further inclusion and exclusion criteria have been described previously.^10,12

Data Collection

In order to ascertain continuous pulse oximetry monitoring status, staff at each hospital performed observational rounds by walking to the bedside of each patient who met inclusion criteria. Additional methodology for the parent study has been published elsewhere.^10,12

Bronchiolitis Admission Volume by Unit

Collaborators at each hospital gathered bronchiolitis census data from each unit that admitted patients with bronchiolitis. Units were identified prior to data collection and were characterized at the institution level based on previous local definitions. Each site was responsible for using institution-specific data collection methods for determining bronchiolitis and total admissions on each unit (eg, departmental reports or directly querying admissions data using International Classification of Diseases, Tenth Revision, diagnosis codes for bronchiolitis) over the same period as the parent study. Following data analysis, bronchiolitis admission burden was classified into five categories, based on less than 10%, 10% to less than 20%, 20% to less than 30%, 30% to less than 40%, or 40% or more of total admissions having a primary discharge diagnosis of bronchiolitis during the study period. This categorization allowed investigators to determine whether there was a dose-dependent response among categories.

Unit Composition

Site investigators also completed a survey identifying which patients were admitted to each unit (eg, general pediatrics only, medical subspecialty, surgical). Based on these results, units were further classified into seven types (Appendix Table). For the final analysis, units caring exclusively for general pediatrics patients were compared to all other unit types.

Analysis

Bronchiolitis admission burden and unit composition data were combined with observations of pulse oximetry monitoring use of patients not requiring supplemental oxygen from the parent study. We determined unadjusted observed monitoring proportions for each unit’s bronchiolitis admission burden category across all 25 hospitals. This was calculated as a simple proportion of the total number of observations during which patients were continuously monitored divided by the total number of observations performed within each unit’s admission category. We then calculated unadjusted odds ratios using the 40% and higher bronchiolitis admission burden category as a reference. We calculated similar proportions and odds ratios for the dichotomous unit composition variable. Next, we used mixed-effects logistic regression with a random intercept for each hospital to allow for differences in baseline monitoring rates, which varied widely between hospitals (2% to 92%),¹⁰ to calculate adjusted odds ratios for the unit’s admission category and unit’s composition. We also adjusted for the same covariates used in the primary study’s analysis (Table).¹⁰

We analyzed 2,366 observations of bronchiolitis patients from 25 hospitals. Most observations were concentrated in freestanding children’s hospitals (89%), and 50% were from hospitals with more than 250 pediatric beds. Observations were well distributed among the five categories of admit burden (Table).

In unadjusted regression, the relationship between admission burden and rate of pulse oximetry use did not appear to be dose-dependent, and 95% CIs were wide. We then analyzed the data accounting for baseline differences in hospital monitoring rates and adjusted for the covariates significantly associated with continuous pulse oximetry monitoring in the primary study’s analysis with use of a mixed-effects model. As shown in the Table, low-burden units in which bronchiolitis constituted less than 10% of total admissions had a 2.16-fold increased odds of unnecessary pulse oximetry monitoring compared to high-burden units in which bronchiolitis constituted 40% or more of total admissions (95% CI, 1.27-3.69; P = .01).

In examining the subspecialty unit composition, 596 observations (25.2%) were conducted on units exclusively caring for general pediatrics patients. In the mixed-effects model adjusted for bronchiolitis admission burden and the covariates used in the study’s primary analysis, units exclusively caring for general pediatrics patients did not have significantly different independent odds of pulse oximetry monitoring use compared to units with a mixed patient population (OR 1.01; 95% CI, 0.71-1.45; P = .95) (Appendix Table).

In this multicenter observational study of children hospitalized with bronchiolitis not concurrently receiving supplemental oxygen, units that only occasionally cared for bronchiolitis patients appeared to be more likely to overuse continuous pulse oximetry during bronchiolitis hospitalizations.

This finding was not immediately apparent when examining the raw data because of wide hospital-level variation in continuous pulse oximetry monitoring use. However, when the high degree of hospital-level variation in baseline overuse was accounted for with use of a random intercept for each hospital in the mixed-effects model, units that cared for higher proportions of bronchiolitis patients had significantly lower odds of continuous pulse oximetry monitoring use compared to units that cared for these infants infrequently.

As many institutions have subspecialized units to cultivate nursing expertise for care of certain diseases and patient populations, we hypothesized that units caring primarily for children on general pediatrics units would also have lower rates of monitoring overuse compared to mixed units. Interestingly, these units did not perform better, likely because potential cultural factors that might contribute to differences in monitoring are accounted for by bronchiolitis admission burden.

Our findings build on prior literature by demonstrating that unit-level, as well as hospital-level, factors appear to drive overuse in healthcare. A prior single-site retrospective cohort study demonstrated an association between higher prevalence of inpatient bronchiolitis and decreased use of unnecessary interventions such as laboratory and radiographic testing, as well as steroid and antibiotic administration.¹¹ Although study of the relationship between volume and quality is not new to healthcare, to our knowledge, this study is the first to examine the relationship between pulse oximetry overuse in bronchiolitis and unit-level factors like admission burden and subspecialty composition.

There are several limitations. First, because the study population included only children not receiving supplemental oxygen, both the parent study and this substudy assumed that all observed use of pulse oximetry monitoring was overuse. In some cases, however, there may have been other compelling clinical reasons, institutional policies, or differences in pulse oximetry availability that were not captured during data collection or in our adjusted model. Second, hospitals used convenience sampling. It is possible this resulted in samples that were not representative of each unit’s underlying patient population or monitoring practice. In addition, not all of the 32 eligible sites were able to provide data related to hospital admissions at the unit level and thus are not included in our analysis. This remains a potential source of hospital-level selection bias.

These findings demonstrate that high bronchiolitis admission burden correlates with lower rates of unnecessary pulse oximetry monitoring in bronchiolitis. We speculate that these outcomes might reflect differing degrees of nursing comfort, expertise, and unit-level norms in caring for bronchiolitis patients, although our study was not designed to establish underlying causes. Identification of operating principles that underpin low pulse oximetry monitoring on high-burden units will provide guidance for decreasing unnecessary monitoring and will inform future studies seeking ways to discourage continuous pulse oximetry monitoring in low-risk infants. Given the institutional variation in monitoring rates, future studies examining both institution-wide and unit-level interventions will be necessary to decrease unnecessary pulse oximetry monitoring in bronchiolitis. Furthermore, these findings may be relevant to studying care quality in other disease processes, with bronchiolitis serving as a model illness for overuse.

The authors acknowledge the National Heart, Lung, and Blood Institute of the National Institutes of Health scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. The authors thank the executive council of the Pediatric Research in Inpatient Settings Network for their contributions to the early scientific development of this project. The network assessed a Collaborative Support Fee for access to the hospitals and support of this project.

The authors thank the PRIS Network collaborators for their major contributions to data collection (see Appendix).

Continuous pulse oximetry monitoring in children with bronchiolitis who don’t require supplemental oxygen is discouraged by practice guidelines and is recognized as a form of medical overuse.^1-3 This practice can be associated with negative outcomes, including prolonged length of stay,^4-6 increased cost of hospitalization,⁷ and alarm fatigue among nurses.⁸ Despite initiatives to reduce continuous pulse oximetry monitoring in stable patients with bronchiolitis,^1,2 wide practice variation exists between hospitals.^9,10 Previous studies have shown that higher prevalence of inpatient bronchiolitis admissions is associated with decreased utilization of unnecessary interventions.¹¹ However, the relationship between pulse oximetry use and bronchiolitis prevalence has not been studied. The objective of this study is to test the hypothesis that hospital units with lower proportions of patients admitted for bronchiolitis and those with fewer general pediatrics patients relative to subspecialty patients would have higher rates of pulse oximetry overuse.

Study Design

We conducted a substudy of the Pediatric Research in Inpatient Settings (PRIS) Network’s Eliminating Monitoring Overuse (EMO) pulse oximetry study,^10,12 a 56-hospital cross-sectional study that used direct observation to measure the prevalence of continuous pulse oximetry monitoring in hospitalized infants with bronchiolitis who did not require supplemental oxygen between December 1, 2018, through March 31, 2019. This substudy was not included as part of the original aims of the project and was proposed as a separate analysis during data collection. For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study and served as the central IRB. The Research Ethics Board at University of Calgary also approved the study.

Site Selection

Hospitals with at least 60 observations were eligible for inclusion. Of the 32 hospitals that conducted the minimum observations, 25 agreed to participate (21 free-standing children’s hospitals, 3 children’s hospitals within general hospitals, and 1 community hospital).

Patient Population

The parent study included patients aged 8 weeks through 23 months with a primary diagnosis of bronchiolitis. Patients were included only if they were not receiving supplemental oxygen or nasal cannula flow at the time of data collection. The inclusion and exclusion criteria were for both the parent study and the substudy. Further inclusion and exclusion criteria have been described previously.^10,12

Data Collection

In order to ascertain continuous pulse oximetry monitoring status, staff at each hospital performed observational rounds by walking to the bedside of each patient who met inclusion criteria. Additional methodology for the parent study has been published elsewhere.^10,12

Bronchiolitis Admission Volume by Unit

Collaborators at each hospital gathered bronchiolitis census data from each unit that admitted patients with bronchiolitis. Units were identified prior to data collection and were characterized at the institution level based on previous local definitions. Each site was responsible for using institution-specific data collection methods for determining bronchiolitis and total admissions on each unit (eg, departmental reports or directly querying admissions data using International Classification of Diseases, Tenth Revision, diagnosis codes for bronchiolitis) over the same period as the parent study. Following data analysis, bronchiolitis admission burden was classified into five categories, based on less than 10%, 10% to less than 20%, 20% to less than 30%, 30% to less than 40%, or 40% or more of total admissions having a primary discharge diagnosis of bronchiolitis during the study period. This categorization allowed investigators to determine whether there was a dose-dependent response among categories.

Unit Composition

Site investigators also completed a survey identifying which patients were admitted to each unit (eg, general pediatrics only, medical subspecialty, surgical). Based on these results, units were further classified into seven types (Appendix Table). For the final analysis, units caring exclusively for general pediatrics patients were compared to all other unit types.

Analysis

Bronchiolitis admission burden and unit composition data were combined with observations of pulse oximetry monitoring use of patients not requiring supplemental oxygen from the parent study. We determined unadjusted observed monitoring proportions for each unit’s bronchiolitis admission burden category across all 25 hospitals. This was calculated as a simple proportion of the total number of observations during which patients were continuously monitored divided by the total number of observations performed within each unit’s admission category. We then calculated unadjusted odds ratios using the 40% and higher bronchiolitis admission burden category as a reference. We calculated similar proportions and odds ratios for the dichotomous unit composition variable. Next, we used mixed-effects logistic regression with a random intercept for each hospital to allow for differences in baseline monitoring rates, which varied widely between hospitals (2% to 92%),¹⁰ to calculate adjusted odds ratios for the unit’s admission category and unit’s composition. We also adjusted for the same covariates used in the primary study’s analysis (Table).¹⁰

We analyzed 2,366 observations of bronchiolitis patients from 25 hospitals. Most observations were concentrated in freestanding children’s hospitals (89%), and 50% were from hospitals with more than 250 pediatric beds. Observations were well distributed among the five categories of admit burden (Table).

In unadjusted regression, the relationship between admission burden and rate of pulse oximetry use did not appear to be dose-dependent, and 95% CIs were wide. We then analyzed the data accounting for baseline differences in hospital monitoring rates and adjusted for the covariates significantly associated with continuous pulse oximetry monitoring in the primary study’s analysis with use of a mixed-effects model. As shown in the Table, low-burden units in which bronchiolitis constituted less than 10% of total admissions had a 2.16-fold increased odds of unnecessary pulse oximetry monitoring compared to high-burden units in which bronchiolitis constituted 40% or more of total admissions (95% CI, 1.27-3.69; P = .01).

In examining the subspecialty unit composition, 596 observations (25.2%) were conducted on units exclusively caring for general pediatrics patients. In the mixed-effects model adjusted for bronchiolitis admission burden and the covariates used in the study’s primary analysis, units exclusively caring for general pediatrics patients did not have significantly different independent odds of pulse oximetry monitoring use compared to units with a mixed patient population (OR 1.01; 95% CI, 0.71-1.45; P = .95) (Appendix Table).

In this multicenter observational study of children hospitalized with bronchiolitis not concurrently receiving supplemental oxygen, units that only occasionally cared for bronchiolitis patients appeared to be more likely to overuse continuous pulse oximetry during bronchiolitis hospitalizations.

This finding was not immediately apparent when examining the raw data because of wide hospital-level variation in continuous pulse oximetry monitoring use. However, when the high degree of hospital-level variation in baseline overuse was accounted for with use of a random intercept for each hospital in the mixed-effects model, units that cared for higher proportions of bronchiolitis patients had significantly lower odds of continuous pulse oximetry monitoring use compared to units that cared for these infants infrequently.

As many institutions have subspecialized units to cultivate nursing expertise for care of certain diseases and patient populations, we hypothesized that units caring primarily for children on general pediatrics units would also have lower rates of monitoring overuse compared to mixed units. Interestingly, these units did not perform better, likely because potential cultural factors that might contribute to differences in monitoring are accounted for by bronchiolitis admission burden.

Our findings build on prior literature by demonstrating that unit-level, as well as hospital-level, factors appear to drive overuse in healthcare. A prior single-site retrospective cohort study demonstrated an association between higher prevalence of inpatient bronchiolitis and decreased use of unnecessary interventions such as laboratory and radiographic testing, as well as steroid and antibiotic administration.¹¹ Although study of the relationship between volume and quality is not new to healthcare, to our knowledge, this study is the first to examine the relationship between pulse oximetry overuse in bronchiolitis and unit-level factors like admission burden and subspecialty composition.

There are several limitations. First, because the study population included only children not receiving supplemental oxygen, both the parent study and this substudy assumed that all observed use of pulse oximetry monitoring was overuse. In some cases, however, there may have been other compelling clinical reasons, institutional policies, or differences in pulse oximetry availability that were not captured during data collection or in our adjusted model. Second, hospitals used convenience sampling. It is possible this resulted in samples that were not representative of each unit’s underlying patient population or monitoring practice. In addition, not all of the 32 eligible sites were able to provide data related to hospital admissions at the unit level and thus are not included in our analysis. This remains a potential source of hospital-level selection bias.

These findings demonstrate that high bronchiolitis admission burden correlates with lower rates of unnecessary pulse oximetry monitoring in bronchiolitis. We speculate that these outcomes might reflect differing degrees of nursing comfort, expertise, and unit-level norms in caring for bronchiolitis patients, although our study was not designed to establish underlying causes. Identification of operating principles that underpin low pulse oximetry monitoring on high-burden units will provide guidance for decreasing unnecessary monitoring and will inform future studies seeking ways to discourage continuous pulse oximetry monitoring in low-risk infants. Given the institutional variation in monitoring rates, future studies examining both institution-wide and unit-level interventions will be necessary to decrease unnecessary pulse oximetry monitoring in bronchiolitis. Furthermore, these findings may be relevant to studying care quality in other disease processes, with bronchiolitis serving as a model illness for overuse.

The authors acknowledge the National Heart, Lung, and Blood Institute of the National Institutes of Health scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. The authors thank the executive council of the Pediatric Research in Inpatient Settings Network for their contributions to the early scientific development of this project. The network assessed a Collaborative Support Fee for access to the hospitals and support of this project.

The authors thank the PRIS Network collaborators for their major contributions to data collection (see Appendix).

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.