Healthcare delivery in rural America faces unique, growing challenges related to health and emergency care access.¹ Telemedicine approaches have the potential to increase rural hospitals’ ability to deliver efficient emergency care and reduce clinician shortages.² While initial evidence of telemedicine success exists, more quality research is needed to understand telemedicine patient and care team experiences,³ especially with real-time, clinician-initiated video conferencing in critical access hospital (CAH) emergency departments (ED). Some experience studies exist,⁴ but results are primarily quantitative⁵ and lack the nuanced qualitative depth needed to understand topics such as satisfaction and communication.⁶ Additionally, few explore combined patient and care team perspectives.⁵ The lack of breadth and depth makes it difficult to provide actionable recommendations for improvements and affects the feasibility of continuing this work and improving telemedicine care quality. To address these gaps, we evaluated a real-time, clinician-­initiated video conferencing program with overnight clinicians servicing ED patients in three Midwestern care system CAHs. This evaluation assessed patient and care team (nurse and clinician) experience with telemedicine using quantitative and qualitative survey data analysis.

Because this evaluation was designed to measure and improve program quality in a single healthcare system, it was deemed non-human subjects research by the organization’s institutional review board. This brief report follows telemedicine reporting guidelines.⁷

Setting and Telemedicine Program

This program, designed to reduce the need for on-call hospitalist clinicians to be onsite at CAHs overnight, was implemented in a large Midwestern nonprofit integrated healthcare system with three rural CAHs (combined capacity for 75 inpatient admissions, with full-time onsite ED clinicians and nurses, as well as on-call hospitalist clinicians) and a large metropolitan tertiary-care hospital. All adult patients presenting to CAH EDs between 6 pm and 8 am were evaluated, as usual, by an onsite ED clinician. If the admitting ED clinician and charge nurse determined that admission was appropriate, patients were signed out to remote hospitalist clinicians and roomed by onsite nurses. Nurses facilitated live audio-video telemedicine “history and physical” visits with remote clinicians via telemedicine carts (AmericanWell C750, Boston, Massachusetts, and ThinkLabs One Electronic Stethoscope, Centennial, Colorado). Already-­hospitalized patients, as well as patients admitted to a remote clinician, were cared for by the remote clinician and onsite nurse for the remainder of the night, which eliminated the need for local on-call clinicians. The onsite ED clinician responded to emergencies of already-hospitalized patients, but often consulted with remote clinicians to assist virtually with necessary orders and documentation. Remote clinicians were located at the metropolitan tertiary care hospital or home work stations.

Following a pilot period, the full-scale program was implemented in September 2017 and included 14 remote clinicians and 60 onsite nurses.

Survey Administration and Design

A postimplementation survey was designed to explore patient and care team experience with telemedicine. Patients who received a telemedicine visit between September 2017 and April 2018 were mailed a paper survey. Nonresponders were called by professional interviewers affiliated with the healthcare system. All participating clinicians (N = 14, all MDs) and nurses (N = 60, all RNs) were emailed an online care team survey with phone-in option. Care team nonresponders were sent up to two reminder emails.

Surveys captured the following five constructs: communication, workflow integration, telemedicine technology, quality of care, and general satisfaction. Existing questionnaires were used where possible; additional items were designed with clinical experts following survey design best practices.⁸ Patient-perceived communication was assessed via three Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey items.⁹ Five additional program-developed patient survey items included satisfaction with clinician-nurse communication, satisfaction with technology, telemedicine quality of care overall and in comparison with traditional care, and whether or not patients would recommend telemedicine (Table). Four open-ended questions asked patients about improvement opportunities and general satisfaction.

Care team surveys included two items regarding ability to effectively communicate, two about satisfaction with workflow integration, one about technical problems, two about quality of care, and one about general satisfaction. Open-ended questions gathered further information and recommendations to improve communication, workflow integration, technology issues, and general satisfaction.

Analysis

Closed-ended items were dichotomized (satisfied yes/no); descriptive statistics (frequencies/percents) are presented to quantify patient and care team experience. Quantitative analyses were conducted in SAS software version 9.4 (SAS Institute, Cary, North Carolina). Open-ended responses were coded separately for patient and care team experience, following qualitative content analysis best practices.¹⁰ A lead coder read all responses, created a coding framework of identified themes, and coded individual responses. A second coder independently coded responses using the same framework. Interrater reliability was calculated for each major theme using percent agreement and prevalence- and bias-adjusted k (PABAK) statistic. A single representative quote was selected and lightly edited for each subtheme to deepen understanding and provide respondent voice.¹¹

Of eligible patients mailed a survey (N = 408), 3% self-reported as ineligible, and 54% completed the survey. This is a maximum response rate (response rate 6) according to the American Association for Public Opinion Research.¹² Patients were 67 years old on average (SD = 15), they were primarily white (97%), and 54% were female. All clinicians and 63% of nurses completed the survey.¹² Clinicians and nurses were 29% and 95% female, respectively.

Quantitative results (Table) show generally positive experience across patient and care team respondents. Over 90% were satisfied with all measures of communication. Care teams had high satisfaction with admissions processes and reported telemedicine improved cross-coverage. Patient-reported technology experience was positive but was less positive from the care team perspective. Care teams reported lower absolute quality of care than did patients but were more likely to perceive telemedicine as high quality, compared with traditional care. Most patients, clinicians, and nurses would recommend telemedicine.

Qualitatively, four major themes were identified in open-­ended responses with high interrater reliability (PABAK ranging from 0.92 to 0.98 in patient responses and 0.88 to 0.95 in care team responses) and aligned with the quantitative survey constructs: clinician-nurse communication, clinician-patient communication, workflow integration, and telemedicine technology. Patients reported satisfaction with communication with remote clinicians:

“[The clinician] was extremely attentive to me and what was going on. She was articulate and clear. I understood what was going to happen.” –Patient

Care teams suggested concrete improvement opportunities:

“I’d prefer to have some time with nursing staff both before and (sometimes) after the patient encounter.” –Clinician

“Since we cannot hear what [the clinicians] are hearing with the stethoscope, it’s nice when they tell us when to move it to the next spot.” –Nurse

Clinicians and nurses gave favorable responses regarding workflow integration, though time (both admissions wait time and session duration) was a reported opportunity:

“It would be helpful if we could speed up the time from admit request to screen time.” –Clinician

“When the [clinicians] get swamped, they’re hard to get a hold of, and admissions can take a long time. They may have too much on their plates dealing with several locations.” –Nurse

Technology issues—internet connection, stethoscope, sound, and screen or camera—were mentioned by patients and care teams, though technology was reviewed favorably overall by most patients:

“I was fascinated by the technology. Visiting someone over a television was impressive. ... The picture, the sound clarity, and the connection itself was flawless.” –Patient

Some patients commented that telemedicine was the best option given the situation, but still preferred an in-person doctor:

“If a doctor wasn’t available, telemedicine is better than nothing.” –Patient

Nurses who would not recommend telemedicine noted the need for personal connection:

“[I] still prefer [an] in-person MD for more personal contact. The older patients often state they wish the doctor would come and see them.” –Nurse

Patients who would not recommend telemedicine also desired personal connection:

“I would sooner talk to a person than a machine.” –Patient

A few clinicians noted the connection with patients would be improved if they knew about others in the room:

“It’d be nice if everyone in the room was introduced. Sometimes people are sitting out of view of the camera and I don’t realize they’re there until later.” –Clinician

These results make important contributions to understanding and improving the telemedicine experience in rural emergency hospital medicine. While the predominantly white patient respondent population limits generalizability, these demographics are representative of the overall population of the participating hospitals. A strength of this evaluation is its contemporaneous consideration of patient and care team experience with both quantitative and rich, qualitative analysis. Patients and care teams alike thought overnight telemedicine was better than the status quo. While our quality of care findings align with some previous literature,¹³ care teams in the current analysis overwhelmingly would recommend telemedicine, whereas some clinicians in prior work would not recommend telemedicine.¹⁴

In terms of communication, in line with existing literature, some patients still preferred in-person visits,¹⁵ a view also shared by some care team members. Workflow and technology barriers were raised, corroborating existing work,¹³ but actionable solutions (eg, adding care team–only time before visits or verbalizing when to move stethoscopes) were also identified.

Embedding patient and care team experience surveys and sharing results is critical in advancing telemedicine. Findings from this evaluation strengthen the case for payer reimbursement of telemedicine in rural acute care. Continued work to improve, test, and publish findings on patient and care team experience with telemedicine is critical to providing quality services in often-underserved communities.

The authors would like to acknowledge the contributions of Ann Werner in identifying the patient survey sample, Brian Barklind in identifying source data for the analysis, and both Brian Barklind and Rachael Rivard for conducting the analyses and summarizing results. We would also like to thank Kelly Logue for her involvement in conceptualizing the telemedicine evaluation described here, as well as Larisa Polynskaya for her help preparing the manuscript for publication, and the care teams and patients who provided valuable input.

Healthcare delivery in rural America faces unique, growing challenges related to health and emergency care access.¹ Telemedicine approaches have the potential to increase rural hospitals’ ability to deliver efficient emergency care and reduce clinician shortages.² While initial evidence of telemedicine success exists, more quality research is needed to understand telemedicine patient and care team experiences,³ especially with real-time, clinician-initiated video conferencing in critical access hospital (CAH) emergency departments (ED). Some experience studies exist,⁴ but results are primarily quantitative⁵ and lack the nuanced qualitative depth needed to understand topics such as satisfaction and communication.⁶ Additionally, few explore combined patient and care team perspectives.⁵ The lack of breadth and depth makes it difficult to provide actionable recommendations for improvements and affects the feasibility of continuing this work and improving telemedicine care quality. To address these gaps, we evaluated a real-time, clinician-­initiated video conferencing program with overnight clinicians servicing ED patients in three Midwestern care system CAHs. This evaluation assessed patient and care team (nurse and clinician) experience with telemedicine using quantitative and qualitative survey data analysis.

Because this evaluation was designed to measure and improve program quality in a single healthcare system, it was deemed non-human subjects research by the organization’s institutional review board. This brief report follows telemedicine reporting guidelines.⁷

Setting and Telemedicine Program

This program, designed to reduce the need for on-call hospitalist clinicians to be onsite at CAHs overnight, was implemented in a large Midwestern nonprofit integrated healthcare system with three rural CAHs (combined capacity for 75 inpatient admissions, with full-time onsite ED clinicians and nurses, as well as on-call hospitalist clinicians) and a large metropolitan tertiary-care hospital. All adult patients presenting to CAH EDs between 6 pm and 8 am were evaluated, as usual, by an onsite ED clinician. If the admitting ED clinician and charge nurse determined that admission was appropriate, patients were signed out to remote hospitalist clinicians and roomed by onsite nurses. Nurses facilitated live audio-video telemedicine “history and physical” visits with remote clinicians via telemedicine carts (AmericanWell C750, Boston, Massachusetts, and ThinkLabs One Electronic Stethoscope, Centennial, Colorado). Already-­hospitalized patients, as well as patients admitted to a remote clinician, were cared for by the remote clinician and onsite nurse for the remainder of the night, which eliminated the need for local on-call clinicians. The onsite ED clinician responded to emergencies of already-hospitalized patients, but often consulted with remote clinicians to assist virtually with necessary orders and documentation. Remote clinicians were located at the metropolitan tertiary care hospital or home work stations.

Following a pilot period, the full-scale program was implemented in September 2017 and included 14 remote clinicians and 60 onsite nurses.

Survey Administration and Design

A postimplementation survey was designed to explore patient and care team experience with telemedicine. Patients who received a telemedicine visit between September 2017 and April 2018 were mailed a paper survey. Nonresponders were called by professional interviewers affiliated with the healthcare system. All participating clinicians (N = 14, all MDs) and nurses (N = 60, all RNs) were emailed an online care team survey with phone-in option. Care team nonresponders were sent up to two reminder emails.

Surveys captured the following five constructs: communication, workflow integration, telemedicine technology, quality of care, and general satisfaction. Existing questionnaires were used where possible; additional items were designed with clinical experts following survey design best practices.⁸ Patient-perceived communication was assessed via three Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey items.⁹ Five additional program-developed patient survey items included satisfaction with clinician-nurse communication, satisfaction with technology, telemedicine quality of care overall and in comparison with traditional care, and whether or not patients would recommend telemedicine (Table). Four open-ended questions asked patients about improvement opportunities and general satisfaction.

Care team surveys included two items regarding ability to effectively communicate, two about satisfaction with workflow integration, one about technical problems, two about quality of care, and one about general satisfaction. Open-ended questions gathered further information and recommendations to improve communication, workflow integration, technology issues, and general satisfaction.

Analysis

Closed-ended items were dichotomized (satisfied yes/no); descriptive statistics (frequencies/percents) are presented to quantify patient and care team experience. Quantitative analyses were conducted in SAS software version 9.4 (SAS Institute, Cary, North Carolina). Open-ended responses were coded separately for patient and care team experience, following qualitative content analysis best practices.¹⁰ A lead coder read all responses, created a coding framework of identified themes, and coded individual responses. A second coder independently coded responses using the same framework. Interrater reliability was calculated for each major theme using percent agreement and prevalence- and bias-adjusted k (PABAK) statistic. A single representative quote was selected and lightly edited for each subtheme to deepen understanding and provide respondent voice.¹¹

Of eligible patients mailed a survey (N = 408), 3% self-reported as ineligible, and 54% completed the survey. This is a maximum response rate (response rate 6) according to the American Association for Public Opinion Research.¹² Patients were 67 years old on average (SD = 15), they were primarily white (97%), and 54% were female. All clinicians and 63% of nurses completed the survey.¹² Clinicians and nurses were 29% and 95% female, respectively.

Quantitative results (Table) show generally positive experience across patient and care team respondents. Over 90% were satisfied with all measures of communication. Care teams had high satisfaction with admissions processes and reported telemedicine improved cross-coverage. Patient-reported technology experience was positive but was less positive from the care team perspective. Care teams reported lower absolute quality of care than did patients but were more likely to perceive telemedicine as high quality, compared with traditional care. Most patients, clinicians, and nurses would recommend telemedicine.

Qualitatively, four major themes were identified in open-­ended responses with high interrater reliability (PABAK ranging from 0.92 to 0.98 in patient responses and 0.88 to 0.95 in care team responses) and aligned with the quantitative survey constructs: clinician-nurse communication, clinician-patient communication, workflow integration, and telemedicine technology. Patients reported satisfaction with communication with remote clinicians:

“[The clinician] was extremely attentive to me and what was going on. She was articulate and clear. I understood what was going to happen.” –Patient

Care teams suggested concrete improvement opportunities:

“I’d prefer to have some time with nursing staff both before and (sometimes) after the patient encounter.” –Clinician

“Since we cannot hear what [the clinicians] are hearing with the stethoscope, it’s nice when they tell us when to move it to the next spot.” –Nurse

Clinicians and nurses gave favorable responses regarding workflow integration, though time (both admissions wait time and session duration) was a reported opportunity:

“It would be helpful if we could speed up the time from admit request to screen time.” –Clinician

“When the [clinicians] get swamped, they’re hard to get a hold of, and admissions can take a long time. They may have too much on their plates dealing with several locations.” –Nurse

Technology issues—internet connection, stethoscope, sound, and screen or camera—were mentioned by patients and care teams, though technology was reviewed favorably overall by most patients:

“I was fascinated by the technology. Visiting someone over a television was impressive. ... The picture, the sound clarity, and the connection itself was flawless.” –Patient

Some patients commented that telemedicine was the best option given the situation, but still preferred an in-person doctor:

“If a doctor wasn’t available, telemedicine is better than nothing.” –Patient

Nurses who would not recommend telemedicine noted the need for personal connection:

“[I] still prefer [an] in-person MD for more personal contact. The older patients often state they wish the doctor would come and see them.” –Nurse

Patients who would not recommend telemedicine also desired personal connection:

“I would sooner talk to a person than a machine.” –Patient

A few clinicians noted the connection with patients would be improved if they knew about others in the room:

“It’d be nice if everyone in the room was introduced. Sometimes people are sitting out of view of the camera and I don’t realize they’re there until later.” –Clinician

These results make important contributions to understanding and improving the telemedicine experience in rural emergency hospital medicine. While the predominantly white patient respondent population limits generalizability, these demographics are representative of the overall population of the participating hospitals. A strength of this evaluation is its contemporaneous consideration of patient and care team experience with both quantitative and rich, qualitative analysis. Patients and care teams alike thought overnight telemedicine was better than the status quo. While our quality of care findings align with some previous literature,¹³ care teams in the current analysis overwhelmingly would recommend telemedicine, whereas some clinicians in prior work would not recommend telemedicine.¹⁴

In terms of communication, in line with existing literature, some patients still preferred in-person visits,¹⁵ a view also shared by some care team members. Workflow and technology barriers were raised, corroborating existing work,¹³ but actionable solutions (eg, adding care team–only time before visits or verbalizing when to move stethoscopes) were also identified.

Embedding patient and care team experience surveys and sharing results is critical in advancing telemedicine. Findings from this evaluation strengthen the case for payer reimbursement of telemedicine in rural acute care. Continued work to improve, test, and publish findings on patient and care team experience with telemedicine is critical to providing quality services in often-underserved communities.

The authors would like to acknowledge the contributions of Ann Werner in identifying the patient survey sample, Brian Barklind in identifying source data for the analysis, and both Brian Barklind and Rachael Rivard for conducting the analyses and summarizing results. We would also like to thank Kelly Logue for her involvement in conceptualizing the telemedicine evaluation described here, as well as Larisa Polynskaya for her help preparing the manuscript for publication, and the care teams and patients who provided valuable input.

Healthcare delivery in rural America faces unique, growing challenges related to health and emergency care access.¹ Telemedicine approaches have the potential to increase rural hospitals’ ability to deliver efficient emergency care and reduce clinician shortages.² While initial evidence of telemedicine success exists, more quality research is needed to understand telemedicine patient and care team experiences,³ especially with real-time, clinician-initiated video conferencing in critical access hospital (CAH) emergency departments (ED). Some experience studies exist,⁴ but results are primarily quantitative⁵ and lack the nuanced qualitative depth needed to understand topics such as satisfaction and communication.⁶ Additionally, few explore combined patient and care team perspectives.⁵ The lack of breadth and depth makes it difficult to provide actionable recommendations for improvements and affects the feasibility of continuing this work and improving telemedicine care quality. To address these gaps, we evaluated a real-time, clinician-­initiated video conferencing program with overnight clinicians servicing ED patients in three Midwestern care system CAHs. This evaluation assessed patient and care team (nurse and clinician) experience with telemedicine using quantitative and qualitative survey data analysis.

Because this evaluation was designed to measure and improve program quality in a single healthcare system, it was deemed non-human subjects research by the organization’s institutional review board. This brief report follows telemedicine reporting guidelines.⁷

Setting and Telemedicine Program

This program, designed to reduce the need for on-call hospitalist clinicians to be onsite at CAHs overnight, was implemented in a large Midwestern nonprofit integrated healthcare system with three rural CAHs (combined capacity for 75 inpatient admissions, with full-time onsite ED clinicians and nurses, as well as on-call hospitalist clinicians) and a large metropolitan tertiary-care hospital. All adult patients presenting to CAH EDs between 6 pm and 8 am were evaluated, as usual, by an onsite ED clinician. If the admitting ED clinician and charge nurse determined that admission was appropriate, patients were signed out to remote hospitalist clinicians and roomed by onsite nurses. Nurses facilitated live audio-video telemedicine “history and physical” visits with remote clinicians via telemedicine carts (AmericanWell C750, Boston, Massachusetts, and ThinkLabs One Electronic Stethoscope, Centennial, Colorado). Already-­hospitalized patients, as well as patients admitted to a remote clinician, were cared for by the remote clinician and onsite nurse for the remainder of the night, which eliminated the need for local on-call clinicians. The onsite ED clinician responded to emergencies of already-hospitalized patients, but often consulted with remote clinicians to assist virtually with necessary orders and documentation. Remote clinicians were located at the metropolitan tertiary care hospital or home work stations.

Following a pilot period, the full-scale program was implemented in September 2017 and included 14 remote clinicians and 60 onsite nurses.

Survey Administration and Design

A postimplementation survey was designed to explore patient and care team experience with telemedicine. Patients who received a telemedicine visit between September 2017 and April 2018 were mailed a paper survey. Nonresponders were called by professional interviewers affiliated with the healthcare system. All participating clinicians (N = 14, all MDs) and nurses (N = 60, all RNs) were emailed an online care team survey with phone-in option. Care team nonresponders were sent up to two reminder emails.

Surveys captured the following five constructs: communication, workflow integration, telemedicine technology, quality of care, and general satisfaction. Existing questionnaires were used where possible; additional items were designed with clinical experts following survey design best practices.⁸ Patient-perceived communication was assessed via three Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey items.⁹ Five additional program-developed patient survey items included satisfaction with clinician-nurse communication, satisfaction with technology, telemedicine quality of care overall and in comparison with traditional care, and whether or not patients would recommend telemedicine (Table). Four open-ended questions asked patients about improvement opportunities and general satisfaction.

Care team surveys included two items regarding ability to effectively communicate, two about satisfaction with workflow integration, one about technical problems, two about quality of care, and one about general satisfaction. Open-ended questions gathered further information and recommendations to improve communication, workflow integration, technology issues, and general satisfaction.

Analysis

Closed-ended items were dichotomized (satisfied yes/no); descriptive statistics (frequencies/percents) are presented to quantify patient and care team experience. Quantitative analyses were conducted in SAS software version 9.4 (SAS Institute, Cary, North Carolina). Open-ended responses were coded separately for patient and care team experience, following qualitative content analysis best practices.¹⁰ A lead coder read all responses, created a coding framework of identified themes, and coded individual responses. A second coder independently coded responses using the same framework. Interrater reliability was calculated for each major theme using percent agreement and prevalence- and bias-adjusted k (PABAK) statistic. A single representative quote was selected and lightly edited for each subtheme to deepen understanding and provide respondent voice.¹¹

Of eligible patients mailed a survey (N = 408), 3% self-reported as ineligible, and 54% completed the survey. This is a maximum response rate (response rate 6) according to the American Association for Public Opinion Research.¹² Patients were 67 years old on average (SD = 15), they were primarily white (97%), and 54% were female. All clinicians and 63% of nurses completed the survey.¹² Clinicians and nurses were 29% and 95% female, respectively.

Quantitative results (Table) show generally positive experience across patient and care team respondents. Over 90% were satisfied with all measures of communication. Care teams had high satisfaction with admissions processes and reported telemedicine improved cross-coverage. Patient-reported technology experience was positive but was less positive from the care team perspective. Care teams reported lower absolute quality of care than did patients but were more likely to perceive telemedicine as high quality, compared with traditional care. Most patients, clinicians, and nurses would recommend telemedicine.

Qualitatively, four major themes were identified in open-­ended responses with high interrater reliability (PABAK ranging from 0.92 to 0.98 in patient responses and 0.88 to 0.95 in care team responses) and aligned with the quantitative survey constructs: clinician-nurse communication, clinician-patient communication, workflow integration, and telemedicine technology. Patients reported satisfaction with communication with remote clinicians:

“[The clinician] was extremely attentive to me and what was going on. She was articulate and clear. I understood what was going to happen.” –Patient

Care teams suggested concrete improvement opportunities:

“I’d prefer to have some time with nursing staff both before and (sometimes) after the patient encounter.” –Clinician

“Since we cannot hear what [the clinicians] are hearing with the stethoscope, it’s nice when they tell us when to move it to the next spot.” –Nurse

Clinicians and nurses gave favorable responses regarding workflow integration, though time (both admissions wait time and session duration) was a reported opportunity:

“It would be helpful if we could speed up the time from admit request to screen time.” –Clinician

“When the [clinicians] get swamped, they’re hard to get a hold of, and admissions can take a long time. They may have too much on their plates dealing with several locations.” –Nurse

Technology issues—internet connection, stethoscope, sound, and screen or camera—were mentioned by patients and care teams, though technology was reviewed favorably overall by most patients:

“I was fascinated by the technology. Visiting someone over a television was impressive. ... The picture, the sound clarity, and the connection itself was flawless.” –Patient

Some patients commented that telemedicine was the best option given the situation, but still preferred an in-person doctor:

“If a doctor wasn’t available, telemedicine is better than nothing.” –Patient

Nurses who would not recommend telemedicine noted the need for personal connection:

“[I] still prefer [an] in-person MD for more personal contact. The older patients often state they wish the doctor would come and see them.” –Nurse

Patients who would not recommend telemedicine also desired personal connection:

“I would sooner talk to a person than a machine.” –Patient

A few clinicians noted the connection with patients would be improved if they knew about others in the room:

“It’d be nice if everyone in the room was introduced. Sometimes people are sitting out of view of the camera and I don’t realize they’re there until later.” –Clinician

These results make important contributions to understanding and improving the telemedicine experience in rural emergency hospital medicine. While the predominantly white patient respondent population limits generalizability, these demographics are representative of the overall population of the participating hospitals. A strength of this evaluation is its contemporaneous consideration of patient and care team experience with both quantitative and rich, qualitative analysis. Patients and care teams alike thought overnight telemedicine was better than the status quo. While our quality of care findings align with some previous literature,¹³ care teams in the current analysis overwhelmingly would recommend telemedicine, whereas some clinicians in prior work would not recommend telemedicine.¹⁴

In terms of communication, in line with existing literature, some patients still preferred in-person visits,¹⁵ a view also shared by some care team members. Workflow and technology barriers were raised, corroborating existing work,¹³ but actionable solutions (eg, adding care team–only time before visits or verbalizing when to move stethoscopes) were also identified.

Embedding patient and care team experience surveys and sharing results is critical in advancing telemedicine. Findings from this evaluation strengthen the case for payer reimbursement of telemedicine in rural acute care. Continued work to improve, test, and publish findings on patient and care team experience with telemedicine is critical to providing quality services in often-underserved communities.

The authors would like to acknowledge the contributions of Ann Werner in identifying the patient survey sample, Brian Barklind in identifying source data for the analysis, and both Brian Barklind and Rachael Rivard for conducting the analyses and summarizing results. We would also like to thank Kelly Logue for her involvement in conceptualizing the telemedicine evaluation described here, as well as Larisa Polynskaya for her help preparing the manuscript for publication, and the care teams and patients who provided valuable input.

Sleep disturbance is common in hospitals, and both the quality and quantity of sleep are negatively affected in hospitalized patients.¹ Sleep disturbances in hospitals are associated with hyperglycemia,² delirium,³ lower patient satisfaction,⁴ and increased risk of readmission.⁵

A significant proportion of hospitalized patients receive sleep medications (ie, hypnotic medication) despite limited evidence.^6,7 Sleep medications have adverse effects including falls, fractures, cognitive impairment, and delirium.⁸ Commonly used nonbenzodiazepine sleep medications (eg, zopiclone) are perceived as safer but may have similar risks.⁹

Melatonin is increasingly used to treat insomnia, although evidence for its efficacy and safety is lacking.¹⁰ While melatonin use doubled between 2007 and 2012 in the United States,¹¹ previous hospital-based studies have not included melatonin.⁷ It is not known if increased melatonin use in the hospital mitigates use of higher-risk medications. Melatonin preparations can also have quality issues, including deviations from labelled dosage and contamination with compounds such as serotonin,¹² and patients continuing melatonin after discharge could have adverse effects if switched to different preparations.

In this study, we aimed to determine temporal trends in melatonin use in hospitalized patients, and compare them with trends in use of other sleep medications.

We conducted the study at two urban academic hospitals with a total of 706 acute care beds in the same network in Toronto, Canada. This study was approved by the University Health Network’s research ethics board.

We abstracted pharmacy dispensing data on melatonin, zopiclone, and lorazepam from January 1, 2013, to December 31, 2018. We included oral medications dispensed to inpatient units or admitted patients in the emergency department (ED). Zopiclone is the most commonly used nonbenzodiazepine sleep medication in Canada.¹³ Lorazepam is the most commonly used benzodiazepine at our institution (97% of all oral benzodiazepine doses). While lorazepam is prescribed for many nonsleep indications, we included it to assess the impact of melatonin. We did not include antipsychotics or trazodone, which are rarely newly initiated for insomnia at our institution.

We abstracted the monthly number of doses dispensed by unit and hospital. We categorized units based on the primary patient population as either internal medicine, critical care, or other. Admitted patients in the ED were counted as “other” regardless of service. As the focus of our study was on internal medicine and critical care, we did not analyze by type of unit in the “other” group, which is heterogeneous.

Each medication-dispensing event was counted as one dose, regardless of the number or strength of tablets (eg, a patient dispensed two 3-mg tablets of melatonin, for a total of 6 mg, would be counted as a single dose). Most unused doses are credited back (ie, if a medication was refused and returned to pharmacy, it was not counted). Lorazepam and zopiclone are on hospital formulary, while melatonin is not. To order melatonin, clinicians must select “Nonformulary medication” in the electronic health record and manually enter medication name, dose, route, and frequency, as well as select a justification for use. The hospital supplies nonformulary medications such as melatonin to patients.

To account for changes in patient volumes, we standardized medication dispensing rates per 1,000 inpatient days. We discovered rare instances in which the monthly number of doses was a negative number because of pharmacy inventory accounting. This issue affected only 0.13% of observations and the magnitude was small (8 doses or fewer); in these cases, we assumed the number of doses was zero.

We used line charts to visualize changes in medication dispensing over time by medication and hospital. We compared rates of medications use between unit type and hospital with use of relative difference and rate difference. Statistical analysis was performed with R (The R Foundation for Statistical Computing, 2018) using lubridate (2011), dplyr (2018), ggplot2 (2016), fmsb (2019), and forcats (2018).

A total of 1,542,225 inpatient days were analyzed, of which 60.4% were at hospital A. Internal medicine accounted for 23.5% of inpatient days, critical care for 11.7%, and other units for 64.8%.

Overall Trends in Sleep Medication Use

There were 351,131 dispensed doses of study medications (13% melatonin, 43% lorazepam, and 44% zopiclone). Overall use of the three study medications per 1,000 inpatient days increased by 25.7% during the study.

Melatonin use increased by 71.3 doses per 1,000 inpatient days during 2013-2018, while zopiclone use decreased by 20.4 doses per 1,000 inpatient days (Table). Lorazepam use increased slightly by 3.5 doses per 1,000 inpatient days. All rate differences reported in the results are statistically significant (Appendix Table).

Unit Type Comparison

Melatonin use was highest in critical care and internal medicine (50.9 and 48.4 doses per 1,000 inpatient days, respectively), compared with that in other units (19.3 doses per 1,000 inpatient days). Among critical care units, melatonin use was highest in medical-surgical units (67.4 doses per 1,000 inpatient days) and lower in cardiac and cardiovascular surgery units (24.6 and 18.3 doses per 1,000 inpatient days respectively). Zopiclone use was highest in critical care and other units (117.9 and 112.2 doses per 1,000 inpatient days, respectively) and lowest in internal medicine (57.0 doses per 1,000 inpatient days).

Hospital Site Comparison

Overall melatonin use was 65.4% higher at hospital B than at hospital A (42.4 vs 21.5 doses per 1,000 inpatient days; Figure). Zopiclone use was 81.7% lower at hospital B (54.6 vs 130.0 doses per 1,000 inpatient days).

When similar units were compared between hospitals, the trends were similar. For example, among internal medicine units, melatonin use was 66.7% higher at hospital B than at hospital A (64.4 vs 32.3 doses per 1,000 inpatient days).

During this 6-year study period of sleep medication use at two academic hospitals, overall use of melatonin, zopiclone, and lorazepam increased by 25.7%. Melatonin increased from almost no use to more than 70 doses per 1,000 inpatient days. The increase in melatonin was not accompanied by a proportional decline in zopiclone, which only decreased by 20.4 doses per 1,000 inpatient days. Lorazepam use increased slightly. This suggests that melatonin is not simply being substituted for higher-risk sleep medications and is instead being given to patients who might not have received sleep medications otherwise.

There are a few potential explanations for the disproportionate increase in melatonin. Providers may be more liberal in prescribing melatonin for insomnia because of perceived greater safety, compared with other medications. Melatonin may also be prescribed for delirium, despite a lack of high-quality evidence.¹⁴ Interestingly, melatonin use has increased despite a paucity of evidence for its efficacy or safety in hospital.⁶ Considering the additional barriers that exist to ordering melatonin, a nonformulary medication at our institution, the magnitude of increase is even more striking.

Melatonin use was highest on internal medicine and critical care units. This may reflect patient differences (eg, older patients with more comorbid conditions might leave prescribers reluctant to use benzodiazepines), differences in the physical environment (eg, noise/lighting), differences in nursing practices (eg, intensity of monitoring or medication administration), or differences in prescribing.

Melatonin use was almost twice as high at hospital B as it was at hospital A. While the services differ at each hospital, the results were similar when comparing the same unit type (eg, internal medicine). Internal medicine units have similar (though not identical) patient populations and team structures at both hospitals, and residents rotate between hospitals. Attendings and nurses are based primarily at one hospital and their practice patterns might differ. Geriatricians have a stronger presence at hospital B. Higher zopiclone use at hospital A could explain lower melatonin use. Lastly, improvement initiatives may have contributed (eg, one unit at hospital B promoted melatonin in 2017).

Our study has potential limitations. We studied dispensed rather than administered medications; however, numbers of doses dispensed but not administered are expected to be low because most unused doses are accounted for. By studying dispensing data, we might have underestimated the number of prescriptions (eg, if a patient was prescribed but refused a medication, this would not be captured). Our study did not examine medications after hospital discharge, although medications started in a hospital are often continued at discharge.^7,15 Our study could not determine indications for medication prescribing, and melatonin and lorazepam are both used for nonsleep indications. Our study could not differentiate between continuation of home medications and new prescriptions. Finally, the results may not be generalizable to other settings.

In this 6-year study of sleep medication use at two academic hospitals, we found that overall use of melatonin, zopiclone, and lorazepam increased by 25%, predominantly because of markedly increased melatonin use. Given the current lack of high-quality evidence, further research on the use of melatonin in hospitalized patients is needed.

Sleep disturbance is common in hospitals, and both the quality and quantity of sleep are negatively affected in hospitalized patients.¹ Sleep disturbances in hospitals are associated with hyperglycemia,² delirium,³ lower patient satisfaction,⁴ and increased risk of readmission.⁵

A significant proportion of hospitalized patients receive sleep medications (ie, hypnotic medication) despite limited evidence.^6,7 Sleep medications have adverse effects including falls, fractures, cognitive impairment, and delirium.⁸ Commonly used nonbenzodiazepine sleep medications (eg, zopiclone) are perceived as safer but may have similar risks.⁹

Melatonin is increasingly used to treat insomnia, although evidence for its efficacy and safety is lacking.¹⁰ While melatonin use doubled between 2007 and 2012 in the United States,¹¹ previous hospital-based studies have not included melatonin.⁷ It is not known if increased melatonin use in the hospital mitigates use of higher-risk medications. Melatonin preparations can also have quality issues, including deviations from labelled dosage and contamination with compounds such as serotonin,¹² and patients continuing melatonin after discharge could have adverse effects if switched to different preparations.

In this study, we aimed to determine temporal trends in melatonin use in hospitalized patients, and compare them with trends in use of other sleep medications.

We conducted the study at two urban academic hospitals with a total of 706 acute care beds in the same network in Toronto, Canada. This study was approved by the University Health Network’s research ethics board.

We abstracted pharmacy dispensing data on melatonin, zopiclone, and lorazepam from January 1, 2013, to December 31, 2018. We included oral medications dispensed to inpatient units or admitted patients in the emergency department (ED). Zopiclone is the most commonly used nonbenzodiazepine sleep medication in Canada.¹³ Lorazepam is the most commonly used benzodiazepine at our institution (97% of all oral benzodiazepine doses). While lorazepam is prescribed for many nonsleep indications, we included it to assess the impact of melatonin. We did not include antipsychotics or trazodone, which are rarely newly initiated for insomnia at our institution.

We abstracted the monthly number of doses dispensed by unit and hospital. We categorized units based on the primary patient population as either internal medicine, critical care, or other. Admitted patients in the ED were counted as “other” regardless of service. As the focus of our study was on internal medicine and critical care, we did not analyze by type of unit in the “other” group, which is heterogeneous.

Each medication-dispensing event was counted as one dose, regardless of the number or strength of tablets (eg, a patient dispensed two 3-mg tablets of melatonin, for a total of 6 mg, would be counted as a single dose). Most unused doses are credited back (ie, if a medication was refused and returned to pharmacy, it was not counted). Lorazepam and zopiclone are on hospital formulary, while melatonin is not. To order melatonin, clinicians must select “Nonformulary medication” in the electronic health record and manually enter medication name, dose, route, and frequency, as well as select a justification for use. The hospital supplies nonformulary medications such as melatonin to patients.

To account for changes in patient volumes, we standardized medication dispensing rates per 1,000 inpatient days. We discovered rare instances in which the monthly number of doses was a negative number because of pharmacy inventory accounting. This issue affected only 0.13% of observations and the magnitude was small (8 doses or fewer); in these cases, we assumed the number of doses was zero.

We used line charts to visualize changes in medication dispensing over time by medication and hospital. We compared rates of medications use between unit type and hospital with use of relative difference and rate difference. Statistical analysis was performed with R (The R Foundation for Statistical Computing, 2018) using lubridate (2011), dplyr (2018), ggplot2 (2016), fmsb (2019), and forcats (2018).

A total of 1,542,225 inpatient days were analyzed, of which 60.4% were at hospital A. Internal medicine accounted for 23.5% of inpatient days, critical care for 11.7%, and other units for 64.8%.

Overall Trends in Sleep Medication Use

There were 351,131 dispensed doses of study medications (13% melatonin, 43% lorazepam, and 44% zopiclone). Overall use of the three study medications per 1,000 inpatient days increased by 25.7% during the study.

Melatonin use increased by 71.3 doses per 1,000 inpatient days during 2013-2018, while zopiclone use decreased by 20.4 doses per 1,000 inpatient days (Table). Lorazepam use increased slightly by 3.5 doses per 1,000 inpatient days. All rate differences reported in the results are statistically significant (Appendix Table).

Unit Type Comparison

Melatonin use was highest in critical care and internal medicine (50.9 and 48.4 doses per 1,000 inpatient days, respectively), compared with that in other units (19.3 doses per 1,000 inpatient days). Among critical care units, melatonin use was highest in medical-surgical units (67.4 doses per 1,000 inpatient days) and lower in cardiac and cardiovascular surgery units (24.6 and 18.3 doses per 1,000 inpatient days respectively). Zopiclone use was highest in critical care and other units (117.9 and 112.2 doses per 1,000 inpatient days, respectively) and lowest in internal medicine (57.0 doses per 1,000 inpatient days).

Hospital Site Comparison

Overall melatonin use was 65.4% higher at hospital B than at hospital A (42.4 vs 21.5 doses per 1,000 inpatient days; Figure). Zopiclone use was 81.7% lower at hospital B (54.6 vs 130.0 doses per 1,000 inpatient days).

When similar units were compared between hospitals, the trends were similar. For example, among internal medicine units, melatonin use was 66.7% higher at hospital B than at hospital A (64.4 vs 32.3 doses per 1,000 inpatient days).

During this 6-year study period of sleep medication use at two academic hospitals, overall use of melatonin, zopiclone, and lorazepam increased by 25.7%. Melatonin increased from almost no use to more than 70 doses per 1,000 inpatient days. The increase in melatonin was not accompanied by a proportional decline in zopiclone, which only decreased by 20.4 doses per 1,000 inpatient days. Lorazepam use increased slightly. This suggests that melatonin is not simply being substituted for higher-risk sleep medications and is instead being given to patients who might not have received sleep medications otherwise.

There are a few potential explanations for the disproportionate increase in melatonin. Providers may be more liberal in prescribing melatonin for insomnia because of perceived greater safety, compared with other medications. Melatonin may also be prescribed for delirium, despite a lack of high-quality evidence.¹⁴ Interestingly, melatonin use has increased despite a paucity of evidence for its efficacy or safety in hospital.⁶ Considering the additional barriers that exist to ordering melatonin, a nonformulary medication at our institution, the magnitude of increase is even more striking.

Melatonin use was highest on internal medicine and critical care units. This may reflect patient differences (eg, older patients with more comorbid conditions might leave prescribers reluctant to use benzodiazepines), differences in the physical environment (eg, noise/lighting), differences in nursing practices (eg, intensity of monitoring or medication administration), or differences in prescribing.

Melatonin use was almost twice as high at hospital B as it was at hospital A. While the services differ at each hospital, the results were similar when comparing the same unit type (eg, internal medicine). Internal medicine units have similar (though not identical) patient populations and team structures at both hospitals, and residents rotate between hospitals. Attendings and nurses are based primarily at one hospital and their practice patterns might differ. Geriatricians have a stronger presence at hospital B. Higher zopiclone use at hospital A could explain lower melatonin use. Lastly, improvement initiatives may have contributed (eg, one unit at hospital B promoted melatonin in 2017).

Our study has potential limitations. We studied dispensed rather than administered medications; however, numbers of doses dispensed but not administered are expected to be low because most unused doses are accounted for. By studying dispensing data, we might have underestimated the number of prescriptions (eg, if a patient was prescribed but refused a medication, this would not be captured). Our study did not examine medications after hospital discharge, although medications started in a hospital are often continued at discharge.^7,15 Our study could not determine indications for medication prescribing, and melatonin and lorazepam are both used for nonsleep indications. Our study could not differentiate between continuation of home medications and new prescriptions. Finally, the results may not be generalizable to other settings.

In this 6-year study of sleep medication use at two academic hospitals, we found that overall use of melatonin, zopiclone, and lorazepam increased by 25%, predominantly because of markedly increased melatonin use. Given the current lack of high-quality evidence, further research on the use of melatonin in hospitalized patients is needed.

Sleep disturbance is common in hospitals, and both the quality and quantity of sleep are negatively affected in hospitalized patients.¹ Sleep disturbances in hospitals are associated with hyperglycemia,² delirium,³ lower patient satisfaction,⁴ and increased risk of readmission.⁵

A significant proportion of hospitalized patients receive sleep medications (ie, hypnotic medication) despite limited evidence.^6,7 Sleep medications have adverse effects including falls, fractures, cognitive impairment, and delirium.⁸ Commonly used nonbenzodiazepine sleep medications (eg, zopiclone) are perceived as safer but may have similar risks.⁹

Melatonin is increasingly used to treat insomnia, although evidence for its efficacy and safety is lacking.¹⁰ While melatonin use doubled between 2007 and 2012 in the United States,¹¹ previous hospital-based studies have not included melatonin.⁷ It is not known if increased melatonin use in the hospital mitigates use of higher-risk medications. Melatonin preparations can also have quality issues, including deviations from labelled dosage and contamination with compounds such as serotonin,¹² and patients continuing melatonin after discharge could have adverse effects if switched to different preparations.

In this study, we aimed to determine temporal trends in melatonin use in hospitalized patients, and compare them with trends in use of other sleep medications.

We conducted the study at two urban academic hospitals with a total of 706 acute care beds in the same network in Toronto, Canada. This study was approved by the University Health Network’s research ethics board.

We abstracted pharmacy dispensing data on melatonin, zopiclone, and lorazepam from January 1, 2013, to December 31, 2018. We included oral medications dispensed to inpatient units or admitted patients in the emergency department (ED). Zopiclone is the most commonly used nonbenzodiazepine sleep medication in Canada.¹³ Lorazepam is the most commonly used benzodiazepine at our institution (97% of all oral benzodiazepine doses). While lorazepam is prescribed for many nonsleep indications, we included it to assess the impact of melatonin. We did not include antipsychotics or trazodone, which are rarely newly initiated for insomnia at our institution.

We abstracted the monthly number of doses dispensed by unit and hospital. We categorized units based on the primary patient population as either internal medicine, critical care, or other. Admitted patients in the ED were counted as “other” regardless of service. As the focus of our study was on internal medicine and critical care, we did not analyze by type of unit in the “other” group, which is heterogeneous.

Each medication-dispensing event was counted as one dose, regardless of the number or strength of tablets (eg, a patient dispensed two 3-mg tablets of melatonin, for a total of 6 mg, would be counted as a single dose). Most unused doses are credited back (ie, if a medication was refused and returned to pharmacy, it was not counted). Lorazepam and zopiclone are on hospital formulary, while melatonin is not. To order melatonin, clinicians must select “Nonformulary medication” in the electronic health record and manually enter medication name, dose, route, and frequency, as well as select a justification for use. The hospital supplies nonformulary medications such as melatonin to patients.

To account for changes in patient volumes, we standardized medication dispensing rates per 1,000 inpatient days. We discovered rare instances in which the monthly number of doses was a negative number because of pharmacy inventory accounting. This issue affected only 0.13% of observations and the magnitude was small (8 doses or fewer); in these cases, we assumed the number of doses was zero.

We used line charts to visualize changes in medication dispensing over time by medication and hospital. We compared rates of medications use between unit type and hospital with use of relative difference and rate difference. Statistical analysis was performed with R (The R Foundation for Statistical Computing, 2018) using lubridate (2011), dplyr (2018), ggplot2 (2016), fmsb (2019), and forcats (2018).

A total of 1,542,225 inpatient days were analyzed, of which 60.4% were at hospital A. Internal medicine accounted for 23.5% of inpatient days, critical care for 11.7%, and other units for 64.8%.

Overall Trends in Sleep Medication Use

There were 351,131 dispensed doses of study medications (13% melatonin, 43% lorazepam, and 44% zopiclone). Overall use of the three study medications per 1,000 inpatient days increased by 25.7% during the study.

Melatonin use increased by 71.3 doses per 1,000 inpatient days during 2013-2018, while zopiclone use decreased by 20.4 doses per 1,000 inpatient days (Table). Lorazepam use increased slightly by 3.5 doses per 1,000 inpatient days. All rate differences reported in the results are statistically significant (Appendix Table).

Unit Type Comparison

Melatonin use was highest in critical care and internal medicine (50.9 and 48.4 doses per 1,000 inpatient days, respectively), compared with that in other units (19.3 doses per 1,000 inpatient days). Among critical care units, melatonin use was highest in medical-surgical units (67.4 doses per 1,000 inpatient days) and lower in cardiac and cardiovascular surgery units (24.6 and 18.3 doses per 1,000 inpatient days respectively). Zopiclone use was highest in critical care and other units (117.9 and 112.2 doses per 1,000 inpatient days, respectively) and lowest in internal medicine (57.0 doses per 1,000 inpatient days).

Hospital Site Comparison

Overall melatonin use was 65.4% higher at hospital B than at hospital A (42.4 vs 21.5 doses per 1,000 inpatient days; Figure). Zopiclone use was 81.7% lower at hospital B (54.6 vs 130.0 doses per 1,000 inpatient days).

When similar units were compared between hospitals, the trends were similar. For example, among internal medicine units, melatonin use was 66.7% higher at hospital B than at hospital A (64.4 vs 32.3 doses per 1,000 inpatient days).

During this 6-year study period of sleep medication use at two academic hospitals, overall use of melatonin, zopiclone, and lorazepam increased by 25.7%. Melatonin increased from almost no use to more than 70 doses per 1,000 inpatient days. The increase in melatonin was not accompanied by a proportional decline in zopiclone, which only decreased by 20.4 doses per 1,000 inpatient days. Lorazepam use increased slightly. This suggests that melatonin is not simply being substituted for higher-risk sleep medications and is instead being given to patients who might not have received sleep medications otherwise.

There are a few potential explanations for the disproportionate increase in melatonin. Providers may be more liberal in prescribing melatonin for insomnia because of perceived greater safety, compared with other medications. Melatonin may also be prescribed for delirium, despite a lack of high-quality evidence.¹⁴ Interestingly, melatonin use has increased despite a paucity of evidence for its efficacy or safety in hospital.⁶ Considering the additional barriers that exist to ordering melatonin, a nonformulary medication at our institution, the magnitude of increase is even more striking.

Melatonin use was highest on internal medicine and critical care units. This may reflect patient differences (eg, older patients with more comorbid conditions might leave prescribers reluctant to use benzodiazepines), differences in the physical environment (eg, noise/lighting), differences in nursing practices (eg, intensity of monitoring or medication administration), or differences in prescribing.

Melatonin use was almost twice as high at hospital B as it was at hospital A. While the services differ at each hospital, the results were similar when comparing the same unit type (eg, internal medicine). Internal medicine units have similar (though not identical) patient populations and team structures at both hospitals, and residents rotate between hospitals. Attendings and nurses are based primarily at one hospital and their practice patterns might differ. Geriatricians have a stronger presence at hospital B. Higher zopiclone use at hospital A could explain lower melatonin use. Lastly, improvement initiatives may have contributed (eg, one unit at hospital B promoted melatonin in 2017).

Our study has potential limitations. We studied dispensed rather than administered medications; however, numbers of doses dispensed but not administered are expected to be low because most unused doses are accounted for. By studying dispensing data, we might have underestimated the number of prescriptions (eg, if a patient was prescribed but refused a medication, this would not be captured). Our study did not examine medications after hospital discharge, although medications started in a hospital are often continued at discharge.^7,15 Our study could not determine indications for medication prescribing, and melatonin and lorazepam are both used for nonsleep indications. Our study could not differentiate between continuation of home medications and new prescriptions. Finally, the results may not be generalizable to other settings.

In this 6-year study of sleep medication use at two academic hospitals, we found that overall use of melatonin, zopiclone, and lorazepam increased by 25%, predominantly because of markedly increased melatonin use. Given the current lack of high-quality evidence, further research on the use of melatonin in hospitalized patients is needed.

“Knowing yourself is the beginning of all wisdom.”—Aristotle

On the wards, your white coat and stethoscope signal your role as a healthcare provider. These external symbols of your work represent your expertise, experience, and commitment to service, and your patients look to these signals for comfort and reassurance. But when you are running a meeting, managing projects, or leading people in your organization, how do others know what you have to offer? Although signaling your values, skills, and intentions is as important in leadership as it is in the clinical setting, few clinicians spend time reflecting on how best to do this. Crafting a strong, consistent personal leadership brand can help.

As described by Norm Smallwood and Dave Ulrich, a personal leadership brand is the external projection of your strengths and interests, which demonstrates how you create value for others.^1,2 In other words, a personal leadership brand helps constituents, stakeholders, and potential partners understand what you offer as a leader. Having a brand keeps you on track as a leader and helps get you noticed for future opportunities by helping you shape and meet expectations in a way that is deliberate, dynamic, and authentic.

Building your personal leadership brand is an exercise in reflection. Leaders should challenge themselves to answer the following questions:

• What do I have to offer, and what do others appreciate about me?
• What are my values?
• Where am I trying to go?
• How does my path align with organizational goals?

The answers to these simple questions can help you create your personal leadership brand. First, reflect on what you want to be known for, your values, and how you are currently perceived. Then, identify the results you are aiming to produce, aligning them with your strengths and organizational goals. Write these down, and share your reflections with trusted peers and your mentoring team. Shape your thoughts into a personal vision statement with a focus on what you put out into the world to help you stay true to yourself while producing the desired results. For example, a vision statement for a gifted communicator with a background in quality improvement may be: “I will use my strong communication skills to address complex problems impacting our hospital to reduce cost and improve quality with the goal of building a career as a health system leader.” Finally, be authentic, and share your personal brand in an articulate and succinct way to help others understand your place in the structure and narrative of an organization.

Your personal leadership brand should not be static; rather, it is a process that should iterate over time. Ask for direct feedback from trusted advisers and allies at regular intervals. Investigate whether your organization offers a formal structure, such as a “360 Evaluation,” to get perspective on how your unique strengths, skills, and goals are perceived. Then, explore and clarify discrepancies between where you think you are and how others see you. Approaching these conversations with humility will keep you aligned with your values, which makes it easier for others to be invested in your development.

A strong personal leadership brand is a force multiplier, providing clarity within teams and helping align a leader’s assets and values with organizational goals. It is a solid external signal of what others can expect from your work and will help you focus on your strengths while identifying areas for growth. A personal leadership brand is formed through reflection and, at its core, its authenticity. In the words of Paracelsus, a Renaissance physician, astrologer, and alchemist, “Be not another, if you can be yourself.”³

“Knowing yourself is the beginning of all wisdom.”—Aristotle

On the wards, your white coat and stethoscope signal your role as a healthcare provider. These external symbols of your work represent your expertise, experience, and commitment to service, and your patients look to these signals for comfort and reassurance. But when you are running a meeting, managing projects, or leading people in your organization, how do others know what you have to offer? Although signaling your values, skills, and intentions is as important in leadership as it is in the clinical setting, few clinicians spend time reflecting on how best to do this. Crafting a strong, consistent personal leadership brand can help.

As described by Norm Smallwood and Dave Ulrich, a personal leadership brand is the external projection of your strengths and interests, which demonstrates how you create value for others.^1,2 In other words, a personal leadership brand helps constituents, stakeholders, and potential partners understand what you offer as a leader. Having a brand keeps you on track as a leader and helps get you noticed for future opportunities by helping you shape and meet expectations in a way that is deliberate, dynamic, and authentic.

Building your personal leadership brand is an exercise in reflection. Leaders should challenge themselves to answer the following questions:

• What do I have to offer, and what do others appreciate about me?
• What are my values?
• Where am I trying to go?
• How does my path align with organizational goals?

The answers to these simple questions can help you create your personal leadership brand. First, reflect on what you want to be known for, your values, and how you are currently perceived. Then, identify the results you are aiming to produce, aligning them with your strengths and organizational goals. Write these down, and share your reflections with trusted peers and your mentoring team. Shape your thoughts into a personal vision statement with a focus on what you put out into the world to help you stay true to yourself while producing the desired results. For example, a vision statement for a gifted communicator with a background in quality improvement may be: “I will use my strong communication skills to address complex problems impacting our hospital to reduce cost and improve quality with the goal of building a career as a health system leader.” Finally, be authentic, and share your personal brand in an articulate and succinct way to help others understand your place in the structure and narrative of an organization.

Your personal leadership brand should not be static; rather, it is a process that should iterate over time. Ask for direct feedback from trusted advisers and allies at regular intervals. Investigate whether your organization offers a formal structure, such as a “360 Evaluation,” to get perspective on how your unique strengths, skills, and goals are perceived. Then, explore and clarify discrepancies between where you think you are and how others see you. Approaching these conversations with humility will keep you aligned with your values, which makes it easier for others to be invested in your development.

A strong personal leadership brand is a force multiplier, providing clarity within teams and helping align a leader’s assets and values with organizational goals. It is a solid external signal of what others can expect from your work and will help you focus on your strengths while identifying areas for growth. A personal leadership brand is formed through reflection and, at its core, its authenticity. In the words of Paracelsus, a Renaissance physician, astrologer, and alchemist, “Be not another, if you can be yourself.”³

“Knowing yourself is the beginning of all wisdom.”—Aristotle

On the wards, your white coat and stethoscope signal your role as a healthcare provider. These external symbols of your work represent your expertise, experience, and commitment to service, and your patients look to these signals for comfort and reassurance. But when you are running a meeting, managing projects, or leading people in your organization, how do others know what you have to offer? Although signaling your values, skills, and intentions is as important in leadership as it is in the clinical setting, few clinicians spend time reflecting on how best to do this. Crafting a strong, consistent personal leadership brand can help.

As described by Norm Smallwood and Dave Ulrich, a personal leadership brand is the external projection of your strengths and interests, which demonstrates how you create value for others.^1,2 In other words, a personal leadership brand helps constituents, stakeholders, and potential partners understand what you offer as a leader. Having a brand keeps you on track as a leader and helps get you noticed for future opportunities by helping you shape and meet expectations in a way that is deliberate, dynamic, and authentic.

Building your personal leadership brand is an exercise in reflection. Leaders should challenge themselves to answer the following questions:

• What do I have to offer, and what do others appreciate about me?
• What are my values?
• Where am I trying to go?
• How does my path align with organizational goals?

The answers to these simple questions can help you create your personal leadership brand. First, reflect on what you want to be known for, your values, and how you are currently perceived. Then, identify the results you are aiming to produce, aligning them with your strengths and organizational goals. Write these down, and share your reflections with trusted peers and your mentoring team. Shape your thoughts into a personal vision statement with a focus on what you put out into the world to help you stay true to yourself while producing the desired results. For example, a vision statement for a gifted communicator with a background in quality improvement may be: “I will use my strong communication skills to address complex problems impacting our hospital to reduce cost and improve quality with the goal of building a career as a health system leader.” Finally, be authentic, and share your personal brand in an articulate and succinct way to help others understand your place in the structure and narrative of an organization.

Your personal leadership brand should not be static; rather, it is a process that should iterate over time. Ask for direct feedback from trusted advisers and allies at regular intervals. Investigate whether your organization offers a formal structure, such as a “360 Evaluation,” to get perspective on how your unique strengths, skills, and goals are perceived. Then, explore and clarify discrepancies between where you think you are and how others see you. Approaching these conversations with humility will keep you aligned with your values, which makes it easier for others to be invested in your development.

A strong personal leadership brand is a force multiplier, providing clarity within teams and helping align a leader’s assets and values with organizational goals. It is a solid external signal of what others can expect from your work and will help you focus on your strengths while identifying areas for growth. A personal leadership brand is formed through reflection and, at its core, its authenticity. In the words of Paracelsus, a Renaissance physician, astrologer, and alchemist, “Be not another, if you can be yourself.”³

COVID-19, the disease caused by the novel coronavirus SARS-CoV-2, was declared a pandemic on March 11, 2020. Although most patients (81%) develop mild illness, 14% develop severe illness, and 5% develop critical illness, including acute respiratory failure, septic shock, and multiorgan dysfunction.¹

Point-of-care ultrasound (POCUS), or bedside ultrasound performed by a clinician caring for the patient, is being used to support the diagnosis and serially monitor patients with COVID-19. We performed a literature search of electronically discoverable peer-reviewed publications on POCUS use in COVID-19 from December 1, 2019, to April 10, 2020. We review key POCUS applications that are most relevant to frontline providers in the care of COVID-19 patients.

Diagnosing COVID-19 disease by polymerase chain reaction is limited by availability of testing, delays in test positivity (mean 5.1 days), and high false-negative rate early in the course of the disease (sensitivity 81%).² Chest computed tomography (CT) scans are often requested during the initial evaluation of suspected COVID-19, but the American College of Radiology has recommend against the routine use of CT scans for diagnosing COVID-19.³

The diagnostic accuracy of lung ultrasound (LUS) has been shown to be similar to chest CT scans in patients presenting with respiratory complaints, such as dyspnea and hypoxemia, caused by non–COVID-19 pneumonia (sensitivity, 85%; specificity, 93%).⁴ Normal LUS findings correlate well with CT chest scans showing absence of typical ground glass opacities. This negative predictive value is very important.⁵ However, early in the course of COVID-19, similar to CT scans, LUS may be normal during the first 5 days or in patients with mild disease.² Unique advantages of LUS in COVID-19 include immediate availability of results, repeatability over time, and performance at the bedside, which avoids transportation of patients to radiology suites and disinfection of large imaging equipment.

LUS findings in COVID-19 include (a) an irregular, thickened pleural line, (b) B-lines in various patterns (discrete and confluent), (c) small subpleural consolidations, and (d) absence of pleural effusions (Figure). Bilateral, multifocal disease is common, while lobar alveolar consolidation is less common.^6,7 In addition to supporting the initial diagnosis, LUS is being used to serially monitor hospitalized COVID-19 patients. As lung interstitial fluid content increases, discrete B-lines become confluent, and the number of affected lung zones increases, which can guide decisions about escalation of care. LUS is often used to guide decisions about prone ventilation, extracorporeal membrane oxygenation, and weaning from mechanical ventilation in acute respiratory failure of non–COVID-19 patients,⁸ and these concepts are being applied to COVID-19 patients. During recovery, reappearance of A-lines can be seen, but normalization of the LUS pattern is gradual over several weeks based on our experience and one report.⁹ Multiple LUS protocols examining 6 to 12 lung zones have been published prior to the COVID-19 pandemic. We recommend continuing to use an institutional protocol and evaluating at least one to two rib interspaces on the anterior, lateral, and posterior chest wall.

Myriad cardiac complications have been described in COVID-19 – including acute coronary syndrome, myocarditis, cardiomyopathy with heart failure, and arrhythmias – secondary to increased cardiac stress from hypoxia, direct myocardial infection, or indirect injury from a hyperinflammatory response. Mortality is higher in patients with hypertension, diabetes, and coronary artery disease.^10,11 Cardiac POCUS is being used to evaluate COVID-19 patients when troponin and B-type natriuretic peptide (BNP) are elevated or when there are hemodynamic or electrocardiogram changes. Given the high incidence of venous thromboembolism (VTE) in COVID-19,¹² cardiac POCUS is being used to rapidly assess for right ventricular (RV) dysfunction and acute pulmonary hypertension.

The American Society of Echocardiography has recommended the use of cardiac POCUS by frontline providers for detection or characterization of preexisting cardiovascular disease, early identification of worsening cardiac function, serial monitoring and examination, and elucidation of cardiovascular pathologies associated with COVID-19.¹³ Sharing cardiac POCUS images in real time or through an image archive can reduce the need for consultative echocardiography, which ultimately reduces staff exposure, conserves personal protective equipment, and reduces need for decontamination of echocardiographic equipment.

The minimum cardiac POCUS views recommended in COVID-19 patients include the parasternal long-axis and short-axis views (midventricular level), either the apical or subcostal four-chamber view, and the subcostal long-axis view of the inferior vena cava.¹³ The goal of a cardiac POCUS exam is to qualitatively assess left ventricular (LV) systolic function, RV size and contractility, gross valvular and regional wall motion abnormalities, and pericardial effusion. In prone position ventilation, the swimmer’s position with one arm elevated above the shoulder may permit acquisition of apical views. Finally, integrated cardiopulmonary ultrasonography, including evaluation for deep vein thrombosis (DVT; see below), is ideal for proper characterization of underlying LV and RV function, volume status, and titration of vasopressor and inotropic support.

COVID-19 has been associated with a proinflammatory and hypercoagulable state with elevated d-dimer and higher-than-­expected incidence of VTE (27%) in critically ill patients.^12,14 Previous studies have demonstrated that frontline providers, including hospitalists, can detect lower extremity (LE) DVTs with high diagnostic accuracy using POCUS.¹⁵ Given the high incidence of DVTs despite prophylactic anticoagulation, some reports have suggested screening or serially monitoring for LE DVT in hospitalized COVID-19 patients.¹⁶ In patients with suspected pulmonary embolism (PE), POCUS can rapidly detect venous thrombosis that justifies prompt initiation of anticoagulation (eg, finding DVT or clot-in-transit), supportive findings of PE (eg, acute RV dysfunction, pulmonary infarcts), or alternative diagnoses (eg, bacterial pneumonia). However, it is important to recognize POCUS cannot definitively rule out PE. Additionally, subpleural consolidations are common in COVID-19 patients and could be caused by either infection or infarction. The American Society of Hematology has endorsed the use of POCUS, LE compression ultrasonography, and echocardiography in COVID-19 patients with suspected PE when availability of CT pulmonary angiography or ventilation-perfusion lung scans is limited.¹⁴

A POCUS exam for LE DVT consists of two-dimensional venous compression alone and yields results similar to formal vascular studies in both critically ill and noncritically ill patients. Because proximal LE thrombi have the highest risk of embolization, evaluation of the common femoral vein, femoral vein, and popliteal vein is most important.¹⁵ Either inability to compress a vein completely with wall-to-wall apposition or visualization of echogenic thrombus within the vein is diagnostic of DVT. Acute thrombi are gelatinous and may appear anechoic, while subacute or chronic thrombi are echogenic, but all veins with a DVT will not compress completely.

Ultrasound guidance for central venous catheter (CVC) insertion has been shown to increase procedure success rates and decrease mechanical complications, primarily arterial puncture and pneumothorax. Similarly, higher success rates and fewer insertion attempts have been observed with ultrasound-guided peripheral intravenous line and arterial line placement.¹⁷ Ultrasound-­guided PIV placement can reduce referrals for midlines and peripherally inserted central catheters in hospitalized patients.¹⁸

In COVID-19 patients, use of ultrasound guidance for vascular access has distinct advantages. First, given the high incidence of DVT in COVID-19 patients,¹² POCUS allows preprocedural evaluation of the target vessel for thrombosis, as well as anatomic variations and stenosis. Second, visualizing the needle tip and guidewire within the target vein prior to dilation nearly eliminates the risk of arterial puncture and inadvertent arterial dilation, which is particularly important in COVID-19 patients receiving high-dose prophylactic or therapeutic anticoagulation. Third, when inserting internal jugular and subclavian CVCs, visualization of normal lung sliding before and after the procedure safely rules out pneumothorax. However, if lung sliding is not seen before the procedure, it cannot be used to rule out pneumothorax afterward. Additionally, visualizing absence of the catheter tip in the right atrium and presence of a rapid atrial swirl sign within 2 seconds of briskly injecting 10 mL of saline confirms catheter tip placement near the superior vena cava/right atrial junction, which can eliminate the need for a postprocedure chest radiograph.¹⁷

POCUS can be used to rapidly confirm endotracheal tube (ETT) placement, which can reduce reliance on postintubation chest radiographs. A meta-analysis of prospective and randomized trials showed transtracheal ultrasonography had high sensitivity (98.7%) and specificity (97.1%) for confirming tracheal placement of ETTs.¹⁹ Confirming endotracheal intubation involves two steps: First, a linear transducer is placed transversely over the suprasternal notch to visualize the ETT passing through the trachea, and not the esophagus, during insertion. Second, after the ETT cuff has been inflated, bilateral lung sliding should be seen in sync with the respiratory cycle if the ETT is in the trachea. Absent lung sliding, but preserved lung pulse, on the anterior hemithorax is likely caused by main stem bronchial intubation, and withdrawing the ETT until bilateral lung sliding is seen confirms tracheal placement. Additionally, the following steps are recommended to reduce the risk of exposure to healthcare workers: minimizing use of bag-valve-mask ventilation, performing rapid sequence intubation using video laryngoscopy, and connecting the ETT to the ventilator immediately.

Important considerations when selecting an ultrasound machine for use in COVID-19 patients include image quality, portability, functionality, and ease of disinfection. Advantages of handheld devices include portability and ease of disinfection, whereas cart-based systems generally have better image quality and functionality. To minimize the risk of cross contamination, an ultrasound machine should be dedicated exclusively for use on patients with confirmed COVID-19 and not shared with patients with suspected COVID-19.²⁰ To minimize exposure to COVID-19 patients, frontline providers should perform POCUS exams only when findings may change management, and timing of the exam and views acquired should be selected deliberately.

Ultrasound machine disinfection should be integrated into routine donning and doffing procedures. When possible, both handheld and cart-based machines should be draped with protective covers during aerosol-generating procedures. Single use ultrasound gel packets are recommended in order to decrease the risk of nosocomial infection.²⁰ After every use of an ultrasound machine on intact skin or for percutaneous procedures, low-level disinfection should be performed with an Environmental Protection Agency–recommended product that is effective against coronavirus.

Some ultrasound manufacturers have added teleultrasound software that allows remote training of novice POCUS users and remote guidance in actual patient care. Teleultrasound can be utilized to share images in real time with consultants or expert providers.

POCUS is uniquely poised to improve patient care during the COVID-19 pandemic. POCUS can be used to support the diagnosis of COVID-19 patients and monitor patients with confirmed disease. Common POCUS applications used in COVID-19 patients include evaluation of the lungs, heart, and deep veins, as well as performance of bedside procedures. Ultrasound machine portability and disinfection are important considerations in COVID-19 patients.

COVID-19, the disease caused by the novel coronavirus SARS-CoV-2, was declared a pandemic on March 11, 2020. Although most patients (81%) develop mild illness, 14% develop severe illness, and 5% develop critical illness, including acute respiratory failure, septic shock, and multiorgan dysfunction.¹

Point-of-care ultrasound (POCUS), or bedside ultrasound performed by a clinician caring for the patient, is being used to support the diagnosis and serially monitor patients with COVID-19. We performed a literature search of electronically discoverable peer-reviewed publications on POCUS use in COVID-19 from December 1, 2019, to April 10, 2020. We review key POCUS applications that are most relevant to frontline providers in the care of COVID-19 patients.

Diagnosing COVID-19 disease by polymerase chain reaction is limited by availability of testing, delays in test positivity (mean 5.1 days), and high false-negative rate early in the course of the disease (sensitivity 81%).² Chest computed tomography (CT) scans are often requested during the initial evaluation of suspected COVID-19, but the American College of Radiology has recommend against the routine use of CT scans for diagnosing COVID-19.³

The diagnostic accuracy of lung ultrasound (LUS) has been shown to be similar to chest CT scans in patients presenting with respiratory complaints, such as dyspnea and hypoxemia, caused by non–COVID-19 pneumonia (sensitivity, 85%; specificity, 93%).⁴ Normal LUS findings correlate well with CT chest scans showing absence of typical ground glass opacities. This negative predictive value is very important.⁵ However, early in the course of COVID-19, similar to CT scans, LUS may be normal during the first 5 days or in patients with mild disease.² Unique advantages of LUS in COVID-19 include immediate availability of results, repeatability over time, and performance at the bedside, which avoids transportation of patients to radiology suites and disinfection of large imaging equipment.

LUS findings in COVID-19 include (a) an irregular, thickened pleural line, (b) B-lines in various patterns (discrete and confluent), (c) small subpleural consolidations, and (d) absence of pleural effusions (Figure). Bilateral, multifocal disease is common, while lobar alveolar consolidation is less common.^6,7 In addition to supporting the initial diagnosis, LUS is being used to serially monitor hospitalized COVID-19 patients. As lung interstitial fluid content increases, discrete B-lines become confluent, and the number of affected lung zones increases, which can guide decisions about escalation of care. LUS is often used to guide decisions about prone ventilation, extracorporeal membrane oxygenation, and weaning from mechanical ventilation in acute respiratory failure of non–COVID-19 patients,⁸ and these concepts are being applied to COVID-19 patients. During recovery, reappearance of A-lines can be seen, but normalization of the LUS pattern is gradual over several weeks based on our experience and one report.⁹ Multiple LUS protocols examining 6 to 12 lung zones have been published prior to the COVID-19 pandemic. We recommend continuing to use an institutional protocol and evaluating at least one to two rib interspaces on the anterior, lateral, and posterior chest wall.

Myriad cardiac complications have been described in COVID-19 – including acute coronary syndrome, myocarditis, cardiomyopathy with heart failure, and arrhythmias – secondary to increased cardiac stress from hypoxia, direct myocardial infection, or indirect injury from a hyperinflammatory response. Mortality is higher in patients with hypertension, diabetes, and coronary artery disease.^10,11 Cardiac POCUS is being used to evaluate COVID-19 patients when troponin and B-type natriuretic peptide (BNP) are elevated or when there are hemodynamic or electrocardiogram changes. Given the high incidence of venous thromboembolism (VTE) in COVID-19,¹² cardiac POCUS is being used to rapidly assess for right ventricular (RV) dysfunction and acute pulmonary hypertension.

The American Society of Echocardiography has recommended the use of cardiac POCUS by frontline providers for detection or characterization of preexisting cardiovascular disease, early identification of worsening cardiac function, serial monitoring and examination, and elucidation of cardiovascular pathologies associated with COVID-19.¹³ Sharing cardiac POCUS images in real time or through an image archive can reduce the need for consultative echocardiography, which ultimately reduces staff exposure, conserves personal protective equipment, and reduces need for decontamination of echocardiographic equipment.

The minimum cardiac POCUS views recommended in COVID-19 patients include the parasternal long-axis and short-axis views (midventricular level), either the apical or subcostal four-chamber view, and the subcostal long-axis view of the inferior vena cava.¹³ The goal of a cardiac POCUS exam is to qualitatively assess left ventricular (LV) systolic function, RV size and contractility, gross valvular and regional wall motion abnormalities, and pericardial effusion. In prone position ventilation, the swimmer’s position with one arm elevated above the shoulder may permit acquisition of apical views. Finally, integrated cardiopulmonary ultrasonography, including evaluation for deep vein thrombosis (DVT; see below), is ideal for proper characterization of underlying LV and RV function, volume status, and titration of vasopressor and inotropic support.

COVID-19 has been associated with a proinflammatory and hypercoagulable state with elevated d-dimer and higher-than-­expected incidence of VTE (27%) in critically ill patients.^12,14 Previous studies have demonstrated that frontline providers, including hospitalists, can detect lower extremity (LE) DVTs with high diagnostic accuracy using POCUS.¹⁵ Given the high incidence of DVTs despite prophylactic anticoagulation, some reports have suggested screening or serially monitoring for LE DVT in hospitalized COVID-19 patients.¹⁶ In patients with suspected pulmonary embolism (PE), POCUS can rapidly detect venous thrombosis that justifies prompt initiation of anticoagulation (eg, finding DVT or clot-in-transit), supportive findings of PE (eg, acute RV dysfunction, pulmonary infarcts), or alternative diagnoses (eg, bacterial pneumonia). However, it is important to recognize POCUS cannot definitively rule out PE. Additionally, subpleural consolidations are common in COVID-19 patients and could be caused by either infection or infarction. The American Society of Hematology has endorsed the use of POCUS, LE compression ultrasonography, and echocardiography in COVID-19 patients with suspected PE when availability of CT pulmonary angiography or ventilation-perfusion lung scans is limited.¹⁴

A POCUS exam for LE DVT consists of two-dimensional venous compression alone and yields results similar to formal vascular studies in both critically ill and noncritically ill patients. Because proximal LE thrombi have the highest risk of embolization, evaluation of the common femoral vein, femoral vein, and popliteal vein is most important.¹⁵ Either inability to compress a vein completely with wall-to-wall apposition or visualization of echogenic thrombus within the vein is diagnostic of DVT. Acute thrombi are gelatinous and may appear anechoic, while subacute or chronic thrombi are echogenic, but all veins with a DVT will not compress completely.

Ultrasound guidance for central venous catheter (CVC) insertion has been shown to increase procedure success rates and decrease mechanical complications, primarily arterial puncture and pneumothorax. Similarly, higher success rates and fewer insertion attempts have been observed with ultrasound-guided peripheral intravenous line and arterial line placement.¹⁷ Ultrasound-­guided PIV placement can reduce referrals for midlines and peripherally inserted central catheters in hospitalized patients.¹⁸

In COVID-19 patients, use of ultrasound guidance for vascular access has distinct advantages. First, given the high incidence of DVT in COVID-19 patients,¹² POCUS allows preprocedural evaluation of the target vessel for thrombosis, as well as anatomic variations and stenosis. Second, visualizing the needle tip and guidewire within the target vein prior to dilation nearly eliminates the risk of arterial puncture and inadvertent arterial dilation, which is particularly important in COVID-19 patients receiving high-dose prophylactic or therapeutic anticoagulation. Third, when inserting internal jugular and subclavian CVCs, visualization of normal lung sliding before and after the procedure safely rules out pneumothorax. However, if lung sliding is not seen before the procedure, it cannot be used to rule out pneumothorax afterward. Additionally, visualizing absence of the catheter tip in the right atrium and presence of a rapid atrial swirl sign within 2 seconds of briskly injecting 10 mL of saline confirms catheter tip placement near the superior vena cava/right atrial junction, which can eliminate the need for a postprocedure chest radiograph.¹⁷

POCUS can be used to rapidly confirm endotracheal tube (ETT) placement, which can reduce reliance on postintubation chest radiographs. A meta-analysis of prospective and randomized trials showed transtracheal ultrasonography had high sensitivity (98.7%) and specificity (97.1%) for confirming tracheal placement of ETTs.¹⁹ Confirming endotracheal intubation involves two steps: First, a linear transducer is placed transversely over the suprasternal notch to visualize the ETT passing through the trachea, and not the esophagus, during insertion. Second, after the ETT cuff has been inflated, bilateral lung sliding should be seen in sync with the respiratory cycle if the ETT is in the trachea. Absent lung sliding, but preserved lung pulse, on the anterior hemithorax is likely caused by main stem bronchial intubation, and withdrawing the ETT until bilateral lung sliding is seen confirms tracheal placement. Additionally, the following steps are recommended to reduce the risk of exposure to healthcare workers: minimizing use of bag-valve-mask ventilation, performing rapid sequence intubation using video laryngoscopy, and connecting the ETT to the ventilator immediately.

Important considerations when selecting an ultrasound machine for use in COVID-19 patients include image quality, portability, functionality, and ease of disinfection. Advantages of handheld devices include portability and ease of disinfection, whereas cart-based systems generally have better image quality and functionality. To minimize the risk of cross contamination, an ultrasound machine should be dedicated exclusively for use on patients with confirmed COVID-19 and not shared with patients with suspected COVID-19.²⁰ To minimize exposure to COVID-19 patients, frontline providers should perform POCUS exams only when findings may change management, and timing of the exam and views acquired should be selected deliberately.

Ultrasound machine disinfection should be integrated into routine donning and doffing procedures. When possible, both handheld and cart-based machines should be draped with protective covers during aerosol-generating procedures. Single use ultrasound gel packets are recommended in order to decrease the risk of nosocomial infection.²⁰ After every use of an ultrasound machine on intact skin or for percutaneous procedures, low-level disinfection should be performed with an Environmental Protection Agency–recommended product that is effective against coronavirus.

Some ultrasound manufacturers have added teleultrasound software that allows remote training of novice POCUS users and remote guidance in actual patient care. Teleultrasound can be utilized to share images in real time with consultants or expert providers.

POCUS is uniquely poised to improve patient care during the COVID-19 pandemic. POCUS can be used to support the diagnosis of COVID-19 patients and monitor patients with confirmed disease. Common POCUS applications used in COVID-19 patients include evaluation of the lungs, heart, and deep veins, as well as performance of bedside procedures. Ultrasound machine portability and disinfection are important considerations in COVID-19 patients.

COVID-19, the disease caused by the novel coronavirus SARS-CoV-2, was declared a pandemic on March 11, 2020. Although most patients (81%) develop mild illness, 14% develop severe illness, and 5% develop critical illness, including acute respiratory failure, septic shock, and multiorgan dysfunction.¹

Point-of-care ultrasound (POCUS), or bedside ultrasound performed by a clinician caring for the patient, is being used to support the diagnosis and serially monitor patients with COVID-19. We performed a literature search of electronically discoverable peer-reviewed publications on POCUS use in COVID-19 from December 1, 2019, to April 10, 2020. We review key POCUS applications that are most relevant to frontline providers in the care of COVID-19 patients.

Diagnosing COVID-19 disease by polymerase chain reaction is limited by availability of testing, delays in test positivity (mean 5.1 days), and high false-negative rate early in the course of the disease (sensitivity 81%).² Chest computed tomography (CT) scans are often requested during the initial evaluation of suspected COVID-19, but the American College of Radiology has recommend against the routine use of CT scans for diagnosing COVID-19.³

The diagnostic accuracy of lung ultrasound (LUS) has been shown to be similar to chest CT scans in patients presenting with respiratory complaints, such as dyspnea and hypoxemia, caused by non–COVID-19 pneumonia (sensitivity, 85%; specificity, 93%).⁴ Normal LUS findings correlate well with CT chest scans showing absence of typical ground glass opacities. This negative predictive value is very important.⁵ However, early in the course of COVID-19, similar to CT scans, LUS may be normal during the first 5 days or in patients with mild disease.² Unique advantages of LUS in COVID-19 include immediate availability of results, repeatability over time, and performance at the bedside, which avoids transportation of patients to radiology suites and disinfection of large imaging equipment.

LUS findings in COVID-19 include (a) an irregular, thickened pleural line, (b) B-lines in various patterns (discrete and confluent), (c) small subpleural consolidations, and (d) absence of pleural effusions (Figure). Bilateral, multifocal disease is common, while lobar alveolar consolidation is less common.^6,7 In addition to supporting the initial diagnosis, LUS is being used to serially monitor hospitalized COVID-19 patients. As lung interstitial fluid content increases, discrete B-lines become confluent, and the number of affected lung zones increases, which can guide decisions about escalation of care. LUS is often used to guide decisions about prone ventilation, extracorporeal membrane oxygenation, and weaning from mechanical ventilation in acute respiratory failure of non–COVID-19 patients,⁸ and these concepts are being applied to COVID-19 patients. During recovery, reappearance of A-lines can be seen, but normalization of the LUS pattern is gradual over several weeks based on our experience and one report.⁹ Multiple LUS protocols examining 6 to 12 lung zones have been published prior to the COVID-19 pandemic. We recommend continuing to use an institutional protocol and evaluating at least one to two rib interspaces on the anterior, lateral, and posterior chest wall.

Myriad cardiac complications have been described in COVID-19 – including acute coronary syndrome, myocarditis, cardiomyopathy with heart failure, and arrhythmias – secondary to increased cardiac stress from hypoxia, direct myocardial infection, or indirect injury from a hyperinflammatory response. Mortality is higher in patients with hypertension, diabetes, and coronary artery disease.^10,11 Cardiac POCUS is being used to evaluate COVID-19 patients when troponin and B-type natriuretic peptide (BNP) are elevated or when there are hemodynamic or electrocardiogram changes. Given the high incidence of venous thromboembolism (VTE) in COVID-19,¹² cardiac POCUS is being used to rapidly assess for right ventricular (RV) dysfunction and acute pulmonary hypertension.

The American Society of Echocardiography has recommended the use of cardiac POCUS by frontline providers for detection or characterization of preexisting cardiovascular disease, early identification of worsening cardiac function, serial monitoring and examination, and elucidation of cardiovascular pathologies associated with COVID-19.¹³ Sharing cardiac POCUS images in real time or through an image archive can reduce the need for consultative echocardiography, which ultimately reduces staff exposure, conserves personal protective equipment, and reduces need for decontamination of echocardiographic equipment.

The minimum cardiac POCUS views recommended in COVID-19 patients include the parasternal long-axis and short-axis views (midventricular level), either the apical or subcostal four-chamber view, and the subcostal long-axis view of the inferior vena cava.¹³ The goal of a cardiac POCUS exam is to qualitatively assess left ventricular (LV) systolic function, RV size and contractility, gross valvular and regional wall motion abnormalities, and pericardial effusion. In prone position ventilation, the swimmer’s position with one arm elevated above the shoulder may permit acquisition of apical views. Finally, integrated cardiopulmonary ultrasonography, including evaluation for deep vein thrombosis (DVT; see below), is ideal for proper characterization of underlying LV and RV function, volume status, and titration of vasopressor and inotropic support.

COVID-19 has been associated with a proinflammatory and hypercoagulable state with elevated d-dimer and higher-than-­expected incidence of VTE (27%) in critically ill patients.^12,14 Previous studies have demonstrated that frontline providers, including hospitalists, can detect lower extremity (LE) DVTs with high diagnostic accuracy using POCUS.¹⁵ Given the high incidence of DVTs despite prophylactic anticoagulation, some reports have suggested screening or serially monitoring for LE DVT in hospitalized COVID-19 patients.¹⁶ In patients with suspected pulmonary embolism (PE), POCUS can rapidly detect venous thrombosis that justifies prompt initiation of anticoagulation (eg, finding DVT or clot-in-transit), supportive findings of PE (eg, acute RV dysfunction, pulmonary infarcts), or alternative diagnoses (eg, bacterial pneumonia). However, it is important to recognize POCUS cannot definitively rule out PE. Additionally, subpleural consolidations are common in COVID-19 patients and could be caused by either infection or infarction. The American Society of Hematology has endorsed the use of POCUS, LE compression ultrasonography, and echocardiography in COVID-19 patients with suspected PE when availability of CT pulmonary angiography or ventilation-perfusion lung scans is limited.¹⁴

A POCUS exam for LE DVT consists of two-dimensional venous compression alone and yields results similar to formal vascular studies in both critically ill and noncritically ill patients. Because proximal LE thrombi have the highest risk of embolization, evaluation of the common femoral vein, femoral vein, and popliteal vein is most important.¹⁵ Either inability to compress a vein completely with wall-to-wall apposition or visualization of echogenic thrombus within the vein is diagnostic of DVT. Acute thrombi are gelatinous and may appear anechoic, while subacute or chronic thrombi are echogenic, but all veins with a DVT will not compress completely.

Ultrasound guidance for central venous catheter (CVC) insertion has been shown to increase procedure success rates and decrease mechanical complications, primarily arterial puncture and pneumothorax. Similarly, higher success rates and fewer insertion attempts have been observed with ultrasound-guided peripheral intravenous line and arterial line placement.¹⁷ Ultrasound-­guided PIV placement can reduce referrals for midlines and peripherally inserted central catheters in hospitalized patients.¹⁸

In COVID-19 patients, use of ultrasound guidance for vascular access has distinct advantages. First, given the high incidence of DVT in COVID-19 patients,¹² POCUS allows preprocedural evaluation of the target vessel for thrombosis, as well as anatomic variations and stenosis. Second, visualizing the needle tip and guidewire within the target vein prior to dilation nearly eliminates the risk of arterial puncture and inadvertent arterial dilation, which is particularly important in COVID-19 patients receiving high-dose prophylactic or therapeutic anticoagulation. Third, when inserting internal jugular and subclavian CVCs, visualization of normal lung sliding before and after the procedure safely rules out pneumothorax. However, if lung sliding is not seen before the procedure, it cannot be used to rule out pneumothorax afterward. Additionally, visualizing absence of the catheter tip in the right atrium and presence of a rapid atrial swirl sign within 2 seconds of briskly injecting 10 mL of saline confirms catheter tip placement near the superior vena cava/right atrial junction, which can eliminate the need for a postprocedure chest radiograph.¹⁷

POCUS can be used to rapidly confirm endotracheal tube (ETT) placement, which can reduce reliance on postintubation chest radiographs. A meta-analysis of prospective and randomized trials showed transtracheal ultrasonography had high sensitivity (98.7%) and specificity (97.1%) for confirming tracheal placement of ETTs.¹⁹ Confirming endotracheal intubation involves two steps: First, a linear transducer is placed transversely over the suprasternal notch to visualize the ETT passing through the trachea, and not the esophagus, during insertion. Second, after the ETT cuff has been inflated, bilateral lung sliding should be seen in sync with the respiratory cycle if the ETT is in the trachea. Absent lung sliding, but preserved lung pulse, on the anterior hemithorax is likely caused by main stem bronchial intubation, and withdrawing the ETT until bilateral lung sliding is seen confirms tracheal placement. Additionally, the following steps are recommended to reduce the risk of exposure to healthcare workers: minimizing use of bag-valve-mask ventilation, performing rapid sequence intubation using video laryngoscopy, and connecting the ETT to the ventilator immediately.

Important considerations when selecting an ultrasound machine for use in COVID-19 patients include image quality, portability, functionality, and ease of disinfection. Advantages of handheld devices include portability and ease of disinfection, whereas cart-based systems generally have better image quality and functionality. To minimize the risk of cross contamination, an ultrasound machine should be dedicated exclusively for use on patients with confirmed COVID-19 and not shared with patients with suspected COVID-19.²⁰ To minimize exposure to COVID-19 patients, frontline providers should perform POCUS exams only when findings may change management, and timing of the exam and views acquired should be selected deliberately.

Ultrasound machine disinfection should be integrated into routine donning and doffing procedures. When possible, both handheld and cart-based machines should be draped with protective covers during aerosol-generating procedures. Single use ultrasound gel packets are recommended in order to decrease the risk of nosocomial infection.²⁰ After every use of an ultrasound machine on intact skin or for percutaneous procedures, low-level disinfection should be performed with an Environmental Protection Agency–recommended product that is effective against coronavirus.

Some ultrasound manufacturers have added teleultrasound software that allows remote training of novice POCUS users and remote guidance in actual patient care. Teleultrasound can be utilized to share images in real time with consultants or expert providers.

POCUS is uniquely poised to improve patient care during the COVID-19 pandemic. POCUS can be used to support the diagnosis of COVID-19 patients and monitor patients with confirmed disease. Common POCUS applications used in COVID-19 patients include evaluation of the lungs, heart, and deep veins, as well as performance of bedside procedures. Ultrasound machine portability and disinfection are important considerations in COVID-19 patients.

Voluntary accountable care organizations (ACOs) and bundled payments have concurrently become cornerstone strategies in Medicare’s shift from volume-based fee-for-service toward value-based payment.

Physician practice and hospital participation in Medicare’s largest ACO model, the Medicare Shared Savings Program (MSSP),¹ grew to include 561 organizations in 2018. Under MSSP, participants assume financial accountability for the global quality and costs of care for defined populations of Medicare fee-for-service patients. ACOs that manage to maintain or improve quality while achieving savings (ie, containing costs below a predefined population-wide spending benchmark) are eligible to receive a portion of the difference back from Medicare in the form of “shared savings”.

Similarly, hospital participation in Medicare’s bundled payment programs has grown over time. Most notably, more than 700 participants enrolled in the recently concluded Bundled Payments for Care Improvement (BPCI) initiative,² Medicare’s largest bundled payment program over the past five years.³ Under BPCI, participants assumed financial accountability for the quality and costs of care for all Medicare patients triggering a qualifying “episode of care”. Participants that limit episode spending below a predefined benchmark without compromising quality were eligible for financial incentives.

As both ACOs and bundled payments grow in prominence and scale, they may increasingly overlap if patients attributed to ACOs receive care at bundled payment hospitals. Overlap could create synergies by increasing incentives to address shared processes (eg, discharge planning) or outcomes (eg, readmissions).⁴ An ACO focus on reducing hospital admissions could complement bundled payment efforts to increase hospital efficiency.

Conversely, Medicare’s approach to allocating savings and losses can penalize ACOs or bundled payment participants.³ For example, when a patient included in an MSSP ACO population receives episodic care at a hospital participating in BPCI, the historical costs of care for the hospital and the episode type, not the actual costs of care for that specific patient and his/her episode, are counted in the performance of the ACO. In other words, in these cases, the performance of the MSSP ACO is dependent on the historical spending at BPCI hospitals—despite it being out of ACO’s control and having little to do with the actual care its patients receive at BPCI hospitals—and MSSP ACOs cannot benefit from improvements over time. Therefore, MSSP ACOs may be functionally penalized if patients receive care at historically high-cost BPCI hospitals regardless of whether they have considerably improved the value of care delivered. As a corollary, Medicare rules involve a “claw back” stipulation in which savings are recouped from hospitals that participate in both BPCI and MSSP, effectively discouraging participation in both payment models.

Although these dynamics are complex, they highlight an intuitive point that has gained increasing awareness,⁵ ie, policymakers must understand the magnitude of overlap to evaluate the urgency in coordinating between the payment models. Our objective was to describe the extent of overlap and the characteristics of patients affected by it.

We used 100% institutional Medicare claims, MSSP beneficiary attribution, and BPCI hospital data to identify fee-for-service beneficiaries attributed to MSSP and/or receiving care at BPCI hospitals for its 48 included episodes from the start of BPCI in 2013 quarter 4 through 2016 quarter 4.

We examined the trends in the number of episodes across the following three groups: MSSP-attributed patients hospitalized at BPCI hospitals for an episode included in BPCI (Overlap), MSSP-attributed patients hospitalized for that episode at non-BPCI hospitals (MSSP-only), and non-MSSP-attributed patients hospitalized at BPCI hospitals for a BPCI episode (BPCI-only). We used Medicare and United States Census Bureau data to compare groups with respect to sociodemographic (eg, age, sex, residence in a low-income area),⁶ clinical (eg, Elixhauser comorbidity index),⁷ and prior utilization (eg, skilled nursing facility discharge) characteristics.

Categorical and continuous variables were compared using logistic regression and one-way analysis of variance, respectively. Analyses were performed using Stata (StataCorp, College Station, Texas), version 15.0. Statistical tests were 2-tailed and significant at α = 0.05. This study was approved by the institutional review board at the University of Pennsylvania.

The number of MSSP ACOs increased from 220 in 2013 to 432 in 2016. The number of BPCI hospitals increased from 9 to 389 over this period, peaking at 413 hospitals in 2015. Over our study period, a total of 243,392, 2,824,898, and 702,864 episodes occurred in the Overlap, ACO-only, and BPCI-only groups, respectively (Table). Among episodes, patients in the Overlap group generally showed lower severity than those in other groups, although the differences were small. The BPCI-only, MSSP-only, and Overlap groups also exhibited small differences with respect to other characteristics such as the proportion of patients with Medicare/Medicaid dual-eligibility (15% of individual vs 16% and 12%, respectively) and prior use of skilled nursing facilities (33% vs 34% vs 31%, respectively) and acute care hospitals (45% vs 41% vs 39%, respectively) (P < .001 for all).

The overall overlap facing MSSP patients (overlap as a proportion of all MSSP patients) increased from 0.3% at the end of 2013 to 10% at the end of 2016, whereas over the same period, overlap facing bundled payment patients (overlap as a proportion of all bundled payment patients) increased from 11.9% to 27% (Appendix Figure). Overlap facing MSSP ACOs varied according to episode type, ranging from 3% for both acute myocardial infarction and chronic obstructive pulmonary disease episodes to 18% for automatic implantable cardiac defibrillator episodes at the end of 2016. Similarly, overlap facing bundled payment patients varied from 21% for spinal fusion episodes to 32% for lower extremity joint replacement and automatic implantable cardiac defibrillator episodes.

To our knowledge, this is the first study to describe the sizable and growing overlap facing ACOs with attributed patients who receive care at bundled payment hospitals, as well as bundled payment hospitals that treat patients attributed to ACOs.

The major implication of our findings is that policymakers must address and anticipate forthcoming payment model overlap as a key policy priority. Given the emphasis on ACOs and bundled payments as payment models—for example, Medicare continues to implement both nationwide via the Next Generation ACO model⁸ and the recently launched BPCI-Advanced program⁹—policymakers urgently need insights about the extent of payment model overlap. In that context, it is notable that although we have evaluated MSSP and BPCI as flagship programs, true overlap may actually be greater once other programs are considered.

Several factors may underlie the differences in the magnitude of overlap facing bundled payment versus ACO patients. The models differ in how they identify relevant patient populations, with patients falling under bundled payments via hospitalization for certain episode types but patients falling under ACOs via attribution based on the plurality of primary care services. Furthermore, BPCI participation lagged behind MSSP participation in time, while also occurring disproportionately in areas with existing MSSP ACOs.

Given these findings, understanding the implications of overlap should be a priority for future research and policy strategies. Potential policy considerations should include revising cost accounting processes so that when ACO-attributed patients receive episodic care at bundled payment hospitals, actual rather than historical hospital costs are counted toward ACO cost performance. To encourage hospitals to assume more accountability over outcomes—the ostensible overarching goal of value-based payment reform—Medicare could elect not to recoup savings from hospitals in both payment models. Although such changes require careful accounting to protect Medicare from financial losses as it forgoes some savings achieved through payment reforms, this may be worthwhile if hospital engagement in both models yields synergies.

Importantly, any policy changes made to address program overlap would need to accommodate ongoing changes in ACO, bundled payments, and other payment programs. For example, Medicare overhauled MSSP in December 2018. Compared to the earlier rules, in which ACOs could avoid downside financial risk altogether via “upside only” arrangements for up to six years, new MSSP rules require all participants to assume downside risk after several years of participation. Separately, forthcoming payment reforms such as direct contracting¹⁰ may draw clinicians and hospitals previously not participating in either Medicare fee-for-service or value-based payment models into payment reform. These factors may affect overlap in unpredictable ways (eg, they may increase the overlap by increasing the number of patients whose care is covered by different payment models or they may decrease overlap by raising the financial stakes of payment reforms to a degree that organizations drop out altogether).

This study has limitations. First, generalizability is limited by the fact that our analysis did not include bundled payment episodes assigned to physician group participants in BPCI or hospitals in mandatory joint replacement bundles under the Medicare Comprehensive Care for Joint Replacement model.¹¹ Second, although this study provides the first description of overlap between ACO and bundled payment programs, it was descriptive in nature. Future research is needed to evaluate the impact of overlap on clinical, quality, and cost outcomes. This is particularly important because although we observed only small differences in patient characteristics among MSSP-only, BPCI-only, and Overlap groups, characteristics could change differentially over time. Payment reforms must be carefully monitored for potentially unintended consequences that could arise from differential changes in patient characteristics (eg, cherry-picking behavior that is disadvantageous to vulnerable individuals).

Nonetheless, this study underscores the importance and extent of overlap and the urgency to consider policy measures to coordinate between the payment models.

Acknowledgments

The authors thank research assistance from Sandra Vanderslice who did not receive any compensation for her work. This research was supported in part by The Commonwealth Fund. Rachel Werner was supported in part by K24-AG047908 from the NIA.

Voluntary accountable care organizations (ACOs) and bundled payments have concurrently become cornerstone strategies in Medicare’s shift from volume-based fee-for-service toward value-based payment.

Physician practice and hospital participation in Medicare’s largest ACO model, the Medicare Shared Savings Program (MSSP),¹ grew to include 561 organizations in 2018. Under MSSP, participants assume financial accountability for the global quality and costs of care for defined populations of Medicare fee-for-service patients. ACOs that manage to maintain or improve quality while achieving savings (ie, containing costs below a predefined population-wide spending benchmark) are eligible to receive a portion of the difference back from Medicare in the form of “shared savings”.

Similarly, hospital participation in Medicare’s bundled payment programs has grown over time. Most notably, more than 700 participants enrolled in the recently concluded Bundled Payments for Care Improvement (BPCI) initiative,² Medicare’s largest bundled payment program over the past five years.³ Under BPCI, participants assumed financial accountability for the quality and costs of care for all Medicare patients triggering a qualifying “episode of care”. Participants that limit episode spending below a predefined benchmark without compromising quality were eligible for financial incentives.

As both ACOs and bundled payments grow in prominence and scale, they may increasingly overlap if patients attributed to ACOs receive care at bundled payment hospitals. Overlap could create synergies by increasing incentives to address shared processes (eg, discharge planning) or outcomes (eg, readmissions).⁴ An ACO focus on reducing hospital admissions could complement bundled payment efforts to increase hospital efficiency.

Conversely, Medicare’s approach to allocating savings and losses can penalize ACOs or bundled payment participants.³ For example, when a patient included in an MSSP ACO population receives episodic care at a hospital participating in BPCI, the historical costs of care for the hospital and the episode type, not the actual costs of care for that specific patient and his/her episode, are counted in the performance of the ACO. In other words, in these cases, the performance of the MSSP ACO is dependent on the historical spending at BPCI hospitals—despite it being out of ACO’s control and having little to do with the actual care its patients receive at BPCI hospitals—and MSSP ACOs cannot benefit from improvements over time. Therefore, MSSP ACOs may be functionally penalized if patients receive care at historically high-cost BPCI hospitals regardless of whether they have considerably improved the value of care delivered. As a corollary, Medicare rules involve a “claw back” stipulation in which savings are recouped from hospitals that participate in both BPCI and MSSP, effectively discouraging participation in both payment models.

Although these dynamics are complex, they highlight an intuitive point that has gained increasing awareness,⁵ ie, policymakers must understand the magnitude of overlap to evaluate the urgency in coordinating between the payment models. Our objective was to describe the extent of overlap and the characteristics of patients affected by it.

We used 100% institutional Medicare claims, MSSP beneficiary attribution, and BPCI hospital data to identify fee-for-service beneficiaries attributed to MSSP and/or receiving care at BPCI hospitals for its 48 included episodes from the start of BPCI in 2013 quarter 4 through 2016 quarter 4.

We examined the trends in the number of episodes across the following three groups: MSSP-attributed patients hospitalized at BPCI hospitals for an episode included in BPCI (Overlap), MSSP-attributed patients hospitalized for that episode at non-BPCI hospitals (MSSP-only), and non-MSSP-attributed patients hospitalized at BPCI hospitals for a BPCI episode (BPCI-only). We used Medicare and United States Census Bureau data to compare groups with respect to sociodemographic (eg, age, sex, residence in a low-income area),⁶ clinical (eg, Elixhauser comorbidity index),⁷ and prior utilization (eg, skilled nursing facility discharge) characteristics.

Categorical and continuous variables were compared using logistic regression and one-way analysis of variance, respectively. Analyses were performed using Stata (StataCorp, College Station, Texas), version 15.0. Statistical tests were 2-tailed and significant at α = 0.05. This study was approved by the institutional review board at the University of Pennsylvania.

The number of MSSP ACOs increased from 220 in 2013 to 432 in 2016. The number of BPCI hospitals increased from 9 to 389 over this period, peaking at 413 hospitals in 2015. Over our study period, a total of 243,392, 2,824,898, and 702,864 episodes occurred in the Overlap, ACO-only, and BPCI-only groups, respectively (Table). Among episodes, patients in the Overlap group generally showed lower severity than those in other groups, although the differences were small. The BPCI-only, MSSP-only, and Overlap groups also exhibited small differences with respect to other characteristics such as the proportion of patients with Medicare/Medicaid dual-eligibility (15% of individual vs 16% and 12%, respectively) and prior use of skilled nursing facilities (33% vs 34% vs 31%, respectively) and acute care hospitals (45% vs 41% vs 39%, respectively) (P < .001 for all).

The overall overlap facing MSSP patients (overlap as a proportion of all MSSP patients) increased from 0.3% at the end of 2013 to 10% at the end of 2016, whereas over the same period, overlap facing bundled payment patients (overlap as a proportion of all bundled payment patients) increased from 11.9% to 27% (Appendix Figure). Overlap facing MSSP ACOs varied according to episode type, ranging from 3% for both acute myocardial infarction and chronic obstructive pulmonary disease episodes to 18% for automatic implantable cardiac defibrillator episodes at the end of 2016. Similarly, overlap facing bundled payment patients varied from 21% for spinal fusion episodes to 32% for lower extremity joint replacement and automatic implantable cardiac defibrillator episodes.

To our knowledge, this is the first study to describe the sizable and growing overlap facing ACOs with attributed patients who receive care at bundled payment hospitals, as well as bundled payment hospitals that treat patients attributed to ACOs.

The major implication of our findings is that policymakers must address and anticipate forthcoming payment model overlap as a key policy priority. Given the emphasis on ACOs and bundled payments as payment models—for example, Medicare continues to implement both nationwide via the Next Generation ACO model⁸ and the recently launched BPCI-Advanced program⁹—policymakers urgently need insights about the extent of payment model overlap. In that context, it is notable that although we have evaluated MSSP and BPCI as flagship programs, true overlap may actually be greater once other programs are considered.

Several factors may underlie the differences in the magnitude of overlap facing bundled payment versus ACO patients. The models differ in how they identify relevant patient populations, with patients falling under bundled payments via hospitalization for certain episode types but patients falling under ACOs via attribution based on the plurality of primary care services. Furthermore, BPCI participation lagged behind MSSP participation in time, while also occurring disproportionately in areas with existing MSSP ACOs.

Given these findings, understanding the implications of overlap should be a priority for future research and policy strategies. Potential policy considerations should include revising cost accounting processes so that when ACO-attributed patients receive episodic care at bundled payment hospitals, actual rather than historical hospital costs are counted toward ACO cost performance. To encourage hospitals to assume more accountability over outcomes—the ostensible overarching goal of value-based payment reform—Medicare could elect not to recoup savings from hospitals in both payment models. Although such changes require careful accounting to protect Medicare from financial losses as it forgoes some savings achieved through payment reforms, this may be worthwhile if hospital engagement in both models yields synergies.

Importantly, any policy changes made to address program overlap would need to accommodate ongoing changes in ACO, bundled payments, and other payment programs. For example, Medicare overhauled MSSP in December 2018. Compared to the earlier rules, in which ACOs could avoid downside financial risk altogether via “upside only” arrangements for up to six years, new MSSP rules require all participants to assume downside risk after several years of participation. Separately, forthcoming payment reforms such as direct contracting¹⁰ may draw clinicians and hospitals previously not participating in either Medicare fee-for-service or value-based payment models into payment reform. These factors may affect overlap in unpredictable ways (eg, they may increase the overlap by increasing the number of patients whose care is covered by different payment models or they may decrease overlap by raising the financial stakes of payment reforms to a degree that organizations drop out altogether).

This study has limitations. First, generalizability is limited by the fact that our analysis did not include bundled payment episodes assigned to physician group participants in BPCI or hospitals in mandatory joint replacement bundles under the Medicare Comprehensive Care for Joint Replacement model.¹¹ Second, although this study provides the first description of overlap between ACO and bundled payment programs, it was descriptive in nature. Future research is needed to evaluate the impact of overlap on clinical, quality, and cost outcomes. This is particularly important because although we observed only small differences in patient characteristics among MSSP-only, BPCI-only, and Overlap groups, characteristics could change differentially over time. Payment reforms must be carefully monitored for potentially unintended consequences that could arise from differential changes in patient characteristics (eg, cherry-picking behavior that is disadvantageous to vulnerable individuals).

Nonetheless, this study underscores the importance and extent of overlap and the urgency to consider policy measures to coordinate between the payment models.

Acknowledgments

The authors thank research assistance from Sandra Vanderslice who did not receive any compensation for her work. This research was supported in part by The Commonwealth Fund. Rachel Werner was supported in part by K24-AG047908 from the NIA.

Voluntary accountable care organizations (ACOs) and bundled payments have concurrently become cornerstone strategies in Medicare’s shift from volume-based fee-for-service toward value-based payment.

Physician practice and hospital participation in Medicare’s largest ACO model, the Medicare Shared Savings Program (MSSP),¹ grew to include 561 organizations in 2018. Under MSSP, participants assume financial accountability for the global quality and costs of care for defined populations of Medicare fee-for-service patients. ACOs that manage to maintain or improve quality while achieving savings (ie, containing costs below a predefined population-wide spending benchmark) are eligible to receive a portion of the difference back from Medicare in the form of “shared savings”.

Similarly, hospital participation in Medicare’s bundled payment programs has grown over time. Most notably, more than 700 participants enrolled in the recently concluded Bundled Payments for Care Improvement (BPCI) initiative,² Medicare’s largest bundled payment program over the past five years.³ Under BPCI, participants assumed financial accountability for the quality and costs of care for all Medicare patients triggering a qualifying “episode of care”. Participants that limit episode spending below a predefined benchmark without compromising quality were eligible for financial incentives.

As both ACOs and bundled payments grow in prominence and scale, they may increasingly overlap if patients attributed to ACOs receive care at bundled payment hospitals. Overlap could create synergies by increasing incentives to address shared processes (eg, discharge planning) or outcomes (eg, readmissions).⁴ An ACO focus on reducing hospital admissions could complement bundled payment efforts to increase hospital efficiency.

Conversely, Medicare’s approach to allocating savings and losses can penalize ACOs or bundled payment participants.³ For example, when a patient included in an MSSP ACO population receives episodic care at a hospital participating in BPCI, the historical costs of care for the hospital and the episode type, not the actual costs of care for that specific patient and his/her episode, are counted in the performance of the ACO. In other words, in these cases, the performance of the MSSP ACO is dependent on the historical spending at BPCI hospitals—despite it being out of ACO’s control and having little to do with the actual care its patients receive at BPCI hospitals—and MSSP ACOs cannot benefit from improvements over time. Therefore, MSSP ACOs may be functionally penalized if patients receive care at historically high-cost BPCI hospitals regardless of whether they have considerably improved the value of care delivered. As a corollary, Medicare rules involve a “claw back” stipulation in which savings are recouped from hospitals that participate in both BPCI and MSSP, effectively discouraging participation in both payment models.

Although these dynamics are complex, they highlight an intuitive point that has gained increasing awareness,⁵ ie, policymakers must understand the magnitude of overlap to evaluate the urgency in coordinating between the payment models. Our objective was to describe the extent of overlap and the characteristics of patients affected by it.

We used 100% institutional Medicare claims, MSSP beneficiary attribution, and BPCI hospital data to identify fee-for-service beneficiaries attributed to MSSP and/or receiving care at BPCI hospitals for its 48 included episodes from the start of BPCI in 2013 quarter 4 through 2016 quarter 4.

We examined the trends in the number of episodes across the following three groups: MSSP-attributed patients hospitalized at BPCI hospitals for an episode included in BPCI (Overlap), MSSP-attributed patients hospitalized for that episode at non-BPCI hospitals (MSSP-only), and non-MSSP-attributed patients hospitalized at BPCI hospitals for a BPCI episode (BPCI-only). We used Medicare and United States Census Bureau data to compare groups with respect to sociodemographic (eg, age, sex, residence in a low-income area),⁶ clinical (eg, Elixhauser comorbidity index),⁷ and prior utilization (eg, skilled nursing facility discharge) characteristics.

Categorical and continuous variables were compared using logistic regression and one-way analysis of variance, respectively. Analyses were performed using Stata (StataCorp, College Station, Texas), version 15.0. Statistical tests were 2-tailed and significant at α = 0.05. This study was approved by the institutional review board at the University of Pennsylvania.

The number of MSSP ACOs increased from 220 in 2013 to 432 in 2016. The number of BPCI hospitals increased from 9 to 389 over this period, peaking at 413 hospitals in 2015. Over our study period, a total of 243,392, 2,824,898, and 702,864 episodes occurred in the Overlap, ACO-only, and BPCI-only groups, respectively (Table). Among episodes, patients in the Overlap group generally showed lower severity than those in other groups, although the differences were small. The BPCI-only, MSSP-only, and Overlap groups also exhibited small differences with respect to other characteristics such as the proportion of patients with Medicare/Medicaid dual-eligibility (15% of individual vs 16% and 12%, respectively) and prior use of skilled nursing facilities (33% vs 34% vs 31%, respectively) and acute care hospitals (45% vs 41% vs 39%, respectively) (P < .001 for all).

The overall overlap facing MSSP patients (overlap as a proportion of all MSSP patients) increased from 0.3% at the end of 2013 to 10% at the end of 2016, whereas over the same period, overlap facing bundled payment patients (overlap as a proportion of all bundled payment patients) increased from 11.9% to 27% (Appendix Figure). Overlap facing MSSP ACOs varied according to episode type, ranging from 3% for both acute myocardial infarction and chronic obstructive pulmonary disease episodes to 18% for automatic implantable cardiac defibrillator episodes at the end of 2016. Similarly, overlap facing bundled payment patients varied from 21% for spinal fusion episodes to 32% for lower extremity joint replacement and automatic implantable cardiac defibrillator episodes.

To our knowledge, this is the first study to describe the sizable and growing overlap facing ACOs with attributed patients who receive care at bundled payment hospitals, as well as bundled payment hospitals that treat patients attributed to ACOs.

The major implication of our findings is that policymakers must address and anticipate forthcoming payment model overlap as a key policy priority. Given the emphasis on ACOs and bundled payments as payment models—for example, Medicare continues to implement both nationwide via the Next Generation ACO model⁸ and the recently launched BPCI-Advanced program⁹—policymakers urgently need insights about the extent of payment model overlap. In that context, it is notable that although we have evaluated MSSP and BPCI as flagship programs, true overlap may actually be greater once other programs are considered.

Several factors may underlie the differences in the magnitude of overlap facing bundled payment versus ACO patients. The models differ in how they identify relevant patient populations, with patients falling under bundled payments via hospitalization for certain episode types but patients falling under ACOs via attribution based on the plurality of primary care services. Furthermore, BPCI participation lagged behind MSSP participation in time, while also occurring disproportionately in areas with existing MSSP ACOs.

Given these findings, understanding the implications of overlap should be a priority for future research and policy strategies. Potential policy considerations should include revising cost accounting processes so that when ACO-attributed patients receive episodic care at bundled payment hospitals, actual rather than historical hospital costs are counted toward ACO cost performance. To encourage hospitals to assume more accountability over outcomes—the ostensible overarching goal of value-based payment reform—Medicare could elect not to recoup savings from hospitals in both payment models. Although such changes require careful accounting to protect Medicare from financial losses as it forgoes some savings achieved through payment reforms, this may be worthwhile if hospital engagement in both models yields synergies.

Importantly, any policy changes made to address program overlap would need to accommodate ongoing changes in ACO, bundled payments, and other payment programs. For example, Medicare overhauled MSSP in December 2018. Compared to the earlier rules, in which ACOs could avoid downside financial risk altogether via “upside only” arrangements for up to six years, new MSSP rules require all participants to assume downside risk after several years of participation. Separately, forthcoming payment reforms such as direct contracting¹⁰ may draw clinicians and hospitals previously not participating in either Medicare fee-for-service or value-based payment models into payment reform. These factors may affect overlap in unpredictable ways (eg, they may increase the overlap by increasing the number of patients whose care is covered by different payment models or they may decrease overlap by raising the financial stakes of payment reforms to a degree that organizations drop out altogether).

This study has limitations. First, generalizability is limited by the fact that our analysis did not include bundled payment episodes assigned to physician group participants in BPCI or hospitals in mandatory joint replacement bundles under the Medicare Comprehensive Care for Joint Replacement model.¹¹ Second, although this study provides the first description of overlap between ACO and bundled payment programs, it was descriptive in nature. Future research is needed to evaluate the impact of overlap on clinical, quality, and cost outcomes. This is particularly important because although we observed only small differences in patient characteristics among MSSP-only, BPCI-only, and Overlap groups, characteristics could change differentially over time. Payment reforms must be carefully monitored for potentially unintended consequences that could arise from differential changes in patient characteristics (eg, cherry-picking behavior that is disadvantageous to vulnerable individuals).

Nonetheless, this study underscores the importance and extent of overlap and the urgency to consider policy measures to coordinate between the payment models.

Acknowledgments

The authors thank research assistance from Sandra Vanderslice who did not receive any compensation for her work. This research was supported in part by The Commonwealth Fund. Rachel Werner was supported in part by K24-AG047908 from the NIA.

With more than 3 million people diagnosed and more than 200,000 deaths worldwide at the time this article was written, coronavirus disease of 2019 (COVID-19) poses an unprecedented challenge to the public and to our healthcare system.¹ The United States has surpassed every other country in the total number of COVID-19 cases. Hospitals in hotspots are operating beyond capacity, while others prepare for a predicted surge of patients suffering from COVID-19. Now more than ever, clinicians need to prioritize limited time and resources wisely in this rapidly changing environment. Our most precious limited resource, healthcare workers (HCWs), bravely care for patients while trying to avoid acquiring the infection. With each test and treatment, clinicians must carefully consider harms and benefits, including exposing themselves and other HCWs to SARS-CoV-2, the virus causing this disease.

Delivering any healthcare service in which the potential harm exceeds benefit represents one form of overuse. In the era of COVID-19, the harmful consequences of overuse go beyond the patient to the healthcare team. For example, unnecessary chest computed tomography (CT) to help diagnose COVID-19 comes with the usual risks to the patient including radiation, but it may also reveal a suspicious nodule. That incidental finding can lead to downstream consequences, such as more imaging, blood work, and biopsy. In the current pandemic, however, that CT comes with more than just the usual risk. The initial unnecessary chest CT can risk exposing the transporter, the staff in the hallways and elevator en route, the radiology staff operating the CT scanner, and the maintenance staff who must clean the room and scanner afterward. Potential downstream harms to staff include exposure of the pulmonary and interventional radiology consultants, as well as the staff who perform repeat imaging after the biopsy. Evaluation of the nodule potentially prolongs the patient’s stay and exposes more staff. Clinicians must weigh the benefits and harms of each test and treatment carefully with consideration of both the patient and the staff involved. Moreover, it may turn out that the patient and staff without symptoms of COVID-19 may pose the most risk to one another.

Choosing Wisely® partnered with patients and clinician societies to develop a Top 5 recommendations list for eliminating unnecessary testing and treatment. Our multi-institutional group from the High Value Practice Academic Alliance proposed this Top 5 list of overuse practices in hospital medicine that can lead to harm of both patients and HCWs in the COVID-19 era (Table). The following recommendations apply to all patients with unsuspected, suspected, or confirmed SARS-CoV-2 infection in the hospital setting.

• Do not obtain nonurgent labs in separate blood draws if they can be batched together.

This recommendation expands on the original Society of Hospital Medicine Choosing Wisely recommendation: Don’t perform repetitive complete blood count and chemistry testing in the face of clinical and lab stability.² Aside from patient harms such as pain and hospital-acquired anemia, the risk of exposure to HCWs who perform phlebotomy (phlebotomists, nurses, and other clinicians), as well as staff who transport, handle, and process the bloodwork in the lab, must be minimized. Most prior interventions to eliminate unnecessary bloodwork focused on the number of lab tests,³ but some also aimed to batch nonurgent labs together to effectively reduce unnecessary needlesticks (“think twice, stick once”).⁴ This concept can be brought into this pandemic to provide safe and appropriate care for both patients and HCWs.

• Do not use bronchodilators unless there is active obstructive airway disease, and if needed, use metered dose inhalers instead of nebulizers.

We do not recommend using bronchodilators to treat COVID-19 symptoms unless patients develop acute bronchospastic symptoms of their underlying obstructive airway disease.⁵ When needed, use metered dose inhalers (MDIs),⁶ if available, instead of nebulizers because the latter potentiates aerosolization that could lead to higher risk of spreading the infection. The risk extends to respiratory technicians and nurses who administer the nebulizer, as well as other HCWs who enter the room during or after administration. The Centers for Disease Control and Prevention (CDC) considers nebulized bronchodilator therapy a “high-risk” exposure for HCWs not wearing the proper personal protectvie equipment.⁷ Moreover, MDI therapy produces equivalent outcomes to nebulized treatments for patients who are not critically ill.⁶ Unfortunately, the supply of MDIs during this crisis has not kept up with the increased demand.⁸

There are no clear guidelines for reuse of MDIs in COVID-19; however, options include labeling patients’ MDIs to use for hospitalization and discharge or labeling an MDI for use during hospitalization and then disinfecting for reuse. For safety reasons, MDIs of COVID-19 patients should be reused only for other patients with COVID-19.⁸

• Do not use posteroanterior and lateral chest X-ray as initial imaging. Use a portable chest X-ray instead.

The CDC does not currently recommend diagnosing COVID-19 by chest X-ray (CXR).⁷ When used appropriately, CXR can provide information to support a COVID-19 diagnosis and rule out other etiologies that cause respiratory symptoms.⁹ Posteroanterior (PA) and lateral CXR are more sensitive than portable CXR for detecting pleural effusions, and lateral CXR is needed to examine structures along the axis of the body. Portable CXR also may cause the heart to appear magnified and the mediastinum widened, the diaphragm to appear higher, and vascular shadows to be obscured.¹⁰ The improved ability to detect these subtle differences should be weighed against the increased risk to HCWs required to perform PA and lateral CXR. A portable CXR exposes a relatively smaller number of staff who come to the bedside versus the larger number of people exposed in transporting the patient out of the room and into the hallway, elevator, and the radiology suite for a PA and lateral CXR.

• Avoid in-person evaluations in favor of virtual communication unless necessary.

To minimize HCW exposure to COVID-19 and optimize infection control, the CDC recommends the use of telemedicine when possible.⁷ Telemedicine refers to the use of technology to support clinical care across some distance, which includes video visits and remote clinical monitoring. At the time of writing, the Centers for Medicare & Medicaid Services had waived the rural site of care requirement for Medicare beneficiaries, granted 49 Medicaid waivers to states to enhance flexibility, and (at least temporarily) added inpatient care to the list of reimbursed telemedicine services.¹¹ Funding for expanded coverage under Medicare is included in the recent Coronavirus Preparedness and Response Supplemental Appropriations Act.¹² These federal changes open the door for commercial payers and state Medicaid programs to further boost telemedicine through reimbursement parity to in-person visits and other coverage policies. Hospitalists can ride this momentum and learn from ambulatory colleagues to harness the power of telemedicine and minimize unnecessary face-to-face interactions with patients who are suspected or confirmed to have COVID-19.¹³ Even if providers have to enter the patient’s room, telemedicine may still allow for large virtual family meetings despite strict visitor restrictions and physical distance with loved ones. If in-person visits are necessary, only one designated person should enter the patient’s room instead of the entire team.

• Do not delay goals of care conversations for hospitalized patients who are unlikely to benefit from life-sustaining treatments.

The COVID-19 pandemic amplifies the need for early goals of care discussions. Mortality rates range higher with acute respiratory distress syndrome from COVID-19, compared with other etiologies, and is associated with extended intensive care unit stays.¹⁴ The harms extend beyond the patient and families to our HCWs through psychological distress and heightened exposure from aerosolization during resuscitation. Advance care planning should center on the values and preferences of the patient. Rather than asking if the patient or family would want certain treatments, it is crucial for clinicians to be direct in making do-not-resuscitate recommendations if deemed futile care.¹⁵ This practice is well within legal confines and is distinct from withdrawal or withholding of life-sustaining resources.¹⁵

HCWs providing inpatient care during this pandemic remain among the highest risk for contracting the infection. As of April 9, 2020, nearly 9,300 HCWs in the United States have contracted COVID-19.¹⁶ One thing remains clear: If we want to protect our patients, we must start by protecting our HCWs. We must think critically to evaluate the potential harms to our extended healthcare teams and strive further to eliminate overuse from our care.

The authors represent members of the High Value Practice Academic Alliance. The High Value Practice Academic Alliance is a consortium of academic medical centers in the United States and Canada working to advance high-value healthcare through collaborative quality improvement, research, and education. Additional information is available at http://www.hvpaa.org.

With more than 3 million people diagnosed and more than 200,000 deaths worldwide at the time this article was written, coronavirus disease of 2019 (COVID-19) poses an unprecedented challenge to the public and to our healthcare system.¹ The United States has surpassed every other country in the total number of COVID-19 cases. Hospitals in hotspots are operating beyond capacity, while others prepare for a predicted surge of patients suffering from COVID-19. Now more than ever, clinicians need to prioritize limited time and resources wisely in this rapidly changing environment. Our most precious limited resource, healthcare workers (HCWs), bravely care for patients while trying to avoid acquiring the infection. With each test and treatment, clinicians must carefully consider harms and benefits, including exposing themselves and other HCWs to SARS-CoV-2, the virus causing this disease.

Delivering any healthcare service in which the potential harm exceeds benefit represents one form of overuse. In the era of COVID-19, the harmful consequences of overuse go beyond the patient to the healthcare team. For example, unnecessary chest computed tomography (CT) to help diagnose COVID-19 comes with the usual risks to the patient including radiation, but it may also reveal a suspicious nodule. That incidental finding can lead to downstream consequences, such as more imaging, blood work, and biopsy. In the current pandemic, however, that CT comes with more than just the usual risk. The initial unnecessary chest CT can risk exposing the transporter, the staff in the hallways and elevator en route, the radiology staff operating the CT scanner, and the maintenance staff who must clean the room and scanner afterward. Potential downstream harms to staff include exposure of the pulmonary and interventional radiology consultants, as well as the staff who perform repeat imaging after the biopsy. Evaluation of the nodule potentially prolongs the patient’s stay and exposes more staff. Clinicians must weigh the benefits and harms of each test and treatment carefully with consideration of both the patient and the staff involved. Moreover, it may turn out that the patient and staff without symptoms of COVID-19 may pose the most risk to one another.

Choosing Wisely® partnered with patients and clinician societies to develop a Top 5 recommendations list for eliminating unnecessary testing and treatment. Our multi-institutional group from the High Value Practice Academic Alliance proposed this Top 5 list of overuse practices in hospital medicine that can lead to harm of both patients and HCWs in the COVID-19 era (Table). The following recommendations apply to all patients with unsuspected, suspected, or confirmed SARS-CoV-2 infection in the hospital setting.

• Do not obtain nonurgent labs in separate blood draws if they can be batched together.

This recommendation expands on the original Society of Hospital Medicine Choosing Wisely recommendation: Don’t perform repetitive complete blood count and chemistry testing in the face of clinical and lab stability.² Aside from patient harms such as pain and hospital-acquired anemia, the risk of exposure to HCWs who perform phlebotomy (phlebotomists, nurses, and other clinicians), as well as staff who transport, handle, and process the bloodwork in the lab, must be minimized. Most prior interventions to eliminate unnecessary bloodwork focused on the number of lab tests,³ but some also aimed to batch nonurgent labs together to effectively reduce unnecessary needlesticks (“think twice, stick once”).⁴ This concept can be brought into this pandemic to provide safe and appropriate care for both patients and HCWs.

• Do not use bronchodilators unless there is active obstructive airway disease, and if needed, use metered dose inhalers instead of nebulizers.

We do not recommend using bronchodilators to treat COVID-19 symptoms unless patients develop acute bronchospastic symptoms of their underlying obstructive airway disease.⁵ When needed, use metered dose inhalers (MDIs),⁶ if available, instead of nebulizers because the latter potentiates aerosolization that could lead to higher risk of spreading the infection. The risk extends to respiratory technicians and nurses who administer the nebulizer, as well as other HCWs who enter the room during or after administration. The Centers for Disease Control and Prevention (CDC) considers nebulized bronchodilator therapy a “high-risk” exposure for HCWs not wearing the proper personal protectvie equipment.⁷ Moreover, MDI therapy produces equivalent outcomes to nebulized treatments for patients who are not critically ill.⁶ Unfortunately, the supply of MDIs during this crisis has not kept up with the increased demand.⁸

There are no clear guidelines for reuse of MDIs in COVID-19; however, options include labeling patients’ MDIs to use for hospitalization and discharge or labeling an MDI for use during hospitalization and then disinfecting for reuse. For safety reasons, MDIs of COVID-19 patients should be reused only for other patients with COVID-19.⁸

• Do not use posteroanterior and lateral chest X-ray as initial imaging. Use a portable chest X-ray instead.

The CDC does not currently recommend diagnosing COVID-19 by chest X-ray (CXR).⁷ When used appropriately, CXR can provide information to support a COVID-19 diagnosis and rule out other etiologies that cause respiratory symptoms.⁹ Posteroanterior (PA) and lateral CXR are more sensitive than portable CXR for detecting pleural effusions, and lateral CXR is needed to examine structures along the axis of the body. Portable CXR also may cause the heart to appear magnified and the mediastinum widened, the diaphragm to appear higher, and vascular shadows to be obscured.¹⁰ The improved ability to detect these subtle differences should be weighed against the increased risk to HCWs required to perform PA and lateral CXR. A portable CXR exposes a relatively smaller number of staff who come to the bedside versus the larger number of people exposed in transporting the patient out of the room and into the hallway, elevator, and the radiology suite for a PA and lateral CXR.

• Avoid in-person evaluations in favor of virtual communication unless necessary.

To minimize HCW exposure to COVID-19 and optimize infection control, the CDC recommends the use of telemedicine when possible.⁷ Telemedicine refers to the use of technology to support clinical care across some distance, which includes video visits and remote clinical monitoring. At the time of writing, the Centers for Medicare & Medicaid Services had waived the rural site of care requirement for Medicare beneficiaries, granted 49 Medicaid waivers to states to enhance flexibility, and (at least temporarily) added inpatient care to the list of reimbursed telemedicine services.¹¹ Funding for expanded coverage under Medicare is included in the recent Coronavirus Preparedness and Response Supplemental Appropriations Act.¹² These federal changes open the door for commercial payers and state Medicaid programs to further boost telemedicine through reimbursement parity to in-person visits and other coverage policies. Hospitalists can ride this momentum and learn from ambulatory colleagues to harness the power of telemedicine and minimize unnecessary face-to-face interactions with patients who are suspected or confirmed to have COVID-19.¹³ Even if providers have to enter the patient’s room, telemedicine may still allow for large virtual family meetings despite strict visitor restrictions and physical distance with loved ones. If in-person visits are necessary, only one designated person should enter the patient’s room instead of the entire team.

• Do not delay goals of care conversations for hospitalized patients who are unlikely to benefit from life-sustaining treatments.

The COVID-19 pandemic amplifies the need for early goals of care discussions. Mortality rates range higher with acute respiratory distress syndrome from COVID-19, compared with other etiologies, and is associated with extended intensive care unit stays.¹⁴ The harms extend beyond the patient and families to our HCWs through psychological distress and heightened exposure from aerosolization during resuscitation. Advance care planning should center on the values and preferences of the patient. Rather than asking if the patient or family would want certain treatments, it is crucial for clinicians to be direct in making do-not-resuscitate recommendations if deemed futile care.¹⁵ This practice is well within legal confines and is distinct from withdrawal or withholding of life-sustaining resources.¹⁵

HCWs providing inpatient care during this pandemic remain among the highest risk for contracting the infection. As of April 9, 2020, nearly 9,300 HCWs in the United States have contracted COVID-19.¹⁶ One thing remains clear: If we want to protect our patients, we must start by protecting our HCWs. We must think critically to evaluate the potential harms to our extended healthcare teams and strive further to eliminate overuse from our care.

The authors represent members of the High Value Practice Academic Alliance. The High Value Practice Academic Alliance is a consortium of academic medical centers in the United States and Canada working to advance high-value healthcare through collaborative quality improvement, research, and education. Additional information is available at http://www.hvpaa.org.

With more than 3 million people diagnosed and more than 200,000 deaths worldwide at the time this article was written, coronavirus disease of 2019 (COVID-19) poses an unprecedented challenge to the public and to our healthcare system.¹ The United States has surpassed every other country in the total number of COVID-19 cases. Hospitals in hotspots are operating beyond capacity, while others prepare for a predicted surge of patients suffering from COVID-19. Now more than ever, clinicians need to prioritize limited time and resources wisely in this rapidly changing environment. Our most precious limited resource, healthcare workers (HCWs), bravely care for patients while trying to avoid acquiring the infection. With each test and treatment, clinicians must carefully consider harms and benefits, including exposing themselves and other HCWs to SARS-CoV-2, the virus causing this disease.

Delivering any healthcare service in which the potential harm exceeds benefit represents one form of overuse. In the era of COVID-19, the harmful consequences of overuse go beyond the patient to the healthcare team. For example, unnecessary chest computed tomography (CT) to help diagnose COVID-19 comes with the usual risks to the patient including radiation, but it may also reveal a suspicious nodule. That incidental finding can lead to downstream consequences, such as more imaging, blood work, and biopsy. In the current pandemic, however, that CT comes with more than just the usual risk. The initial unnecessary chest CT can risk exposing the transporter, the staff in the hallways and elevator en route, the radiology staff operating the CT scanner, and the maintenance staff who must clean the room and scanner afterward. Potential downstream harms to staff include exposure of the pulmonary and interventional radiology consultants, as well as the staff who perform repeat imaging after the biopsy. Evaluation of the nodule potentially prolongs the patient’s stay and exposes more staff. Clinicians must weigh the benefits and harms of each test and treatment carefully with consideration of both the patient and the staff involved. Moreover, it may turn out that the patient and staff without symptoms of COVID-19 may pose the most risk to one another.

Choosing Wisely® partnered with patients and clinician societies to develop a Top 5 recommendations list for eliminating unnecessary testing and treatment. Our multi-institutional group from the High Value Practice Academic Alliance proposed this Top 5 list of overuse practices in hospital medicine that can lead to harm of both patients and HCWs in the COVID-19 era (Table). The following recommendations apply to all patients with unsuspected, suspected, or confirmed SARS-CoV-2 infection in the hospital setting.

• Do not obtain nonurgent labs in separate blood draws if they can be batched together.

This recommendation expands on the original Society of Hospital Medicine Choosing Wisely recommendation: Don’t perform repetitive complete blood count and chemistry testing in the face of clinical and lab stability.² Aside from patient harms such as pain and hospital-acquired anemia, the risk of exposure to HCWs who perform phlebotomy (phlebotomists, nurses, and other clinicians), as well as staff who transport, handle, and process the bloodwork in the lab, must be minimized. Most prior interventions to eliminate unnecessary bloodwork focused on the number of lab tests,³ but some also aimed to batch nonurgent labs together to effectively reduce unnecessary needlesticks (“think twice, stick once”).⁴ This concept can be brought into this pandemic to provide safe and appropriate care for both patients and HCWs.

• Do not use bronchodilators unless there is active obstructive airway disease, and if needed, use metered dose inhalers instead of nebulizers.

We do not recommend using bronchodilators to treat COVID-19 symptoms unless patients develop acute bronchospastic symptoms of their underlying obstructive airway disease.⁵ When needed, use metered dose inhalers (MDIs),⁶ if available, instead of nebulizers because the latter potentiates aerosolization that could lead to higher risk of spreading the infection. The risk extends to respiratory technicians and nurses who administer the nebulizer, as well as other HCWs who enter the room during or after administration. The Centers for Disease Control and Prevention (CDC) considers nebulized bronchodilator therapy a “high-risk” exposure for HCWs not wearing the proper personal protectvie equipment.⁷ Moreover, MDI therapy produces equivalent outcomes to nebulized treatments for patients who are not critically ill.⁶ Unfortunately, the supply of MDIs during this crisis has not kept up with the increased demand.⁸

There are no clear guidelines for reuse of MDIs in COVID-19; however, options include labeling patients’ MDIs to use for hospitalization and discharge or labeling an MDI for use during hospitalization and then disinfecting for reuse. For safety reasons, MDIs of COVID-19 patients should be reused only for other patients with COVID-19.⁸

• Do not use posteroanterior and lateral chest X-ray as initial imaging. Use a portable chest X-ray instead.

The CDC does not currently recommend diagnosing COVID-19 by chest X-ray (CXR).⁷ When used appropriately, CXR can provide information to support a COVID-19 diagnosis and rule out other etiologies that cause respiratory symptoms.⁹ Posteroanterior (PA) and lateral CXR are more sensitive than portable CXR for detecting pleural effusions, and lateral CXR is needed to examine structures along the axis of the body. Portable CXR also may cause the heart to appear magnified and the mediastinum widened, the diaphragm to appear higher, and vascular shadows to be obscured.¹⁰ The improved ability to detect these subtle differences should be weighed against the increased risk to HCWs required to perform PA and lateral CXR. A portable CXR exposes a relatively smaller number of staff who come to the bedside versus the larger number of people exposed in transporting the patient out of the room and into the hallway, elevator, and the radiology suite for a PA and lateral CXR.

• Avoid in-person evaluations in favor of virtual communication unless necessary.

To minimize HCW exposure to COVID-19 and optimize infection control, the CDC recommends the use of telemedicine when possible.⁷ Telemedicine refers to the use of technology to support clinical care across some distance, which includes video visits and remote clinical monitoring. At the time of writing, the Centers for Medicare & Medicaid Services had waived the rural site of care requirement for Medicare beneficiaries, granted 49 Medicaid waivers to states to enhance flexibility, and (at least temporarily) added inpatient care to the list of reimbursed telemedicine services.¹¹ Funding for expanded coverage under Medicare is included in the recent Coronavirus Preparedness and Response Supplemental Appropriations Act.¹² These federal changes open the door for commercial payers and state Medicaid programs to further boost telemedicine through reimbursement parity to in-person visits and other coverage policies. Hospitalists can ride this momentum and learn from ambulatory colleagues to harness the power of telemedicine and minimize unnecessary face-to-face interactions with patients who are suspected or confirmed to have COVID-19.¹³ Even if providers have to enter the patient’s room, telemedicine may still allow for large virtual family meetings despite strict visitor restrictions and physical distance with loved ones. If in-person visits are necessary, only one designated person should enter the patient’s room instead of the entire team.

• Do not delay goals of care conversations for hospitalized patients who are unlikely to benefit from life-sustaining treatments.

The COVID-19 pandemic amplifies the need for early goals of care discussions. Mortality rates range higher with acute respiratory distress syndrome from COVID-19, compared with other etiologies, and is associated with extended intensive care unit stays.¹⁴ The harms extend beyond the patient and families to our HCWs through psychological distress and heightened exposure from aerosolization during resuscitation. Advance care planning should center on the values and preferences of the patient. Rather than asking if the patient or family would want certain treatments, it is crucial for clinicians to be direct in making do-not-resuscitate recommendations if deemed futile care.¹⁵ This practice is well within legal confines and is distinct from withdrawal or withholding of life-sustaining resources.¹⁵

HCWs providing inpatient care during this pandemic remain among the highest risk for contracting the infection. As of April 9, 2020, nearly 9,300 HCWs in the United States have contracted COVID-19.¹⁶ One thing remains clear: If we want to protect our patients, we must start by protecting our HCWs. We must think critically to evaluate the potential harms to our extended healthcare teams and strive further to eliminate overuse from our care.

The authors represent members of the High Value Practice Academic Alliance. The High Value Practice Academic Alliance is a consortium of academic medical centers in the United States and Canada working to advance high-value healthcare through collaborative quality improvement, research, and education. Additional information is available at http://www.hvpaa.org.

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

The coronavirus disease of 2019 (COVID-19) pandemic has strained the healthcare system by rapidly depleting multiple resources including hospital space, medications, ventilators, personal protective equipment (PPE), clinical revenue, and morale. One of the most essential at-risk resources is healthcare providers. Healthcare providers have been overwhelmed as hospital systems have experienced local surges in COVID-19 patients. Compounding this is the fact that providers are more likely to contract COVID-19, which could sideline portions of an already taxed workforce.

Multiple “surge” interventions have been planned or implemented to mitigate a current or anticipated dearth of physicians. Some institutions are reallocating subspecialists and surgeons to general ward and intensive care unit (ICU) roles, often with support from hospitalists and ICU physicians.¹ Others have used telemedicine to reduce personnel exposure and conserve PPE.² A novel and perhaps paradigm-shifting solution arose in March 2020 when several medical schools around the world announced they would graduate final year students early to allow them to join the workforce during the COVID-19 surge.^3-7 In the United States, fourth-year medical students at multiple institutions in cities such as New York, Boston, Phoenix, Tucson, Newark, Portland, and Bethesda were offered the opportunity to graduate in April rather than in May or June. The Liaison Committee on Medical Education stated that for students to graduate early, they must have already met all curricular requirements and be deemed ready by an evaluations and promotions committee.⁸ What these early graduates do with their “gap time” before residency is neither standardized nor prescribed. The Accreditation Council for Graduate Medical Education has discouraged individuals from joining their newly matched residency programs early.⁹ Some early graduates who wish to bolster the workforce have signed temporary training agreements with local healthcare systems to work for a 1- to 2-month period before moving on to their matched residency program. Some institutions have already been working with local and state officials to rapidly grant provisional temporary licenses for this purpose.¹⁰

Early medical school graduation in times of international crisis is not without precedent. When faced with physician shortages during World War II, the United States federal government urged medical colleges to graduate trainees in 3 years.¹¹ The national medical education milieu was different then, with standardized medical school training still crystalizing merely 30 years following the Flexner report. However, there was pressure from the federal government during World War II, whereas decisions around early graduation today are driven by institutional and local officials. While a few accelerated programs persist today, there has not been an urgent, unplanned early release of graduates to meet a public health need on such a large scale in recent history. The seasonal timing of the pandemic surge in the United States may have been a key factor in deciding to graduate students early. With a late winter and early spring peak, final year students are graduating only 2 to 3 months early. But what if another peak occurs in late summer or early fall, and some students are graduated even earlier? With which aspects of patient care would hospitalists trust these graduates, and with what level of supervision? Whether now or with a future COVID-19 peak, we describe how trust develops with learners and provide hospitalists with a framework for deliberate entrustment if and when they are asked to integrate early medical school graduates into their workforce.

The degree of supervision that is provided to a learner is linked to how much a supervisor trusts the learner, as well as the specific context. Trust has many forms, often depending on what type of information informs it. Presumptive trust is trust based on credentials, without any actual interaction with the learner.¹² Healthcare systems typically assume that medical school graduates are ready to perform intern-level tasks based on their medical degree. This presumptive trust may be bolstered by the assumption that a residency program director has vetted a learner’s credentials during the match process. On meeting a learner, we develop initial trust, which is based on first impressions and snap-judgment. Over time, presumptive and initial trust can be replaced by grounded trust, or trust based on demonstrated performance after prolonged experience with a learner. Under normal circumstances, supervisors use observations of learner performance in the clinical environment to develop grounded trust. With early graduates, especially those who sign temporary work agreements, the usual progression of trust may be compressed. Hospitalists may have less presumptive trust because these students graduated early and little time to develop grounded trust before integrating new graduates into patient care. How should hospitalists navigate supervision in this setting?

Missing a few months at the end of medical school likely does not significantly affect competence and, therefore, should not affect presumptive trust. The value of the fourth year of medical school has been questioned because, after fulfilling graduation requirements, students often spend significant amounts of time interviewing, traveling, taking electives with lighter workloads, or exploring nonclinical interests late in the year.¹³ More intense “subintern” rotations, which are important for the residency application process, occur earlier in the academic year. It is therefore reasonable to presume that most students graduating in April are not less prepared than those graduating in June.

Additionally, there is significant interlearner variability in rates of competence attainment.¹⁴ This means that there is no magic point in time at which students are fully ready for resident-­level responsibilities. Some students are likely competent to be interns without a fourth year at all, while others are still facing challenges in their development at the end of medical school. As Englander and Carraccio wrote, “The notion that every medical student across the nation has somehow achieved all the competencies necessary to start residency training on July 1 of their graduation year is magical thinking.”¹⁵ Since there is no universal, time-based finish line for competence, we should not be thrown by a slight change in the arbitrary line currently drawn in June. Whether students graduate in April or June, it remains true that some will be more ready than others.

With compressed timelines, hospitalists may default to initial trust, relying heavily on first impressions to determine how much supervision an early graduate requires. For example, a graduate who is extroverted, assertive, and articulate may give off an air of confidence, which could entice a supervising hospitalist to give a “longer leash” with higher-risk patient care tasks. It is easy to fall prey to the “confidence equals competence” heuristic, but this has been shown to be unreliable.¹⁶ Initial trust is influenced by both social biases (eg, gender, race, age) and cognitive biases (eg, halo effect) that have little or nothing to do with the actual abilities of learners. While initial trust and accompanying biases often develop unconsciously, it is important to reflect on how unfounded first impressions can influence trust and supervision decisions.

Hospitalists must be deliberate with entrustment decisions, especially in a pandemic environment. There are useful guides for making these decisions that can be used in a point-of-care manner.¹⁷ First, it is important to acknowledge that entrustment is based in part on the perceived trustworthiness of a person. Kennedy and colleagues have described four components of trustworthiness, all of which can be assessed by hospitalists in the moment of care delivery: (1) knowledge and skill (Does the trainee possess the requisite knowledge and skill to perform the task?), (2) conscientiousness (Does the trainee follow through on tasks? Are they thorough and dependable?), (3) discernment (Does the trainee recognize personal limitations and seek help when needed?), and (4) truthfulness (Does the trainee tell the truth?).¹⁷

Entrustment decisions also depend on the specific task being observed (eg, high risk vs low risk) and context (eg, severity of illness of the patient, acuity of the setting).¹⁸ Trust is linked with perceived risk and benefits.¹⁹ More entrustment (less supervision) may be given when perceived risk is low, such as prescribing acetaminophen on a stable patient or taking an initial history. Less entrustment (more supervision) may be given when perceived risk is high, such as with managing septic shock or inserting a central venous catheter. However, the duress of the COVID-19 pandemic may tilt the risk/benefit balance toward less-than-usual supervision if an early graduate is the only provider available for some higher-risk tasks. This underscores the importance of direct observation leading to grounded trust with progressively higher-risk tasks as dictated by the local pandemic environment.

As much as possible, trust should be determined based on direct observation, not fallible first impressions or inference. Supervisors often use inference when assuming that performance on one task reflects performance on others. For example, if learners are observed to be competent when interpreting electrocardiograms, one might infer they also know how to manage tachyarrhythmias. If they can manage tachyarrhythmias, one might infer they also know how to manage acute coronary syndrome. These inferences are not the way to build grounded trust because competence is task and context dependent.

Direct observation can include watching patient interactions, being present for procedures, think-alouds during didactics, cognitive autopsies, reviewing notes, and informal conversations. Being deliberate with direct observation and entrustment decision-making can be challenging because of the high cognitive load of caring for sick and complex patients, maintaining proper PPE practices, and simultaneously assessing an early graduate’s performance. However, maintaining a level of supervision that is appropriate for trainee competence is paramount for patient safety. It may be valuable to identify tasks needing to be performed by early graduates and using focused simulation to generate a significant number of observations over a short period of time. Trust should be gained once competence is observed, not inferred or assumed. Instead of “trust, but verify,” we should “observe, then trust.”

There is a moral obligation to patients to avoid placing trainees in situations for which they are ill prepared based on their current abilities. We must balance the risk that exists both in leaving early graduates on the sidelines (overprotecting them as learners) and in asking them to perform tasks for which they are not prepared (overextending them as a workforce). Focusing on grounded trust derived from direct observation of performance while also balancing the risks and benefits inherent in the local pandemic context can help hospitalists calibrate supervision to a level that helps extend the workforce in a time of crisis while maintaining patient safety.

The coronavirus disease of 2019 (COVID-19) pandemic has strained the healthcare system by rapidly depleting multiple resources including hospital space, medications, ventilators, personal protective equipment (PPE), clinical revenue, and morale. One of the most essential at-risk resources is healthcare providers. Healthcare providers have been overwhelmed as hospital systems have experienced local surges in COVID-19 patients. Compounding this is the fact that providers are more likely to contract COVID-19, which could sideline portions of an already taxed workforce.

Multiple “surge” interventions have been planned or implemented to mitigate a current or anticipated dearth of physicians. Some institutions are reallocating subspecialists and surgeons to general ward and intensive care unit (ICU) roles, often with support from hospitalists and ICU physicians.¹ Others have used telemedicine to reduce personnel exposure and conserve PPE.² A novel and perhaps paradigm-shifting solution arose in March 2020 when several medical schools around the world announced they would graduate final year students early to allow them to join the workforce during the COVID-19 surge.^3-7 In the United States, fourth-year medical students at multiple institutions in cities such as New York, Boston, Phoenix, Tucson, Newark, Portland, and Bethesda were offered the opportunity to graduate in April rather than in May or June. The Liaison Committee on Medical Education stated that for students to graduate early, they must have already met all curricular requirements and be deemed ready by an evaluations and promotions committee.⁸ What these early graduates do with their “gap time” before residency is neither standardized nor prescribed. The Accreditation Council for Graduate Medical Education has discouraged individuals from joining their newly matched residency programs early.⁹ Some early graduates who wish to bolster the workforce have signed temporary training agreements with local healthcare systems to work for a 1- to 2-month period before moving on to their matched residency program. Some institutions have already been working with local and state officials to rapidly grant provisional temporary licenses for this purpose.¹⁰

Early medical school graduation in times of international crisis is not without precedent. When faced with physician shortages during World War II, the United States federal government urged medical colleges to graduate trainees in 3 years.¹¹ The national medical education milieu was different then, with standardized medical school training still crystalizing merely 30 years following the Flexner report. However, there was pressure from the federal government during World War II, whereas decisions around early graduation today are driven by institutional and local officials. While a few accelerated programs persist today, there has not been an urgent, unplanned early release of graduates to meet a public health need on such a large scale in recent history. The seasonal timing of the pandemic surge in the United States may have been a key factor in deciding to graduate students early. With a late winter and early spring peak, final year students are graduating only 2 to 3 months early. But what if another peak occurs in late summer or early fall, and some students are graduated even earlier? With which aspects of patient care would hospitalists trust these graduates, and with what level of supervision? Whether now or with a future COVID-19 peak, we describe how trust develops with learners and provide hospitalists with a framework for deliberate entrustment if and when they are asked to integrate early medical school graduates into their workforce.

The degree of supervision that is provided to a learner is linked to how much a supervisor trusts the learner, as well as the specific context. Trust has many forms, often depending on what type of information informs it. Presumptive trust is trust based on credentials, without any actual interaction with the learner.¹² Healthcare systems typically assume that medical school graduates are ready to perform intern-level tasks based on their medical degree. This presumptive trust may be bolstered by the assumption that a residency program director has vetted a learner’s credentials during the match process. On meeting a learner, we develop initial trust, which is based on first impressions and snap-judgment. Over time, presumptive and initial trust can be replaced by grounded trust, or trust based on demonstrated performance after prolonged experience with a learner. Under normal circumstances, supervisors use observations of learner performance in the clinical environment to develop grounded trust. With early graduates, especially those who sign temporary work agreements, the usual progression of trust may be compressed. Hospitalists may have less presumptive trust because these students graduated early and little time to develop grounded trust before integrating new graduates into patient care. How should hospitalists navigate supervision in this setting?

Missing a few months at the end of medical school likely does not significantly affect competence and, therefore, should not affect presumptive trust. The value of the fourth year of medical school has been questioned because, after fulfilling graduation requirements, students often spend significant amounts of time interviewing, traveling, taking electives with lighter workloads, or exploring nonclinical interests late in the year.¹³ More intense “subintern” rotations, which are important for the residency application process, occur earlier in the academic year. It is therefore reasonable to presume that most students graduating in April are not less prepared than those graduating in June.

Additionally, there is significant interlearner variability in rates of competence attainment.¹⁴ This means that there is no magic point in time at which students are fully ready for resident-­level responsibilities. Some students are likely competent to be interns without a fourth year at all, while others are still facing challenges in their development at the end of medical school. As Englander and Carraccio wrote, “The notion that every medical student across the nation has somehow achieved all the competencies necessary to start residency training on July 1 of their graduation year is magical thinking.”¹⁵ Since there is no universal, time-based finish line for competence, we should not be thrown by a slight change in the arbitrary line currently drawn in June. Whether students graduate in April or June, it remains true that some will be more ready than others.

With compressed timelines, hospitalists may default to initial trust, relying heavily on first impressions to determine how much supervision an early graduate requires. For example, a graduate who is extroverted, assertive, and articulate may give off an air of confidence, which could entice a supervising hospitalist to give a “longer leash” with higher-risk patient care tasks. It is easy to fall prey to the “confidence equals competence” heuristic, but this has been shown to be unreliable.¹⁶ Initial trust is influenced by both social biases (eg, gender, race, age) and cognitive biases (eg, halo effect) that have little or nothing to do with the actual abilities of learners. While initial trust and accompanying biases often develop unconsciously, it is important to reflect on how unfounded first impressions can influence trust and supervision decisions.

Hospitalists must be deliberate with entrustment decisions, especially in a pandemic environment. There are useful guides for making these decisions that can be used in a point-of-care manner.¹⁷ First, it is important to acknowledge that entrustment is based in part on the perceived trustworthiness of a person. Kennedy and colleagues have described four components of trustworthiness, all of which can be assessed by hospitalists in the moment of care delivery: (1) knowledge and skill (Does the trainee possess the requisite knowledge and skill to perform the task?), (2) conscientiousness (Does the trainee follow through on tasks? Are they thorough and dependable?), (3) discernment (Does the trainee recognize personal limitations and seek help when needed?), and (4) truthfulness (Does the trainee tell the truth?).¹⁷

Entrustment decisions also depend on the specific task being observed (eg, high risk vs low risk) and context (eg, severity of illness of the patient, acuity of the setting).¹⁸ Trust is linked with perceived risk and benefits.¹⁹ More entrustment (less supervision) may be given when perceived risk is low, such as prescribing acetaminophen on a stable patient or taking an initial history. Less entrustment (more supervision) may be given when perceived risk is high, such as with managing septic shock or inserting a central venous catheter. However, the duress of the COVID-19 pandemic may tilt the risk/benefit balance toward less-than-usual supervision if an early graduate is the only provider available for some higher-risk tasks. This underscores the importance of direct observation leading to grounded trust with progressively higher-risk tasks as dictated by the local pandemic environment.

As much as possible, trust should be determined based on direct observation, not fallible first impressions or inference. Supervisors often use inference when assuming that performance on one task reflects performance on others. For example, if learners are observed to be competent when interpreting electrocardiograms, one might infer they also know how to manage tachyarrhythmias. If they can manage tachyarrhythmias, one might infer they also know how to manage acute coronary syndrome. These inferences are not the way to build grounded trust because competence is task and context dependent.

Direct observation can include watching patient interactions, being present for procedures, think-alouds during didactics, cognitive autopsies, reviewing notes, and informal conversations. Being deliberate with direct observation and entrustment decision-making can be challenging because of the high cognitive load of caring for sick and complex patients, maintaining proper PPE practices, and simultaneously assessing an early graduate’s performance. However, maintaining a level of supervision that is appropriate for trainee competence is paramount for patient safety. It may be valuable to identify tasks needing to be performed by early graduates and using focused simulation to generate a significant number of observations over a short period of time. Trust should be gained once competence is observed, not inferred or assumed. Instead of “trust, but verify,” we should “observe, then trust.”

There is a moral obligation to patients to avoid placing trainees in situations for which they are ill prepared based on their current abilities. We must balance the risk that exists both in leaving early graduates on the sidelines (overprotecting them as learners) and in asking them to perform tasks for which they are not prepared (overextending them as a workforce). Focusing on grounded trust derived from direct observation of performance while also balancing the risks and benefits inherent in the local pandemic context can help hospitalists calibrate supervision to a level that helps extend the workforce in a time of crisis while maintaining patient safety.

The coronavirus disease of 2019 (COVID-19) pandemic has strained the healthcare system by rapidly depleting multiple resources including hospital space, medications, ventilators, personal protective equipment (PPE), clinical revenue, and morale. One of the most essential at-risk resources is healthcare providers. Healthcare providers have been overwhelmed as hospital systems have experienced local surges in COVID-19 patients. Compounding this is the fact that providers are more likely to contract COVID-19, which could sideline portions of an already taxed workforce.

Multiple “surge” interventions have been planned or implemented to mitigate a current or anticipated dearth of physicians. Some institutions are reallocating subspecialists and surgeons to general ward and intensive care unit (ICU) roles, often with support from hospitalists and ICU physicians.¹ Others have used telemedicine to reduce personnel exposure and conserve PPE.² A novel and perhaps paradigm-shifting solution arose in March 2020 when several medical schools around the world announced they would graduate final year students early to allow them to join the workforce during the COVID-19 surge.^3-7 In the United States, fourth-year medical students at multiple institutions in cities such as New York, Boston, Phoenix, Tucson, Newark, Portland, and Bethesda were offered the opportunity to graduate in April rather than in May or June. The Liaison Committee on Medical Education stated that for students to graduate early, they must have already met all curricular requirements and be deemed ready by an evaluations and promotions committee.⁸ What these early graduates do with their “gap time” before residency is neither standardized nor prescribed. The Accreditation Council for Graduate Medical Education has discouraged individuals from joining their newly matched residency programs early.⁹ Some early graduates who wish to bolster the workforce have signed temporary training agreements with local healthcare systems to work for a 1- to 2-month period before moving on to their matched residency program. Some institutions have already been working with local and state officials to rapidly grant provisional temporary licenses for this purpose.¹⁰

Early medical school graduation in times of international crisis is not without precedent. When faced with physician shortages during World War II, the United States federal government urged medical colleges to graduate trainees in 3 years.¹¹ The national medical education milieu was different then, with standardized medical school training still crystalizing merely 30 years following the Flexner report. However, there was pressure from the federal government during World War II, whereas decisions around early graduation today are driven by institutional and local officials. While a few accelerated programs persist today, there has not been an urgent, unplanned early release of graduates to meet a public health need on such a large scale in recent history. The seasonal timing of the pandemic surge in the United States may have been a key factor in deciding to graduate students early. With a late winter and early spring peak, final year students are graduating only 2 to 3 months early. But what if another peak occurs in late summer or early fall, and some students are graduated even earlier? With which aspects of patient care would hospitalists trust these graduates, and with what level of supervision? Whether now or with a future COVID-19 peak, we describe how trust develops with learners and provide hospitalists with a framework for deliberate entrustment if and when they are asked to integrate early medical school graduates into their workforce.

The degree of supervision that is provided to a learner is linked to how much a supervisor trusts the learner, as well as the specific context. Trust has many forms, often depending on what type of information informs it. Presumptive trust is trust based on credentials, without any actual interaction with the learner.¹² Healthcare systems typically assume that medical school graduates are ready to perform intern-level tasks based on their medical degree. This presumptive trust may be bolstered by the assumption that a residency program director has vetted a learner’s credentials during the match process. On meeting a learner, we develop initial trust, which is based on first impressions and snap-judgment. Over time, presumptive and initial trust can be replaced by grounded trust, or trust based on demonstrated performance after prolonged experience with a learner. Under normal circumstances, supervisors use observations of learner performance in the clinical environment to develop grounded trust. With early graduates, especially those who sign temporary work agreements, the usual progression of trust may be compressed. Hospitalists may have less presumptive trust because these students graduated early and little time to develop grounded trust before integrating new graduates into patient care. How should hospitalists navigate supervision in this setting?

Missing a few months at the end of medical school likely does not significantly affect competence and, therefore, should not affect presumptive trust. The value of the fourth year of medical school has been questioned because, after fulfilling graduation requirements, students often spend significant amounts of time interviewing, traveling, taking electives with lighter workloads, or exploring nonclinical interests late in the year.¹³ More intense “subintern” rotations, which are important for the residency application process, occur earlier in the academic year. It is therefore reasonable to presume that most students graduating in April are not less prepared than those graduating in June.

Additionally, there is significant interlearner variability in rates of competence attainment.¹⁴ This means that there is no magic point in time at which students are fully ready for resident-­level responsibilities. Some students are likely competent to be interns without a fourth year at all, while others are still facing challenges in their development at the end of medical school. As Englander and Carraccio wrote, “The notion that every medical student across the nation has somehow achieved all the competencies necessary to start residency training on July 1 of their graduation year is magical thinking.”¹⁵ Since there is no universal, time-based finish line for competence, we should not be thrown by a slight change in the arbitrary line currently drawn in June. Whether students graduate in April or June, it remains true that some will be more ready than others.

With compressed timelines, hospitalists may default to initial trust, relying heavily on first impressions to determine how much supervision an early graduate requires. For example, a graduate who is extroverted, assertive, and articulate may give off an air of confidence, which could entice a supervising hospitalist to give a “longer leash” with higher-risk patient care tasks. It is easy to fall prey to the “confidence equals competence” heuristic, but this has been shown to be unreliable.¹⁶ Initial trust is influenced by both social biases (eg, gender, race, age) and cognitive biases (eg, halo effect) that have little or nothing to do with the actual abilities of learners. While initial trust and accompanying biases often develop unconsciously, it is important to reflect on how unfounded first impressions can influence trust and supervision decisions.

Hospitalists must be deliberate with entrustment decisions, especially in a pandemic environment. There are useful guides for making these decisions that can be used in a point-of-care manner.¹⁷ First, it is important to acknowledge that entrustment is based in part on the perceived trustworthiness of a person. Kennedy and colleagues have described four components of trustworthiness, all of which can be assessed by hospitalists in the moment of care delivery: (1) knowledge and skill (Does the trainee possess the requisite knowledge and skill to perform the task?), (2) conscientiousness (Does the trainee follow through on tasks? Are they thorough and dependable?), (3) discernment (Does the trainee recognize personal limitations and seek help when needed?), and (4) truthfulness (Does the trainee tell the truth?).¹⁷

Entrustment decisions also depend on the specific task being observed (eg, high risk vs low risk) and context (eg, severity of illness of the patient, acuity of the setting).¹⁸ Trust is linked with perceived risk and benefits.¹⁹ More entrustment (less supervision) may be given when perceived risk is low, such as prescribing acetaminophen on a stable patient or taking an initial history. Less entrustment (more supervision) may be given when perceived risk is high, such as with managing septic shock or inserting a central venous catheter. However, the duress of the COVID-19 pandemic may tilt the risk/benefit balance toward less-than-usual supervision if an early graduate is the only provider available for some higher-risk tasks. This underscores the importance of direct observation leading to grounded trust with progressively higher-risk tasks as dictated by the local pandemic environment.

As much as possible, trust should be determined based on direct observation, not fallible first impressions or inference. Supervisors often use inference when assuming that performance on one task reflects performance on others. For example, if learners are observed to be competent when interpreting electrocardiograms, one might infer they also know how to manage tachyarrhythmias. If they can manage tachyarrhythmias, one might infer they also know how to manage acute coronary syndrome. These inferences are not the way to build grounded trust because competence is task and context dependent.

Direct observation can include watching patient interactions, being present for procedures, think-alouds during didactics, cognitive autopsies, reviewing notes, and informal conversations. Being deliberate with direct observation and entrustment decision-making can be challenging because of the high cognitive load of caring for sick and complex patients, maintaining proper PPE practices, and simultaneously assessing an early graduate’s performance. However, maintaining a level of supervision that is appropriate for trainee competence is paramount for patient safety. It may be valuable to identify tasks needing to be performed by early graduates and using focused simulation to generate a significant number of observations over a short period of time. Trust should be gained once competence is observed, not inferred or assumed. Instead of “trust, but verify,” we should “observe, then trust.”

There is a moral obligation to patients to avoid placing trainees in situations for which they are ill prepared based on their current abilities. We must balance the risk that exists both in leaving early graduates on the sidelines (overprotecting them as learners) and in asking them to perform tasks for which they are not prepared (overextending them as a workforce). Focusing on grounded trust derived from direct observation of performance while also balancing the risks and benefits inherent in the local pandemic context can help hospitalists calibrate supervision to a level that helps extend the workforce in a time of crisis while maintaining patient safety.

During the severe acute respiratory syndrome (SARS) outbreak in 2003, Taiwan reported 346 confirmed cases and 73 deaths.¹ Of all known infections, 94% were transmitted inside hospitals. Nine major hospitals were fully or partially shut down, and many doctors and nurses quit for fear of becoming infected. The Taipei Municipal Ho-Ping Hospital was most severely affected. Its index patient, a 42-year-old undocumented hospital laundry worker who interacted with staff and patients for 6 days before being hospitalized, became a superspreader, infecting at least 20 other patients and 10 staff members.^2,3 The entire 450-bed hospital was ordered to shut down, and all 930 staff and 240 patients were quarantined within the hospital. The central government appointed the previous Minister of Health as head of the Anti-SARS Taskforce. Ultimately the hospital was evacuated; the outbreak resulted in 26 deaths.² Events surrounding the hospital’s evacuation offer important lessons for hospitals struggling to cope with the COVID-19 pandemic, which has been caused by spread of a similar coronavirus.

Flexibility about case definition is important, as is use of clinical criteria for diagnosis when reliable laboratory tests are not available.

The laundry worker of Ho-Ping Hospital was initially misdiagnosed with infectious enteritis, which delayed proper management and, crucially, isolation from other patients. The low index of suspicion for SARS reflected the initial World Health Organization diagnostic criteria for SARS, which included travel to or residence in an area with recent local transmission of SARS within 10 days of symptom onset.⁴ The laundry worker did not have a recent travel history.³ Additionally, SARS manifested as a lower respiratory tract infection, so many patients were hospitalized for pneumonia before being diagnosed with SARS. Similarly, the Wuhan Municipal Health Commission initially issued diagnostic criteria for COVID-19 that, in addition to fever and symptoms of respiratory infections, emphasized direct exposure to the Huanan Seafood Wholesale Market.⁵ As a result, many cases of COVID-19 were not identified.

Diagnosing SARS was challenging. Early symptoms such as fever and malaise were nonspecific. Polymerase chain reaction tests, although available, were unreliable especially in early stages of the disease and had a high false-negative rate. As cases of SARS increased rapidly, Taiwan began using fever alone for early detection.⁶ Patients and hospital staff received temperature measurements twice daily. Despite the late start to SARS screening, the fever criterion identified many suspected patients, which ensured widespread detection and containment.

For COVID-19, symptoms such as fever, dry cough, and shortness of breath can be used as clinical criteria to triage patients for quarantine in endemic areas when reliable diagnostic tests are not readily available, but all frontline clinical staff should receive daily temperature checks and/or COVID-19 tests, if available, to protect their families and the public.

Ineffective coordination between central and local governments can delay response, but this can be remedied.

During the SARS outbreak, the Taipei City Government and the Taiwan central government were controlled by opposing political parties. Responses to SARS were initially impeded by political skirmishes, which hindered implementation of policies regarding criteria for diagnosis, tracking of suspected cases and their contacts, duration of quarantine, and allocation of resources and facilities for confirmed cases. To avoid further delays, the central government acted swiftly to create the nonpartisan Anti-SARS Taskforce and appointed leaders who could work cordially with both local and central government agencies. To help to deal with the crisis, the central government also designated a new Minister of Health, an epidemiologist, who became the first nonphysician to hold this position.

Treatment in place vs evacuation during hospital infections is a critical decision.

The surge of SARS cases at Ho-Ping Hospital led to confusion and panic among patients and hospital staff. Whether to treat its SARS patients on site or to evacuate the hospital was a complex decision and reflected many concerns, including the following: How many wards had been infected? Was there sufficient equipment (eg, respirators) to monitor or treat infected patients? How many isolation beds were available? How many hospital staff were already infected and quarantined? Were they in different wards? Were there neighboring facilities (eg, hospitals, military camps, dorms) available for quarantine?

If a hospital has sufficient capacity to isolate persons under investigation and to treat confirmed patients, on-site treatment is possible. However, evacuation should be considered when there is widespread infection involving different hospital wards and hospital staff. In such cases, patients should be transferred to different facilities based on clinical severity: Patients with new onset fever or respiratory symptoms but who are relatively healthy should be sent to community or regional hospitals with isolation rooms for monitoring; sicker infected patients should be sent to medical centers; and other hospitalized patients (eg, admitted for heart failure) without infection risk or symptoms should seek care elsewhere.

So far, there has been only one instance of hospital-based COVID-19 infections in Taiwan, and the spread of infection was quickly contained within one ward. All nine confirmed cases (including the index patient, one patient in the same ward but a different room, three nurses, one laundry worker, and three members of patients’ families) and their known contacts were identified, then isolated or quarantined individually. Because the affected hospital is part of a complex with more than 3,000 beds, it was big enough to accommodate all infected patients and no evacuation measures were needed. To further reduce potential nodes in the chain of transmission, interns and many other healthcare workers were temporarily relieved of their duties, elective surgeries were canceled, and hospital visitation was limited to immediate family members. The clear communication of intervention measures ensured rapid cooperation and staved off both social panic and further spread of the disease.

Hospitals should establish different flows for different patients.

Having learned from the SARS experience in 2003, hospitals in Taiwan have designated specific pathways to manage patient flow during the COVID-19 outbreak, in addition to checking all patients for travel history and fever: Patients with fever were quickly triaged to a designated fever clinic so they did not mingle with other patients, patients visiting the hospital to obtain chronic disease medications were directed to a “drive-through” lane, patients needing emergent care went through the emergency department, all other regular outpatients were seen in outpatient departments, and visitors of patients were restricted to one visitor per patient at a given time.

Healthcare providers should be organized into blocks and modular teams to avoid hospital-wide infection.

After SARS, Taiwan learned that one way to reduce the spread of something like COVID-19 among healthcare providers and from providers to patients is to divide providers’ work areas into discrete blocks and organize providers into modular teams. This approach was inspired by the design of watertight compartments in ships: Should the hull be breached, flooding is restricted to the breached compartments. Under this organizational strategy, movements of physicians and nurses would be restricted to their designated locations: They would be routinely exposed only to other staff and patients within their division. Doctors and nurses would be asked to practice in modular teams within their blocked locations, reducing the likelihood that infection in one team would spread to another, which could lead to hospital-wide infections. Movement of senior hospital executives would be similarly restricted. Common areas such as cafeterias where people mingle would be closed. Owing to these stringent initiatives, aside from the hospital-­based infection mentioned in Lesson 3, no other hospital-­based infections have been reported in Taiwan so far.

Lessons from previous hospital-based coronavirus infections can be used to minimize future infections.

The authors would like to thank Dr Lee Ming-Liang, former Health Minister of Taiwan and director of that central government’s Anti-SARS Taskforce during the 2003 outbreak, for providing valuable recommendations to this work.

During the severe acute respiratory syndrome (SARS) outbreak in 2003, Taiwan reported 346 confirmed cases and 73 deaths.¹ Of all known infections, 94% were transmitted inside hospitals. Nine major hospitals were fully or partially shut down, and many doctors and nurses quit for fear of becoming infected. The Taipei Municipal Ho-Ping Hospital was most severely affected. Its index patient, a 42-year-old undocumented hospital laundry worker who interacted with staff and patients for 6 days before being hospitalized, became a superspreader, infecting at least 20 other patients and 10 staff members.^2,3 The entire 450-bed hospital was ordered to shut down, and all 930 staff and 240 patients were quarantined within the hospital. The central government appointed the previous Minister of Health as head of the Anti-SARS Taskforce. Ultimately the hospital was evacuated; the outbreak resulted in 26 deaths.² Events surrounding the hospital’s evacuation offer important lessons for hospitals struggling to cope with the COVID-19 pandemic, which has been caused by spread of a similar coronavirus.

Flexibility about case definition is important, as is use of clinical criteria for diagnosis when reliable laboratory tests are not available.

The laundry worker of Ho-Ping Hospital was initially misdiagnosed with infectious enteritis, which delayed proper management and, crucially, isolation from other patients. The low index of suspicion for SARS reflected the initial World Health Organization diagnostic criteria for SARS, which included travel to or residence in an area with recent local transmission of SARS within 10 days of symptom onset.⁴ The laundry worker did not have a recent travel history.³ Additionally, SARS manifested as a lower respiratory tract infection, so many patients were hospitalized for pneumonia before being diagnosed with SARS. Similarly, the Wuhan Municipal Health Commission initially issued diagnostic criteria for COVID-19 that, in addition to fever and symptoms of respiratory infections, emphasized direct exposure to the Huanan Seafood Wholesale Market.⁵ As a result, many cases of COVID-19 were not identified.

Diagnosing SARS was challenging. Early symptoms such as fever and malaise were nonspecific. Polymerase chain reaction tests, although available, were unreliable especially in early stages of the disease and had a high false-negative rate. As cases of SARS increased rapidly, Taiwan began using fever alone for early detection.⁶ Patients and hospital staff received temperature measurements twice daily. Despite the late start to SARS screening, the fever criterion identified many suspected patients, which ensured widespread detection and containment.

For COVID-19, symptoms such as fever, dry cough, and shortness of breath can be used as clinical criteria to triage patients for quarantine in endemic areas when reliable diagnostic tests are not readily available, but all frontline clinical staff should receive daily temperature checks and/or COVID-19 tests, if available, to protect their families and the public.

Ineffective coordination between central and local governments can delay response, but this can be remedied.

During the SARS outbreak, the Taipei City Government and the Taiwan central government were controlled by opposing political parties. Responses to SARS were initially impeded by political skirmishes, which hindered implementation of policies regarding criteria for diagnosis, tracking of suspected cases and their contacts, duration of quarantine, and allocation of resources and facilities for confirmed cases. To avoid further delays, the central government acted swiftly to create the nonpartisan Anti-SARS Taskforce and appointed leaders who could work cordially with both local and central government agencies. To help to deal with the crisis, the central government also designated a new Minister of Health, an epidemiologist, who became the first nonphysician to hold this position.

Treatment in place vs evacuation during hospital infections is a critical decision.

The surge of SARS cases at Ho-Ping Hospital led to confusion and panic among patients and hospital staff. Whether to treat its SARS patients on site or to evacuate the hospital was a complex decision and reflected many concerns, including the following: How many wards had been infected? Was there sufficient equipment (eg, respirators) to monitor or treat infected patients? How many isolation beds were available? How many hospital staff were already infected and quarantined? Were they in different wards? Were there neighboring facilities (eg, hospitals, military camps, dorms) available for quarantine?

If a hospital has sufficient capacity to isolate persons under investigation and to treat confirmed patients, on-site treatment is possible. However, evacuation should be considered when there is widespread infection involving different hospital wards and hospital staff. In such cases, patients should be transferred to different facilities based on clinical severity: Patients with new onset fever or respiratory symptoms but who are relatively healthy should be sent to community or regional hospitals with isolation rooms for monitoring; sicker infected patients should be sent to medical centers; and other hospitalized patients (eg, admitted for heart failure) without infection risk or symptoms should seek care elsewhere.

So far, there has been only one instance of hospital-based COVID-19 infections in Taiwan, and the spread of infection was quickly contained within one ward. All nine confirmed cases (including the index patient, one patient in the same ward but a different room, three nurses, one laundry worker, and three members of patients’ families) and their known contacts were identified, then isolated or quarantined individually. Because the affected hospital is part of a complex with more than 3,000 beds, it was big enough to accommodate all infected patients and no evacuation measures were needed. To further reduce potential nodes in the chain of transmission, interns and many other healthcare workers were temporarily relieved of their duties, elective surgeries were canceled, and hospital visitation was limited to immediate family members. The clear communication of intervention measures ensured rapid cooperation and staved off both social panic and further spread of the disease.

Hospitals should establish different flows for different patients.

Having learned from the SARS experience in 2003, hospitals in Taiwan have designated specific pathways to manage patient flow during the COVID-19 outbreak, in addition to checking all patients for travel history and fever: Patients with fever were quickly triaged to a designated fever clinic so they did not mingle with other patients, patients visiting the hospital to obtain chronic disease medications were directed to a “drive-through” lane, patients needing emergent care went through the emergency department, all other regular outpatients were seen in outpatient departments, and visitors of patients were restricted to one visitor per patient at a given time.

Healthcare providers should be organized into blocks and modular teams to avoid hospital-wide infection.

After SARS, Taiwan learned that one way to reduce the spread of something like COVID-19 among healthcare providers and from providers to patients is to divide providers’ work areas into discrete blocks and organize providers into modular teams. This approach was inspired by the design of watertight compartments in ships: Should the hull be breached, flooding is restricted to the breached compartments. Under this organizational strategy, movements of physicians and nurses would be restricted to their designated locations: They would be routinely exposed only to other staff and patients within their division. Doctors and nurses would be asked to practice in modular teams within their blocked locations, reducing the likelihood that infection in one team would spread to another, which could lead to hospital-wide infections. Movement of senior hospital executives would be similarly restricted. Common areas such as cafeterias where people mingle would be closed. Owing to these stringent initiatives, aside from the hospital-­based infection mentioned in Lesson 3, no other hospital-­based infections have been reported in Taiwan so far.

Lessons from previous hospital-based coronavirus infections can be used to minimize future infections.

The authors would like to thank Dr Lee Ming-Liang, former Health Minister of Taiwan and director of that central government’s Anti-SARS Taskforce during the 2003 outbreak, for providing valuable recommendations to this work.

During the severe acute respiratory syndrome (SARS) outbreak in 2003, Taiwan reported 346 confirmed cases and 73 deaths.¹ Of all known infections, 94% were transmitted inside hospitals. Nine major hospitals were fully or partially shut down, and many doctors and nurses quit for fear of becoming infected. The Taipei Municipal Ho-Ping Hospital was most severely affected. Its index patient, a 42-year-old undocumented hospital laundry worker who interacted with staff and patients for 6 days before being hospitalized, became a superspreader, infecting at least 20 other patients and 10 staff members.^2,3 The entire 450-bed hospital was ordered to shut down, and all 930 staff and 240 patients were quarantined within the hospital. The central government appointed the previous Minister of Health as head of the Anti-SARS Taskforce. Ultimately the hospital was evacuated; the outbreak resulted in 26 deaths.² Events surrounding the hospital’s evacuation offer important lessons for hospitals struggling to cope with the COVID-19 pandemic, which has been caused by spread of a similar coronavirus.

Flexibility about case definition is important, as is use of clinical criteria for diagnosis when reliable laboratory tests are not available.

The laundry worker of Ho-Ping Hospital was initially misdiagnosed with infectious enteritis, which delayed proper management and, crucially, isolation from other patients. The low index of suspicion for SARS reflected the initial World Health Organization diagnostic criteria for SARS, which included travel to or residence in an area with recent local transmission of SARS within 10 days of symptom onset.⁴ The laundry worker did not have a recent travel history.³ Additionally, SARS manifested as a lower respiratory tract infection, so many patients were hospitalized for pneumonia before being diagnosed with SARS. Similarly, the Wuhan Municipal Health Commission initially issued diagnostic criteria for COVID-19 that, in addition to fever and symptoms of respiratory infections, emphasized direct exposure to the Huanan Seafood Wholesale Market.⁵ As a result, many cases of COVID-19 were not identified.

Diagnosing SARS was challenging. Early symptoms such as fever and malaise were nonspecific. Polymerase chain reaction tests, although available, were unreliable especially in early stages of the disease and had a high false-negative rate. As cases of SARS increased rapidly, Taiwan began using fever alone for early detection.⁶ Patients and hospital staff received temperature measurements twice daily. Despite the late start to SARS screening, the fever criterion identified many suspected patients, which ensured widespread detection and containment.

For COVID-19, symptoms such as fever, dry cough, and shortness of breath can be used as clinical criteria to triage patients for quarantine in endemic areas when reliable diagnostic tests are not readily available, but all frontline clinical staff should receive daily temperature checks and/or COVID-19 tests, if available, to protect their families and the public.

Ineffective coordination between central and local governments can delay response, but this can be remedied.

During the SARS outbreak, the Taipei City Government and the Taiwan central government were controlled by opposing political parties. Responses to SARS were initially impeded by political skirmishes, which hindered implementation of policies regarding criteria for diagnosis, tracking of suspected cases and their contacts, duration of quarantine, and allocation of resources and facilities for confirmed cases. To avoid further delays, the central government acted swiftly to create the nonpartisan Anti-SARS Taskforce and appointed leaders who could work cordially with both local and central government agencies. To help to deal with the crisis, the central government also designated a new Minister of Health, an epidemiologist, who became the first nonphysician to hold this position.

Treatment in place vs evacuation during hospital infections is a critical decision.

The surge of SARS cases at Ho-Ping Hospital led to confusion and panic among patients and hospital staff. Whether to treat its SARS patients on site or to evacuate the hospital was a complex decision and reflected many concerns, including the following: How many wards had been infected? Was there sufficient equipment (eg, respirators) to monitor or treat infected patients? How many isolation beds were available? How many hospital staff were already infected and quarantined? Were they in different wards? Were there neighboring facilities (eg, hospitals, military camps, dorms) available for quarantine?

If a hospital has sufficient capacity to isolate persons under investigation and to treat confirmed patients, on-site treatment is possible. However, evacuation should be considered when there is widespread infection involving different hospital wards and hospital staff. In such cases, patients should be transferred to different facilities based on clinical severity: Patients with new onset fever or respiratory symptoms but who are relatively healthy should be sent to community or regional hospitals with isolation rooms for monitoring; sicker infected patients should be sent to medical centers; and other hospitalized patients (eg, admitted for heart failure) without infection risk or symptoms should seek care elsewhere.

So far, there has been only one instance of hospital-based COVID-19 infections in Taiwan, and the spread of infection was quickly contained within one ward. All nine confirmed cases (including the index patient, one patient in the same ward but a different room, three nurses, one laundry worker, and three members of patients’ families) and their known contacts were identified, then isolated or quarantined individually. Because the affected hospital is part of a complex with more than 3,000 beds, it was big enough to accommodate all infected patients and no evacuation measures were needed. To further reduce potential nodes in the chain of transmission, interns and many other healthcare workers were temporarily relieved of their duties, elective surgeries were canceled, and hospital visitation was limited to immediate family members. The clear communication of intervention measures ensured rapid cooperation and staved off both social panic and further spread of the disease.

Hospitals should establish different flows for different patients.

Having learned from the SARS experience in 2003, hospitals in Taiwan have designated specific pathways to manage patient flow during the COVID-19 outbreak, in addition to checking all patients for travel history and fever: Patients with fever were quickly triaged to a designated fever clinic so they did not mingle with other patients, patients visiting the hospital to obtain chronic disease medications were directed to a “drive-through” lane, patients needing emergent care went through the emergency department, all other regular outpatients were seen in outpatient departments, and visitors of patients were restricted to one visitor per patient at a given time.

Healthcare providers should be organized into blocks and modular teams to avoid hospital-wide infection.

After SARS, Taiwan learned that one way to reduce the spread of something like COVID-19 among healthcare providers and from providers to patients is to divide providers’ work areas into discrete blocks and organize providers into modular teams. This approach was inspired by the design of watertight compartments in ships: Should the hull be breached, flooding is restricted to the breached compartments. Under this organizational strategy, movements of physicians and nurses would be restricted to their designated locations: They would be routinely exposed only to other staff and patients within their division. Doctors and nurses would be asked to practice in modular teams within their blocked locations, reducing the likelihood that infection in one team would spread to another, which could lead to hospital-wide infections. Movement of senior hospital executives would be similarly restricted. Common areas such as cafeterias where people mingle would be closed. Owing to these stringent initiatives, aside from the hospital-­based infection mentioned in Lesson 3, no other hospital-­based infections have been reported in Taiwan so far.

Lessons from previous hospital-based coronavirus infections can be used to minimize future infections.

The authors would like to thank Dr Lee Ming-Liang, former Health Minister of Taiwan and director of that central government’s Anti-SARS Taskforce during the 2003 outbreak, for providing valuable recommendations to this work.

My colleague asked, “Do you remember that patient?” I froze because, like most emergency physicians, this phrase haunts me. It was the early days of the COVID-19 epidemic, and the story that followed was upsetting. A patient who looked comfortable when I admitted him was intubated hours later by the rapid response team who was called to the floor. All I could think was, “But he looked so comfortable when I admitted him; he was just on a couple of liters of oxygen. Why was he intubated?”

In the days after COVID-19 arrived in our region, there were many such stories of patients sent to the floor from the Emergency Department who were intubated shortly after admission. Many of those patients subsequently endured prolonged and complicated courses on the ventilator. While we would typically use noninvasive modalities such as high-flow nasal cannula (HFNC) or noninvasive ventilation (NIV) for acute respiratory failure, our quickness to intubate was driven by two factors: (1) early reports that noninvasive modalities posed a high risk of failure and subsequent intubation and (2) fear that HFNC and NIV would aerosolize SARS-CoV-2 and unnecessarily expose the heath care team.¹ We would soon find out that our thinking was flawed on both accounts.

When we dug into the evidence for early intubation, we realized that these recommendations were based on a 12-patient series in which 5 patients were trialed on NIV but ultimately intubated and placed on invasive mechanical ventilation (IMV). As the pandemic progressed, more case series and small studies were published, revealing a different picture.² Sun and colleagues reported a multifaceted intervention of 610 inpatients, of whom 10% were critically ill, that identified at-risk patients and used NIV or HFNC and awake proning. Reportedly, fewer than 1% required IMV.³ Similarly, a small study found intubation was avoided in 85% of patients with severe acute respiratory failure caused by COVID-19 with use of HFNC and NIV.⁴ Early findings from New York University in New York, New York, where only 8.5% of patients undergoing IMV were extubated by the time of outcome reporting, suggest early IMV could lead to poor outcomes.⁵

Still, we had concerns about use of HFNC and NIV because of worries about the health and safety of other patients and particularly that of healthcare workers (HCWs) because they have been disproportionately affected by the disease.⁶ Fortunately, we identified emerging data that revealed that HFNC is no more aerosolizing than low-flow nasal cannula or a non­rebreather mask and droplet spread is reduced with a surgical mask.^7,8 In light of these new studies and our own developing experience with the disease, we felt that there was insufficient evidence to continue following the “early intubation” protocol in patients with COVID-19. It was time for a new paradigm.

In order to effectively and quickly change our respiratory pathway for these patients, we initially sought out protocols from other institutions through social media. These protocols, supported by early data from those sites, informed our process. We considered data from various sources, including emergency medicine, hospital medicine, and critical care. We then assembled stakeholders within our organization from emergency medicine, hospital medicine, critical care, and respiratory therapy because our protocol would need endorsement from all key players within our organization who cared for these patients across the potential spectrum of care. We made sure that all stakeholders understood that the quality of the evidence for treatment of this novel disease was much lower than our typical threshold to change practice, but that we aimed to reflect the best evidence to date. We also were careful to identify pathways that would be amenable to near-immediate implementation.

Our group reached consensus within 48 hours and quickly disseminated our first draft of the protocol (Appendix Figure). Dubbed the “Early Intervention Respiratory Protocol,” it differed from usual management in several ways. First, we had consistently observed (and confirmed from the literature) a phenotype of patients with “silent hypoxemia”⁹ (that is, a subset of patients who presented with profound hypoxemia but minimally increased work of breathing). The protocol encouraged tolerance of lower oxygen saturations than is usually seen on inpatient units. This required ensuring all stakeholders were comfortable with a target oxygen saturation of 88%. Second, the protocol leveraged early “awake” proning by patients. Historically, proning is used in mechanically ventilated patients with acute respiratory distress syndrome (ARDS) to improve ventilation-perfusion matching, promote more uniform ventilation, and increase end-expiratory lung volume.¹⁰ Prior literature was limited to the use of awake proning in small case series of ARDS, but given our limitations in terms of ICU capacity, we agreed to trial awake proning in a sizable proportion of our COVID-19 patients outside the ICU.^11,12 Finally, we clarified safe practices regarding the risk of aerosolization with noninvasive modalities. Local infection control determined that HFNC wa not aerosol generating, and use of surgical masks was added for further protection from respiratory droplets. In addition, airborne personal protective equipment was to be worn on the inpatient ward, and we used NIV sparingly and preferentially placed these patients in negative pressure rooms, if available.¹³

Implementation of the protocol involved aggressive dissemination and education (Table). A single-page protocol was designed for ease of use at the bedside that included anticipatory guidance regarding aerosolization and addressed potential resistance to awake proning because of concerns regarding safety and hassle. Departmental leaders disseminated the protocol throughout the institution with tailored education on the rationale and acknowledgment of a reversal in approach. In addition to email, we used text messaging (WhatsApp) and a comprehensive living document (Google Drive) to reach clinicians.

For ease of monitoring and safety, we designated a COVID-19 intermediate care unit. We partnered with the unit medical director, nurse educator, and a focused group of hospitalists, conducting individual train-the-trainer sessions. This training was carried forward, and all nurses, respiratory therapists, and clinicians were trained on the early aggressive respiratory protocol within 12 hours of protocol approval. In addition, the rapid response and critical care teams agreed to round on the COVID-19 intermediate care unit daily.

As a result of these efforts, adoption of the protocol was essentially immediate across the institution. We had shifted the mindset of a diverse group of clinicians regarding how to support the respiratory status of these patients, but also detected reductions in the proportion of patients undergoing IMV and ICU admission (we are planning to report these results separately).

The COVID-19 pandemic has highlighted the importance of having cognitive flexibility when the evidence base is rapidly changing and there is a need for rapid dissemination of knowledge. Even in clinical scenarios with an abundance of high-quality evidence, a gap in knowledge translation on the order of a decade often exists. In contrast, a pandemic involving a novel virus highlights an urgent need for adaptive knowledge translation in the present moment rather than a decade later. In the absence of robust evidence regarding SARS-CoV-2, early management of COVID-19 was based on expert recommendations and experience with other disease processes. Even so, we should anticipate that management paradigms may shift, and we should constantly seek out emerging evidence to adjust our mindset (and protocols like this) accordingly. Our original protocol was based on nearly nonexistent evidence, but we anticipated that, in a pandemic, data would accumulate quickly, so we prioritized rapid translation of new information into practice. In fact, further evidence has emerged regarding the improvement in oxygenation in COVID-19 patients with self-proning.¹⁴

The final step is evaluating the success of both clinical and implementation outcomes. We are attempting to identify changes in intubation, length of stay, days on ventilator, and days in ICU. In addition, we will measure feasibility and adaptability. We are also attempting, in real time, to identify barriers to its use, including conducting qualitative interviews to understand whether there were unintended consequences to use of the protocol. This endeavor highlights how the COVID-19 pandemic, for all its tragedy, may represent an important era for implementation science: a time when emerging literature from a variety of sources can be implemented in days rather than years.

My colleague asked, “Do you remember that patient?” I froze because, like most emergency physicians, this phrase haunts me. It was the early days of the COVID-19 epidemic, and the story that followed was upsetting. A patient who looked comfortable when I admitted him was intubated hours later by the rapid response team who was called to the floor. All I could think was, “But he looked so comfortable when I admitted him; he was just on a couple of liters of oxygen. Why was he intubated?”

In the days after COVID-19 arrived in our region, there were many such stories of patients sent to the floor from the Emergency Department who were intubated shortly after admission. Many of those patients subsequently endured prolonged and complicated courses on the ventilator. While we would typically use noninvasive modalities such as high-flow nasal cannula (HFNC) or noninvasive ventilation (NIV) for acute respiratory failure, our quickness to intubate was driven by two factors: (1) early reports that noninvasive modalities posed a high risk of failure and subsequent intubation and (2) fear that HFNC and NIV would aerosolize SARS-CoV-2 and unnecessarily expose the heath care team.¹ We would soon find out that our thinking was flawed on both accounts.

When we dug into the evidence for early intubation, we realized that these recommendations were based on a 12-patient series in which 5 patients were trialed on NIV but ultimately intubated and placed on invasive mechanical ventilation (IMV). As the pandemic progressed, more case series and small studies were published, revealing a different picture.² Sun and colleagues reported a multifaceted intervention of 610 inpatients, of whom 10% were critically ill, that identified at-risk patients and used NIV or HFNC and awake proning. Reportedly, fewer than 1% required IMV.³ Similarly, a small study found intubation was avoided in 85% of patients with severe acute respiratory failure caused by COVID-19 with use of HFNC and NIV.⁴ Early findings from New York University in New York, New York, where only 8.5% of patients undergoing IMV were extubated by the time of outcome reporting, suggest early IMV could lead to poor outcomes.⁵

Still, we had concerns about use of HFNC and NIV because of worries about the health and safety of other patients and particularly that of healthcare workers (HCWs) because they have been disproportionately affected by the disease.⁶ Fortunately, we identified emerging data that revealed that HFNC is no more aerosolizing than low-flow nasal cannula or a non­rebreather mask and droplet spread is reduced with a surgical mask.^7,8 In light of these new studies and our own developing experience with the disease, we felt that there was insufficient evidence to continue following the “early intubation” protocol in patients with COVID-19. It was time for a new paradigm.

In order to effectively and quickly change our respiratory pathway for these patients, we initially sought out protocols from other institutions through social media. These protocols, supported by early data from those sites, informed our process. We considered data from various sources, including emergency medicine, hospital medicine, and critical care. We then assembled stakeholders within our organization from emergency medicine, hospital medicine, critical care, and respiratory therapy because our protocol would need endorsement from all key players within our organization who cared for these patients across the potential spectrum of care. We made sure that all stakeholders understood that the quality of the evidence for treatment of this novel disease was much lower than our typical threshold to change practice, but that we aimed to reflect the best evidence to date. We also were careful to identify pathways that would be amenable to near-immediate implementation.

Our group reached consensus within 48 hours and quickly disseminated our first draft of the protocol (Appendix Figure). Dubbed the “Early Intervention Respiratory Protocol,” it differed from usual management in several ways. First, we had consistently observed (and confirmed from the literature) a phenotype of patients with “silent hypoxemia”⁹ (that is, a subset of patients who presented with profound hypoxemia but minimally increased work of breathing). The protocol encouraged tolerance of lower oxygen saturations than is usually seen on inpatient units. This required ensuring all stakeholders were comfortable with a target oxygen saturation of 88%. Second, the protocol leveraged early “awake” proning by patients. Historically, proning is used in mechanically ventilated patients with acute respiratory distress syndrome (ARDS) to improve ventilation-perfusion matching, promote more uniform ventilation, and increase end-expiratory lung volume.¹⁰ Prior literature was limited to the use of awake proning in small case series of ARDS, but given our limitations in terms of ICU capacity, we agreed to trial awake proning in a sizable proportion of our COVID-19 patients outside the ICU.^11,12 Finally, we clarified safe practices regarding the risk of aerosolization with noninvasive modalities. Local infection control determined that HFNC wa not aerosol generating, and use of surgical masks was added for further protection from respiratory droplets. In addition, airborne personal protective equipment was to be worn on the inpatient ward, and we used NIV sparingly and preferentially placed these patients in negative pressure rooms, if available.¹³

Implementation of the protocol involved aggressive dissemination and education (Table). A single-page protocol was designed for ease of use at the bedside that included anticipatory guidance regarding aerosolization and addressed potential resistance to awake proning because of concerns regarding safety and hassle. Departmental leaders disseminated the protocol throughout the institution with tailored education on the rationale and acknowledgment of a reversal in approach. In addition to email, we used text messaging (WhatsApp) and a comprehensive living document (Google Drive) to reach clinicians.

For ease of monitoring and safety, we designated a COVID-19 intermediate care unit. We partnered with the unit medical director, nurse educator, and a focused group of hospitalists, conducting individual train-the-trainer sessions. This training was carried forward, and all nurses, respiratory therapists, and clinicians were trained on the early aggressive respiratory protocol within 12 hours of protocol approval. In addition, the rapid response and critical care teams agreed to round on the COVID-19 intermediate care unit daily.

As a result of these efforts, adoption of the protocol was essentially immediate across the institution. We had shifted the mindset of a diverse group of clinicians regarding how to support the respiratory status of these patients, but also detected reductions in the proportion of patients undergoing IMV and ICU admission (we are planning to report these results separately).

The COVID-19 pandemic has highlighted the importance of having cognitive flexibility when the evidence base is rapidly changing and there is a need for rapid dissemination of knowledge. Even in clinical scenarios with an abundance of high-quality evidence, a gap in knowledge translation on the order of a decade often exists. In contrast, a pandemic involving a novel virus highlights an urgent need for adaptive knowledge translation in the present moment rather than a decade later. In the absence of robust evidence regarding SARS-CoV-2, early management of COVID-19 was based on expert recommendations and experience with other disease processes. Even so, we should anticipate that management paradigms may shift, and we should constantly seek out emerging evidence to adjust our mindset (and protocols like this) accordingly. Our original protocol was based on nearly nonexistent evidence, but we anticipated that, in a pandemic, data would accumulate quickly, so we prioritized rapid translation of new information into practice. In fact, further evidence has emerged regarding the improvement in oxygenation in COVID-19 patients with self-proning.¹⁴

The final step is evaluating the success of both clinical and implementation outcomes. We are attempting to identify changes in intubation, length of stay, days on ventilator, and days in ICU. In addition, we will measure feasibility and adaptability. We are also attempting, in real time, to identify barriers to its use, including conducting qualitative interviews to understand whether there were unintended consequences to use of the protocol. This endeavor highlights how the COVID-19 pandemic, for all its tragedy, may represent an important era for implementation science: a time when emerging literature from a variety of sources can be implemented in days rather than years.

My colleague asked, “Do you remember that patient?” I froze because, like most emergency physicians, this phrase haunts me. It was the early days of the COVID-19 epidemic, and the story that followed was upsetting. A patient who looked comfortable when I admitted him was intubated hours later by the rapid response team who was called to the floor. All I could think was, “But he looked so comfortable when I admitted him; he was just on a couple of liters of oxygen. Why was he intubated?”

In the days after COVID-19 arrived in our region, there were many such stories of patients sent to the floor from the Emergency Department who were intubated shortly after admission. Many of those patients subsequently endured prolonged and complicated courses on the ventilator. While we would typically use noninvasive modalities such as high-flow nasal cannula (HFNC) or noninvasive ventilation (NIV) for acute respiratory failure, our quickness to intubate was driven by two factors: (1) early reports that noninvasive modalities posed a high risk of failure and subsequent intubation and (2) fear that HFNC and NIV would aerosolize SARS-CoV-2 and unnecessarily expose the heath care team.¹ We would soon find out that our thinking was flawed on both accounts.

When we dug into the evidence for early intubation, we realized that these recommendations were based on a 12-patient series in which 5 patients were trialed on NIV but ultimately intubated and placed on invasive mechanical ventilation (IMV). As the pandemic progressed, more case series and small studies were published, revealing a different picture.² Sun and colleagues reported a multifaceted intervention of 610 inpatients, of whom 10% were critically ill, that identified at-risk patients and used NIV or HFNC and awake proning. Reportedly, fewer than 1% required IMV.³ Similarly, a small study found intubation was avoided in 85% of patients with severe acute respiratory failure caused by COVID-19 with use of HFNC and NIV.⁴ Early findings from New York University in New York, New York, where only 8.5% of patients undergoing IMV were extubated by the time of outcome reporting, suggest early IMV could lead to poor outcomes.⁵

Still, we had concerns about use of HFNC and NIV because of worries about the health and safety of other patients and particularly that of healthcare workers (HCWs) because they have been disproportionately affected by the disease.⁶ Fortunately, we identified emerging data that revealed that HFNC is no more aerosolizing than low-flow nasal cannula or a non­rebreather mask and droplet spread is reduced with a surgical mask.^7,8 In light of these new studies and our own developing experience with the disease, we felt that there was insufficient evidence to continue following the “early intubation” protocol in patients with COVID-19. It was time for a new paradigm.

In order to effectively and quickly change our respiratory pathway for these patients, we initially sought out protocols from other institutions through social media. These protocols, supported by early data from those sites, informed our process. We considered data from various sources, including emergency medicine, hospital medicine, and critical care. We then assembled stakeholders within our organization from emergency medicine, hospital medicine, critical care, and respiratory therapy because our protocol would need endorsement from all key players within our organization who cared for these patients across the potential spectrum of care. We made sure that all stakeholders understood that the quality of the evidence for treatment of this novel disease was much lower than our typical threshold to change practice, but that we aimed to reflect the best evidence to date. We also were careful to identify pathways that would be amenable to near-immediate implementation.

Our group reached consensus within 48 hours and quickly disseminated our first draft of the protocol (Appendix Figure). Dubbed the “Early Intervention Respiratory Protocol,” it differed from usual management in several ways. First, we had consistently observed (and confirmed from the literature) a phenotype of patients with “silent hypoxemia”⁹ (that is, a subset of patients who presented with profound hypoxemia but minimally increased work of breathing). The protocol encouraged tolerance of lower oxygen saturations than is usually seen on inpatient units. This required ensuring all stakeholders were comfortable with a target oxygen saturation of 88%. Second, the protocol leveraged early “awake” proning by patients. Historically, proning is used in mechanically ventilated patients with acute respiratory distress syndrome (ARDS) to improve ventilation-perfusion matching, promote more uniform ventilation, and increase end-expiratory lung volume.¹⁰ Prior literature was limited to the use of awake proning in small case series of ARDS, but given our limitations in terms of ICU capacity, we agreed to trial awake proning in a sizable proportion of our COVID-19 patients outside the ICU.^11,12 Finally, we clarified safe practices regarding the risk of aerosolization with noninvasive modalities. Local infection control determined that HFNC wa not aerosol generating, and use of surgical masks was added for further protection from respiratory droplets. In addition, airborne personal protective equipment was to be worn on the inpatient ward, and we used NIV sparingly and preferentially placed these patients in negative pressure rooms, if available.¹³

Implementation of the protocol involved aggressive dissemination and education (Table). A single-page protocol was designed for ease of use at the bedside that included anticipatory guidance regarding aerosolization and addressed potential resistance to awake proning because of concerns regarding safety and hassle. Departmental leaders disseminated the protocol throughout the institution with tailored education on the rationale and acknowledgment of a reversal in approach. In addition to email, we used text messaging (WhatsApp) and a comprehensive living document (Google Drive) to reach clinicians.

For ease of monitoring and safety, we designated a COVID-19 intermediate care unit. We partnered with the unit medical director, nurse educator, and a focused group of hospitalists, conducting individual train-the-trainer sessions. This training was carried forward, and all nurses, respiratory therapists, and clinicians were trained on the early aggressive respiratory protocol within 12 hours of protocol approval. In addition, the rapid response and critical care teams agreed to round on the COVID-19 intermediate care unit daily.

As a result of these efforts, adoption of the protocol was essentially immediate across the institution. We had shifted the mindset of a diverse group of clinicians regarding how to support the respiratory status of these patients, but also detected reductions in the proportion of patients undergoing IMV and ICU admission (we are planning to report these results separately).

The COVID-19 pandemic has highlighted the importance of having cognitive flexibility when the evidence base is rapidly changing and there is a need for rapid dissemination of knowledge. Even in clinical scenarios with an abundance of high-quality evidence, a gap in knowledge translation on the order of a decade often exists. In contrast, a pandemic involving a novel virus highlights an urgent need for adaptive knowledge translation in the present moment rather than a decade later. In the absence of robust evidence regarding SARS-CoV-2, early management of COVID-19 was based on expert recommendations and experience with other disease processes. Even so, we should anticipate that management paradigms may shift, and we should constantly seek out emerging evidence to adjust our mindset (and protocols like this) accordingly. Our original protocol was based on nearly nonexistent evidence, but we anticipated that, in a pandemic, data would accumulate quickly, so we prioritized rapid translation of new information into practice. In fact, further evidence has emerged regarding the improvement in oxygenation in COVID-19 patients with self-proning.¹⁴

The final step is evaluating the success of both clinical and implementation outcomes. We are attempting to identify changes in intubation, length of stay, days on ventilator, and days in ICU. In addition, we will measure feasibility and adaptability. We are also attempting, in real time, to identify barriers to its use, including conducting qualitative interviews to understand whether there were unintended consequences to use of the protocol. This endeavor highlights how the COVID-19 pandemic, for all its tragedy, may represent an important era for implementation science: a time when emerging literature from a variety of sources can be implemented in days rather than years.