Hospital Medicine

The early phase of the COVID-19 pandemic was an extraordinarily uncertain, yet innovative, time.¹ Few data describe site-level effects of the many adaptations made to deal with surging case numbers, but studies of larger hospital referral regions (HRR) provide important clues.

In this issue of the Journal of Hospital Medicine, Janke et al² describe how availability of hospital resources in a region relate to COVID-19 mortality between March and June 2020.The authors’ findings suggest that, at least for early periods of the pandemic, having more intensive care unit (ICU), hospital bed, or nursing capacity per COVID-19 case was associated with lower mortality, while physician availability was not. Moreover, months later there were no associations between service or physician availability and HRR COVID-19 mortality. The authors observed variations in mortality rates in places commonly thought to have been overwhelmed early in the pandemic (April 2020), as well as in cities (Boston, Philadelphia, Hartford, Detroit, and Camden, New Jersey) that had a less prominent place in the news at that time.

Larger hospitals tend to have the resources necessary to make wholesale changes when preparing for a pandemic wave. Thus, Janke et al’s results may not have fully captured the pandemic’s potential impact in settings with fewer resources or in smaller hospitals, which are currently being overwhelmed.³

The number of cases and hospitalizations in this third wave of COVID-19 continues to rise, and the strain on healthcare resources has been felt across entire regions, making the results of this study even more salient. Hospital outcomes for COVID-19 are sensitive to limitations in physical locations (number of beds, ICU capacity) and nursing capacity. Nurses more often are assigned specifically to a bed or unit, and the number of patients per nurse is limited by state or local statute. Innovations such as COVID-19 field hospitals or redeploying existing beds (eg, converting postanesthesia care units to ICUs) offset physically constrained resources.⁴ On the other hand, lower acuity in this phase of the pandemic (eg, fewer ICU admissions) and shorter lengths of stay may produce higher turnover, producing more workforce stress, regardless of bed availability.

Early work of our COVID-19 collaborative⁵ suggests that the focus on localizing patients to geographic units or teams has given way to strategies that utilize more flexible team and bed-finding approaches. Clinical care has evolved to focus on more aggressive discharge strategies, with remote monitoring and hospital-at-home capabilities. Overall, the pandemic is providing fodder for future studies examining interaction between case volumes, physician and nurse availability, and evolution in clinical care practices. Most critically, it provides an opportunity to study health system flexibility and robustness with a lens that incorporates a view of the hospital and its surroundings as tightly related parts of care delivery. Because if there is one thing the pandemic is teaching us, it is that, more than ever, no hospital can be an island unto itself, and each hospital is part of a larger ecosystem where rising tides are felt throughout.

The early phase of the COVID-19 pandemic was an extraordinarily uncertain, yet innovative, time.¹ Few data describe site-level effects of the many adaptations made to deal with surging case numbers, but studies of larger hospital referral regions (HRR) provide important clues.

In this issue of the Journal of Hospital Medicine, Janke et al² describe how availability of hospital resources in a region relate to COVID-19 mortality between March and June 2020.The authors’ findings suggest that, at least for early periods of the pandemic, having more intensive care unit (ICU), hospital bed, or nursing capacity per COVID-19 case was associated with lower mortality, while physician availability was not. Moreover, months later there were no associations between service or physician availability and HRR COVID-19 mortality. The authors observed variations in mortality rates in places commonly thought to have been overwhelmed early in the pandemic (April 2020), as well as in cities (Boston, Philadelphia, Hartford, Detroit, and Camden, New Jersey) that had a less prominent place in the news at that time.

Larger hospitals tend to have the resources necessary to make wholesale changes when preparing for a pandemic wave. Thus, Janke et al’s results may not have fully captured the pandemic’s potential impact in settings with fewer resources or in smaller hospitals, which are currently being overwhelmed.³

The number of cases and hospitalizations in this third wave of COVID-19 continues to rise, and the strain on healthcare resources has been felt across entire regions, making the results of this study even more salient. Hospital outcomes for COVID-19 are sensitive to limitations in physical locations (number of beds, ICU capacity) and nursing capacity. Nurses more often are assigned specifically to a bed or unit, and the number of patients per nurse is limited by state or local statute. Innovations such as COVID-19 field hospitals or redeploying existing beds (eg, converting postanesthesia care units to ICUs) offset physically constrained resources.⁴ On the other hand, lower acuity in this phase of the pandemic (eg, fewer ICU admissions) and shorter lengths of stay may produce higher turnover, producing more workforce stress, regardless of bed availability.

Early work of our COVID-19 collaborative⁵ suggests that the focus on localizing patients to geographic units or teams has given way to strategies that utilize more flexible team and bed-finding approaches. Clinical care has evolved to focus on more aggressive discharge strategies, with remote monitoring and hospital-at-home capabilities. Overall, the pandemic is providing fodder for future studies examining interaction between case volumes, physician and nurse availability, and evolution in clinical care practices. Most critically, it provides an opportunity to study health system flexibility and robustness with a lens that incorporates a view of the hospital and its surroundings as tightly related parts of care delivery. Because if there is one thing the pandemic is teaching us, it is that, more than ever, no hospital can be an island unto itself, and each hospital is part of a larger ecosystem where rising tides are felt throughout.

The early phase of the COVID-19 pandemic was an extraordinarily uncertain, yet innovative, time.¹ Few data describe site-level effects of the many adaptations made to deal with surging case numbers, but studies of larger hospital referral regions (HRR) provide important clues.

In this issue of the Journal of Hospital Medicine, Janke et al² describe how availability of hospital resources in a region relate to COVID-19 mortality between March and June 2020.The authors’ findings suggest that, at least for early periods of the pandemic, having more intensive care unit (ICU), hospital bed, or nursing capacity per COVID-19 case was associated with lower mortality, while physician availability was not. Moreover, months later there were no associations between service or physician availability and HRR COVID-19 mortality. The authors observed variations in mortality rates in places commonly thought to have been overwhelmed early in the pandemic (April 2020), as well as in cities (Boston, Philadelphia, Hartford, Detroit, and Camden, New Jersey) that had a less prominent place in the news at that time.

Larger hospitals tend to have the resources necessary to make wholesale changes when preparing for a pandemic wave. Thus, Janke et al’s results may not have fully captured the pandemic’s potential impact in settings with fewer resources or in smaller hospitals, which are currently being overwhelmed.³

The number of cases and hospitalizations in this third wave of COVID-19 continues to rise, and the strain on healthcare resources has been felt across entire regions, making the results of this study even more salient. Hospital outcomes for COVID-19 are sensitive to limitations in physical locations (number of beds, ICU capacity) and nursing capacity. Nurses more often are assigned specifically to a bed or unit, and the number of patients per nurse is limited by state or local statute. Innovations such as COVID-19 field hospitals or redeploying existing beds (eg, converting postanesthesia care units to ICUs) offset physically constrained resources.⁴ On the other hand, lower acuity in this phase of the pandemic (eg, fewer ICU admissions) and shorter lengths of stay may produce higher turnover, producing more workforce stress, regardless of bed availability.

Early work of our COVID-19 collaborative⁵ suggests that the focus on localizing patients to geographic units or teams has given way to strategies that utilize more flexible team and bed-finding approaches. Clinical care has evolved to focus on more aggressive discharge strategies, with remote monitoring and hospital-at-home capabilities. Overall, the pandemic is providing fodder for future studies examining interaction between case volumes, physician and nurse availability, and evolution in clinical care practices. Most critically, it provides an opportunity to study health system flexibility and robustness with a lens that incorporates a view of the hospital and its surroundings as tightly related parts of care delivery. Because if there is one thing the pandemic is teaching us, it is that, more than ever, no hospital can be an island unto itself, and each hospital is part of a larger ecosystem where rising tides are felt throughout.

Several studies have examined variation in outcomes of patients with COVID-19, with emphasis on hospital-level factors such as geographic location, workforce and resource availability, and COVID-19 community prevalence.^1,2 Block et al¹ examine variation in COVID-19 mortality across 117 US hospitals, exploring whether COVID-19 admission volume was associated with mortality. While their results suggest that patients admitted to hospitals in the highest quintiles of COVID-19 caseload had higher odds of in-hospital death, the authors were not able to fully adjust for severity of illness, tempering our ability to draw conclusions. However, their finding is consistent with work showing that emergency department crowding and high hospital utilization are associated with excess mortality.

Block et al¹ also found variation within quintiles of COVID-19 burden, suggesting that other hospital-level factors are influencing their performance. In response to the initial surge of COVID-19 in the United States, hospitals and healthcare systems made rapid, often major, adjustments to provide care. Four interdependent components contribute to an effective surge response: system, space, staff, and supplies. Although all four components are important, effective systems are critical. Systems domains include command, or the creation of leadership teams throughout the organization; control, or management, of infrastructure; communication of rapid, comprehensible messages internally and externally; coordination of resources across departments and professions; and continuity of operations.³ Little is known about how well hospitals have implemented these systems components throughout the pandemic, and while Janke et al² examined the association of resources with outcomes, neither their study nor Block et al’s was able to account for other organizational or systems-based aspects of surge response.

Studies that help us understand the organizational factors and care-delivery adaptations associated with better outcomes for patients with COVID-19 are sorely needed, and could provide important insights for organizational adaptation and change more generally. Janke et al² and, in their accompanying editorial, Auerbach and Greysen,⁴ call for “innovative protocols” and “flexibility” to meet the needs of high-demand, novel situations. However, organizations’ ability to innovate and adapt relies on their relationships and teamwork capability.

The relational infrastructure within an organization provides the basis for effective teamwork, facilitating other aspects of an organization’s surge response and ability to adapt. Relationships characterized by trust and mindfulness create a context of psychological safety that encourages sharing new ideas, and enable teams to rapidly make sense of new situations and create shared understandings that facilitate effective action: improvising, adapting, and learning. Trust and psychological safety are especially important during crises, as decision-making tends to evolve toward top-down processes in times of crisis.

Hospitals currently collect few data that speak to relationships and teamwork, limiting our ability to study these vital organizational characteristics and their role in the larger COVID-19 response. Surveys related to patient safety culture or provider wellness and burnout are likely the only data regularly collected by hospitals. Expanding these data to include measures of relational infrastructure will create more robust data not only to conduct research regarding organizational factors that are associated with patient outcomes, but also to allow health systems to improve relationships and teaming as a means of improving outcomes. Examples include relational coordination,⁵ relationships,⁶and learning scales.⁷

The hospitals to which patients are admitted make a difference in patient survival. The study by Block et al¹ highlights the importance of assessing the factors that enable health systems to adapt and innovate so that we can better understand hospital-level variation in outcomes.

Several studies have examined variation in outcomes of patients with COVID-19, with emphasis on hospital-level factors such as geographic location, workforce and resource availability, and COVID-19 community prevalence.^1,2 Block et al¹ examine variation in COVID-19 mortality across 117 US hospitals, exploring whether COVID-19 admission volume was associated with mortality. While their results suggest that patients admitted to hospitals in the highest quintiles of COVID-19 caseload had higher odds of in-hospital death, the authors were not able to fully adjust for severity of illness, tempering our ability to draw conclusions. However, their finding is consistent with work showing that emergency department crowding and high hospital utilization are associated with excess mortality.

Block et al¹ also found variation within quintiles of COVID-19 burden, suggesting that other hospital-level factors are influencing their performance. In response to the initial surge of COVID-19 in the United States, hospitals and healthcare systems made rapid, often major, adjustments to provide care. Four interdependent components contribute to an effective surge response: system, space, staff, and supplies. Although all four components are important, effective systems are critical. Systems domains include command, or the creation of leadership teams throughout the organization; control, or management, of infrastructure; communication of rapid, comprehensible messages internally and externally; coordination of resources across departments and professions; and continuity of operations.³ Little is known about how well hospitals have implemented these systems components throughout the pandemic, and while Janke et al² examined the association of resources with outcomes, neither their study nor Block et al’s was able to account for other organizational or systems-based aspects of surge response.

Studies that help us understand the organizational factors and care-delivery adaptations associated with better outcomes for patients with COVID-19 are sorely needed, and could provide important insights for organizational adaptation and change more generally. Janke et al² and, in their accompanying editorial, Auerbach and Greysen,⁴ call for “innovative protocols” and “flexibility” to meet the needs of high-demand, novel situations. However, organizations’ ability to innovate and adapt relies on their relationships and teamwork capability.

The relational infrastructure within an organization provides the basis for effective teamwork, facilitating other aspects of an organization’s surge response and ability to adapt. Relationships characterized by trust and mindfulness create a context of psychological safety that encourages sharing new ideas, and enable teams to rapidly make sense of new situations and create shared understandings that facilitate effective action: improvising, adapting, and learning. Trust and psychological safety are especially important during crises, as decision-making tends to evolve toward top-down processes in times of crisis.

Hospitals currently collect few data that speak to relationships and teamwork, limiting our ability to study these vital organizational characteristics and their role in the larger COVID-19 response. Surveys related to patient safety culture or provider wellness and burnout are likely the only data regularly collected by hospitals. Expanding these data to include measures of relational infrastructure will create more robust data not only to conduct research regarding organizational factors that are associated with patient outcomes, but also to allow health systems to improve relationships and teaming as a means of improving outcomes. Examples include relational coordination,⁵ relationships,⁶and learning scales.⁷

The hospitals to which patients are admitted make a difference in patient survival. The study by Block et al¹ highlights the importance of assessing the factors that enable health systems to adapt and innovate so that we can better understand hospital-level variation in outcomes.

Several studies have examined variation in outcomes of patients with COVID-19, with emphasis on hospital-level factors such as geographic location, workforce and resource availability, and COVID-19 community prevalence.^1,2 Block et al¹ examine variation in COVID-19 mortality across 117 US hospitals, exploring whether COVID-19 admission volume was associated with mortality. While their results suggest that patients admitted to hospitals in the highest quintiles of COVID-19 caseload had higher odds of in-hospital death, the authors were not able to fully adjust for severity of illness, tempering our ability to draw conclusions. However, their finding is consistent with work showing that emergency department crowding and high hospital utilization are associated with excess mortality.

Block et al¹ also found variation within quintiles of COVID-19 burden, suggesting that other hospital-level factors are influencing their performance. In response to the initial surge of COVID-19 in the United States, hospitals and healthcare systems made rapid, often major, adjustments to provide care. Four interdependent components contribute to an effective surge response: system, space, staff, and supplies. Although all four components are important, effective systems are critical. Systems domains include command, or the creation of leadership teams throughout the organization; control, or management, of infrastructure; communication of rapid, comprehensible messages internally and externally; coordination of resources across departments and professions; and continuity of operations.³ Little is known about how well hospitals have implemented these systems components throughout the pandemic, and while Janke et al² examined the association of resources with outcomes, neither their study nor Block et al’s was able to account for other organizational or systems-based aspects of surge response.

Studies that help us understand the organizational factors and care-delivery adaptations associated with better outcomes for patients with COVID-19 are sorely needed, and could provide important insights for organizational adaptation and change more generally. Janke et al² and, in their accompanying editorial, Auerbach and Greysen,⁴ call for “innovative protocols” and “flexibility” to meet the needs of high-demand, novel situations. However, organizations’ ability to innovate and adapt relies on their relationships and teamwork capability.

The relational infrastructure within an organization provides the basis for effective teamwork, facilitating other aspects of an organization’s surge response and ability to adapt. Relationships characterized by trust and mindfulness create a context of psychological safety that encourages sharing new ideas, and enable teams to rapidly make sense of new situations and create shared understandings that facilitate effective action: improvising, adapting, and learning. Trust and psychological safety are especially important during crises, as decision-making tends to evolve toward top-down processes in times of crisis.

Hospitals currently collect few data that speak to relationships and teamwork, limiting our ability to study these vital organizational characteristics and their role in the larger COVID-19 response. Surveys related to patient safety culture or provider wellness and burnout are likely the only data regularly collected by hospitals. Expanding these data to include measures of relational infrastructure will create more robust data not only to conduct research regarding organizational factors that are associated with patient outcomes, but also to allow health systems to improve relationships and teaming as a means of improving outcomes. Examples include relational coordination,⁵ relationships,⁶and learning scales.⁷

The hospitals to which patients are admitted make a difference in patient survival. The study by Block et al¹ highlights the importance of assessing the factors that enable health systems to adapt and innovate so that we can better understand hospital-level variation in outcomes.

Since the onset of the COVID-19 pandemic, the proclivity of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) for adults and its relative sparing of pediatric populations has been well characterized. Accordingly, policymakers have devoted significant attention to SARS-CoV-2’s impact on adult hospitals. Less consideration, however, has been given to children’s hospitals, which responded to the pandemic by suspending noncritical care encounters, conserving personal protective equipment, and implementing alternative care models.¹ While important, these strategic decisions may threaten the financial health of children’s hospitals.

In this issue of the Journal of Hospital Medicine, Synhorst et al¹ describe the impact of COVID-19 on US children’s hospitals.The authors utilized the Children’s Hospital Association’s PROSPECT database to compare year-over-year trends in healthcare encounters and hospital charges before and during the COVID-19 pandemic at 26 tertiary hospitals. The analysis focused on the first wave of COVID-19 in the United States from February through June 2020.

The results are staggering. Compared with 2019, the authors found significant decreases in healthcare encounters for all children’s hospitals beginning in March 2020, with a nadir in mid-April (corresponding to the first peak in adult hospitalizations). Inpatient bed days, emergency department (ED) visits, and surgeries decreased by a median of 36%, 65%, and 77%, respectively, per hospital during the nadir. Charges from February 1 to June 30, 2020, decreased by a median 24% per children’s hospital as compared to 2019—corresponding to a median $276 million decrease in charges per hospital. A quarter of hospitals faced more than $400 million in lost charges.¹

Why do these trends matter? Large decreases in utilization and associated charges likely represent significant unmet demand for child healthcare for both acute and chronic disease management. For example, with limited in-person evaluation available at the onset of illness, caregivers are presenting to EDs with sicker children.² With a shift to virtual care, clinicians may miss signs of child abuse from violence in the home—which can escalate during isolation.³ Children with chronic conditions may also be left without surveillance mechanisms, which may partly explain the autumn 2020 surge in acute mental health-related ED presentations.⁴ Furthermore, telemedicine may exacerbate care inequities for vulnerable populations lacking resources and/or English proficiency.

There is also a larger policy perspective to consider in evaluating these data: Because children’s hospitals largely operate in a fee-for-service reimbursement model, they often rely on marginal revenues to support mission-driven programming. In other words, revenue streams from profitable care segments (eg, elective surgeries) often help sustain institutional platforms operating at a loss, such as community safety net programs. Consequently, threats to marginal revenues can place mission-driven programming in jeopardy of being reduced or terminated.

The Synhorst et al¹ study was limited to hospital charges, which likely overestimate revenue losses based on actual reimbursements. Yet, this is the first study to quantify COVID-19’s financial toll on children’s hospitals, and charges offer a reasonable proxy for balance sheet trends. Thus, it is safe to assume that most hospitals incurred substantial losses during the 2020 fiscal year. Unfortunately, as the authors highlight, these losses differentially impacted hospitals based on existing resources¹—so some hospitals were likely forced to cut programs or reduce staff in an effort to return to profitability. In this way, COVID-19 has exposed the fragility of the fee-for-service model that children’s hospitals rely on for both patients and staff.

Children’s hospitals and the services they provide are essential to the health and well-being of children. The critical losses sustained by children’s hospitals due to COVID-19 threaten their ability to promote child health in the near and long term, with the greatest risk to vulnerable populations. Policymakers must act now to preserve these essential services for children.

Since the onset of the COVID-19 pandemic, the proclivity of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) for adults and its relative sparing of pediatric populations has been well characterized. Accordingly, policymakers have devoted significant attention to SARS-CoV-2’s impact on adult hospitals. Less consideration, however, has been given to children’s hospitals, which responded to the pandemic by suspending noncritical care encounters, conserving personal protective equipment, and implementing alternative care models.¹ While important, these strategic decisions may threaten the financial health of children’s hospitals.

In this issue of the Journal of Hospital Medicine, Synhorst et al¹ describe the impact of COVID-19 on US children’s hospitals.The authors utilized the Children’s Hospital Association’s PROSPECT database to compare year-over-year trends in healthcare encounters and hospital charges before and during the COVID-19 pandemic at 26 tertiary hospitals. The analysis focused on the first wave of COVID-19 in the United States from February through June 2020.

The results are staggering. Compared with 2019, the authors found significant decreases in healthcare encounters for all children’s hospitals beginning in March 2020, with a nadir in mid-April (corresponding to the first peak in adult hospitalizations). Inpatient bed days, emergency department (ED) visits, and surgeries decreased by a median of 36%, 65%, and 77%, respectively, per hospital during the nadir. Charges from February 1 to June 30, 2020, decreased by a median 24% per children’s hospital as compared to 2019—corresponding to a median $276 million decrease in charges per hospital. A quarter of hospitals faced more than $400 million in lost charges.¹

Why do these trends matter? Large decreases in utilization and associated charges likely represent significant unmet demand for child healthcare for both acute and chronic disease management. For example, with limited in-person evaluation available at the onset of illness, caregivers are presenting to EDs with sicker children.² With a shift to virtual care, clinicians may miss signs of child abuse from violence in the home—which can escalate during isolation.³ Children with chronic conditions may also be left without surveillance mechanisms, which may partly explain the autumn 2020 surge in acute mental health-related ED presentations.⁴ Furthermore, telemedicine may exacerbate care inequities for vulnerable populations lacking resources and/or English proficiency.

There is also a larger policy perspective to consider in evaluating these data: Because children’s hospitals largely operate in a fee-for-service reimbursement model, they often rely on marginal revenues to support mission-driven programming. In other words, revenue streams from profitable care segments (eg, elective surgeries) often help sustain institutional platforms operating at a loss, such as community safety net programs. Consequently, threats to marginal revenues can place mission-driven programming in jeopardy of being reduced or terminated.

The Synhorst et al¹ study was limited to hospital charges, which likely overestimate revenue losses based on actual reimbursements. Yet, this is the first study to quantify COVID-19’s financial toll on children’s hospitals, and charges offer a reasonable proxy for balance sheet trends. Thus, it is safe to assume that most hospitals incurred substantial losses during the 2020 fiscal year. Unfortunately, as the authors highlight, these losses differentially impacted hospitals based on existing resources¹—so some hospitals were likely forced to cut programs or reduce staff in an effort to return to profitability. In this way, COVID-19 has exposed the fragility of the fee-for-service model that children’s hospitals rely on for both patients and staff.

Children’s hospitals and the services they provide are essential to the health and well-being of children. The critical losses sustained by children’s hospitals due to COVID-19 threaten their ability to promote child health in the near and long term, with the greatest risk to vulnerable populations. Policymakers must act now to preserve these essential services for children.

Since the onset of the COVID-19 pandemic, the proclivity of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) for adults and its relative sparing of pediatric populations has been well characterized. Accordingly, policymakers have devoted significant attention to SARS-CoV-2’s impact on adult hospitals. Less consideration, however, has been given to children’s hospitals, which responded to the pandemic by suspending noncritical care encounters, conserving personal protective equipment, and implementing alternative care models.¹ While important, these strategic decisions may threaten the financial health of children’s hospitals.

In this issue of the Journal of Hospital Medicine, Synhorst et al¹ describe the impact of COVID-19 on US children’s hospitals.The authors utilized the Children’s Hospital Association’s PROSPECT database to compare year-over-year trends in healthcare encounters and hospital charges before and during the COVID-19 pandemic at 26 tertiary hospitals. The analysis focused on the first wave of COVID-19 in the United States from February through June 2020.

The results are staggering. Compared with 2019, the authors found significant decreases in healthcare encounters for all children’s hospitals beginning in March 2020, with a nadir in mid-April (corresponding to the first peak in adult hospitalizations). Inpatient bed days, emergency department (ED) visits, and surgeries decreased by a median of 36%, 65%, and 77%, respectively, per hospital during the nadir. Charges from February 1 to June 30, 2020, decreased by a median 24% per children’s hospital as compared to 2019—corresponding to a median $276 million decrease in charges per hospital. A quarter of hospitals faced more than $400 million in lost charges.¹

Why do these trends matter? Large decreases in utilization and associated charges likely represent significant unmet demand for child healthcare for both acute and chronic disease management. For example, with limited in-person evaluation available at the onset of illness, caregivers are presenting to EDs with sicker children.² With a shift to virtual care, clinicians may miss signs of child abuse from violence in the home—which can escalate during isolation.³ Children with chronic conditions may also be left without surveillance mechanisms, which may partly explain the autumn 2020 surge in acute mental health-related ED presentations.⁴ Furthermore, telemedicine may exacerbate care inequities for vulnerable populations lacking resources and/or English proficiency.

There is also a larger policy perspective to consider in evaluating these data: Because children’s hospitals largely operate in a fee-for-service reimbursement model, they often rely on marginal revenues to support mission-driven programming. In other words, revenue streams from profitable care segments (eg, elective surgeries) often help sustain institutional platforms operating at a loss, such as community safety net programs. Consequently, threats to marginal revenues can place mission-driven programming in jeopardy of being reduced or terminated.

The Synhorst et al¹ study was limited to hospital charges, which likely overestimate revenue losses based on actual reimbursements. Yet, this is the first study to quantify COVID-19’s financial toll on children’s hospitals, and charges offer a reasonable proxy for balance sheet trends. Thus, it is safe to assume that most hospitals incurred substantial losses during the 2020 fiscal year. Unfortunately, as the authors highlight, these losses differentially impacted hospitals based on existing resources¹—so some hospitals were likely forced to cut programs or reduce staff in an effort to return to profitability. In this way, COVID-19 has exposed the fragility of the fee-for-service model that children’s hospitals rely on for both patients and staff.

Children’s hospitals and the services they provide are essential to the health and well-being of children. The critical losses sustained by children’s hospitals due to COVID-19 threaten their ability to promote child health in the near and long term, with the greatest risk to vulnerable populations. Policymakers must act now to preserve these essential services for children.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.