Hospital Medicine

Inspired by the ABIM Foundation’s Choosing Wisel y ^® campaign, the “Things We Do for No Reason ^™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A hospitalist admits a 68-year-old woman for community-acquired pneumonia with a past medical history of hypertension, gastroesophageal reflux disease, and osteoarthritis. Her hospitalist consults physical therapy to maximize mobility; continues her home medications including pantoprazole, hydrochlorothiazide, and acetaminophen; and initiates antimicrobial therapy with ceftriaxone and azithromycin. The hospital admission order set requires administration of subcutaneous unfractionated heparin for venous thromboembolism chemoprophylaxis.

Venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), ranks among the leading preventable causes of morbidity and mortality in hospitalized patients.¹ DVTs can rapidly progress to a PE, which account for 5% to 10% of in-hospital deaths.¹ The negative sequelae of in-hospital VTE, including prolonged hospital stay, increased healthcare costs, and greater risks associated with pharmacologic treatment, add $9 to $18.2 billion in US healthcare expenditures each year.² Various risk-assessment models (RAMs) identify medical patients at high risk for developing VTE based on the presence of risk factors including acute heart failure, prior history of VTE, and reduced mobility.³ Since hospitalization may itself increase the risk for VTE, medical patients often receive universal chemoprophylaxis with anticoagulants such as unfractionated heparin (UFH), low-molecular-weight heparin (LMWH), or fondaparinux.³ A meta-analysis of randomized controlled trials (RCTs) published by Wein et al supports the use of VTE chemoprophylaxis in high-risk patients.⁴ It showed statistically significant reductions in rates of PE in high-risk hospitalized medical patients with UFH (risk ratio [RR], 0.64; 95% CI, 0.50-0.82) or LMWH chemoprophylaxis (RR, 0.37; 95% CI, 0.21-0.64), compared with controls.

In recognition of the magnitude of the problem, national organizations have emphasized routine chemoprophylaxis for prevention of in-hospital VTE as a top-priority measure for patient safety.^5,6 The Joint Commission includes chemoprophylaxis as a quality core metric and failure to adhere to such standards compromises hospital accreditation.⁵ Since 2008, the Centers for Medicare & Medicaid Services no longer reimburses hospitals for preventable VTE and requires institutions to document the rationale for omitting chemoprophylaxis if not commenced on hospital admission.⁶

In order to understand why chemoprophylaxis fails to benefit low-risk medical patients, it is necessary to critically examine the benefits identified in trials of high-risk patients. Although RCTs and meta-analysis of chemoprophylaxis have consistently demonstrated a reduction in VTE, prevention of asymptomatic VTE identified on screening with ultrasound or venography accounts for more than 90% of the composite outcome in the three key trials.^7-9 Hospitalists do not routinely screen for asymptomatic VTE, and incorporation of these events into composite VTE outcomes inflates the magnitude of benefit gained by chemoprophylaxis. Importantly, the standard of care does not include screening for asymptomatic DVTs, and studies have estimated that only 10% to 15% of asymptomatic DVTs progress to a symptomatic VTE.¹⁰

A meta-analysis of trials evaluating unselected general medical patients (ie, not those with specific high-risk conditions such as acute myocardial infarction) did not show a reduction in symptomatic VTE with chemoprophylaxis (odds ratio [OR], 0.59; 95% CI, 0.29-1.23).¹¹ In the meta-analysis by Wein et al, which did include patients with specific high-risk conditions, chemoprophylaxis produced a small absolute risk reduction, resulting in a number needed to treat (NNT) of 345 to prevent one PE.⁴ This demonstrates that, even in high-risk patients, the magnitude of benefit is small. Population-level data also question the benefit of chemoprophylaxis. Flanders et al stratified 35 Michigan hospitals into high-, moderate-, and low-performance tertiles, with performance based on the rate of chemoprophylaxis use on admission for general medical patients at high-risk for VTE. The authors found no significant difference in the rate of VTE at 90 days among tertiles.¹² These findings question the usefulness of universal chemoprophylaxis when applied in a real-world setting.

The high rates of VTE in the absence of chemoprophylaxis reported in historic trials may overestimate the contemporary risk. A 2019 multicenter, observational study examined the rate of hospital-acquired DVT for 1,170 low- and high-risk patients with acute medical illness admitted to the internal medicine ward.¹³ Of them, 250 (21%) underwent prophylaxis with parenteral anticoagulants (mean Padua Prediction Score, 4.5). The remaining 920 (79%) were not treated with prophylaxis (mean Padua Prediction Score, 2.5). All patients underwent ultrasound at admission and discharge. The average length of stay was 13 days, and just three patients (0.3%) experienced in-hospital DVT, two of whom were receiving chemoprophylaxis. Only one (0.09%) DVT was symptomatic.

It should be emphasized that any evidence favoring chemoprophylaxis comes from studies of patients at high-risk of VTE. No data show benefit for low-risk patients. Therefore, any risk of chemoprophylaxis likely outweighs the benefits in low-risk patients. Importantly, the risks are underappreciated. A 2014 meta-analysis reported an increased risk of major hemorrhage (OR, 1.81; 95% CI, 1.10-2.98; P = .02) in high-risk medically ill patients on chemoprophylaxis.¹⁴ This results in a number needed to harm for major bleeding of 336, a value similar to the NNT for benefit reported by Wein et al.⁴ Heparin-induced thrombocytopenia, a potentially limb- and life-threatening complication of UFH or LMWH exposure, has an overall incidence of 0.3% to 0.7% in hospitalized patients on chemoprophylaxis.³ Finally, the most commonly used chemoprophylaxis medications are administered subcutaneously, resulting in injection site pain. Unsurprisingly, hospitalized patients refuse chemoprophylaxis more frequently than any other medication.¹⁵

The negative implications of inappropriate chemoprophylaxis extend beyond direct harms to patients. Poor stratification and overuse results in unnecessary healthcare costs. One single-center retrospective review demonstrated that, after integration of chemoprophylaxis into hospital order sets, 76% of patients received unnecessary administration of chemoprophylaxis, resulting in an annualized expenditure of $77,652.¹⁶ This does not take into account costs associated with major bleeds.

Unfortunately, the pendulum has shifted from an era of underprescribing chemoprophylaxis to hospitalized medical patients to one of overprescribing. Data published in 2018 suggest that providers overuse chemoprophylaxis in low-risk medical patients at more than double the rate of underusing it in high-risk patients (57% vs 21%).¹⁷

Several national societies, including the often cited American College of Chest Physicians (ACCP) and American Society of Hematology (ASH), provide guidance on the use of VTE chemoprophylaxis in acutely ill medical inpatients.^3,18 The ASH guidelines conditionally recommend VTE chemoprophylaxis rather than no chemoprophylaxis.¹⁸ However, the guidelines do not provide guidance on a risk-stratified approach and disclose that this recommendation is supported by a low certainty in the evidence of the net health benefit gained.¹⁸ Guidelines from ACCP lean towards individualized care and recommend against the use of VTE chemoprophylaxis for hospitalized acutely ill, low-risk medical patients.³

Clinicians should risk stratify using validated RAMs when making a patient-centered treatment plan on admission. The table outlines the most common RAMs with evidence for use in acute medically ill hospitalized patients. Although RAMs have limitations (eg, lack of prospective validation and complexity), the ACCP guidelines advocate for their use.³

Given that immobility independently increases risk for VTE, early mobilization is a simple and cost-effective way to potentially prevent VTE in low-risk patients. In addition to this potential benefit, early mobilization shortens the length of hospital stay, improves functional status and rates of delirium in hospitalized elderly patients, and hastens postoperative recovery after major surgeries.¹⁹

• Incorporate a patient-centered, risk-stratified approach to identify low-risk patients. This can be done manually or with use of RAMS embedded in the electronic health record.
• Do not prescribe chemoprophylaxis to low-risk hospitalized medical patients.
• Emphasize the importance of early mobilization in hospitalized patients.

In regard to the case, the hospitalist should use a RAM developed for the nonsurgical, non–critically ill patient to determine her need for chemoprophylaxis. Based on the clinical data presented, the three RAMs available would classify the patient as low risk for developing an in-hospital VTE. She should not receive chemoprophylaxis given the lack of data demonstrating benefit in this population. To mitigate the potential risk of bleeding, heparin-induced thrombocytopenia, and painful injections, the hospitalist should discontinue heparin. The hospitalist should advocate for early mobilization and minimize the duration of hospital stay as appropriate.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason^™”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by emailing [email protected].

Inspired by the ABIM Foundation’s Choosing Wisel y ^® campaign, the “Things We Do for No Reason ^™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A hospitalist admits a 68-year-old woman for community-acquired pneumonia with a past medical history of hypertension, gastroesophageal reflux disease, and osteoarthritis. Her hospitalist consults physical therapy to maximize mobility; continues her home medications including pantoprazole, hydrochlorothiazide, and acetaminophen; and initiates antimicrobial therapy with ceftriaxone and azithromycin. The hospital admission order set requires administration of subcutaneous unfractionated heparin for venous thromboembolism chemoprophylaxis.

Venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), ranks among the leading preventable causes of morbidity and mortality in hospitalized patients.¹ DVTs can rapidly progress to a PE, which account for 5% to 10% of in-hospital deaths.¹ The negative sequelae of in-hospital VTE, including prolonged hospital stay, increased healthcare costs, and greater risks associated with pharmacologic treatment, add $9 to $18.2 billion in US healthcare expenditures each year.² Various risk-assessment models (RAMs) identify medical patients at high risk for developing VTE based on the presence of risk factors including acute heart failure, prior history of VTE, and reduced mobility.³ Since hospitalization may itself increase the risk for VTE, medical patients often receive universal chemoprophylaxis with anticoagulants such as unfractionated heparin (UFH), low-molecular-weight heparin (LMWH), or fondaparinux.³ A meta-analysis of randomized controlled trials (RCTs) published by Wein et al supports the use of VTE chemoprophylaxis in high-risk patients.⁴ It showed statistically significant reductions in rates of PE in high-risk hospitalized medical patients with UFH (risk ratio [RR], 0.64; 95% CI, 0.50-0.82) or LMWH chemoprophylaxis (RR, 0.37; 95% CI, 0.21-0.64), compared with controls.

In recognition of the magnitude of the problem, national organizations have emphasized routine chemoprophylaxis for prevention of in-hospital VTE as a top-priority measure for patient safety.^5,6 The Joint Commission includes chemoprophylaxis as a quality core metric and failure to adhere to such standards compromises hospital accreditation.⁵ Since 2008, the Centers for Medicare & Medicaid Services no longer reimburses hospitals for preventable VTE and requires institutions to document the rationale for omitting chemoprophylaxis if not commenced on hospital admission.⁶

In order to understand why chemoprophylaxis fails to benefit low-risk medical patients, it is necessary to critically examine the benefits identified in trials of high-risk patients. Although RCTs and meta-analysis of chemoprophylaxis have consistently demonstrated a reduction in VTE, prevention of asymptomatic VTE identified on screening with ultrasound or venography accounts for more than 90% of the composite outcome in the three key trials.^7-9 Hospitalists do not routinely screen for asymptomatic VTE, and incorporation of these events into composite VTE outcomes inflates the magnitude of benefit gained by chemoprophylaxis. Importantly, the standard of care does not include screening for asymptomatic DVTs, and studies have estimated that only 10% to 15% of asymptomatic DVTs progress to a symptomatic VTE.¹⁰

A meta-analysis of trials evaluating unselected general medical patients (ie, not those with specific high-risk conditions such as acute myocardial infarction) did not show a reduction in symptomatic VTE with chemoprophylaxis (odds ratio [OR], 0.59; 95% CI, 0.29-1.23).¹¹ In the meta-analysis by Wein et al, which did include patients with specific high-risk conditions, chemoprophylaxis produced a small absolute risk reduction, resulting in a number needed to treat (NNT) of 345 to prevent one PE.⁴ This demonstrates that, even in high-risk patients, the magnitude of benefit is small. Population-level data also question the benefit of chemoprophylaxis. Flanders et al stratified 35 Michigan hospitals into high-, moderate-, and low-performance tertiles, with performance based on the rate of chemoprophylaxis use on admission for general medical patients at high-risk for VTE. The authors found no significant difference in the rate of VTE at 90 days among tertiles.¹² These findings question the usefulness of universal chemoprophylaxis when applied in a real-world setting.

The high rates of VTE in the absence of chemoprophylaxis reported in historic trials may overestimate the contemporary risk. A 2019 multicenter, observational study examined the rate of hospital-acquired DVT for 1,170 low- and high-risk patients with acute medical illness admitted to the internal medicine ward.¹³ Of them, 250 (21%) underwent prophylaxis with parenteral anticoagulants (mean Padua Prediction Score, 4.5). The remaining 920 (79%) were not treated with prophylaxis (mean Padua Prediction Score, 2.5). All patients underwent ultrasound at admission and discharge. The average length of stay was 13 days, and just three patients (0.3%) experienced in-hospital DVT, two of whom were receiving chemoprophylaxis. Only one (0.09%) DVT was symptomatic.

It should be emphasized that any evidence favoring chemoprophylaxis comes from studies of patients at high-risk of VTE. No data show benefit for low-risk patients. Therefore, any risk of chemoprophylaxis likely outweighs the benefits in low-risk patients. Importantly, the risks are underappreciated. A 2014 meta-analysis reported an increased risk of major hemorrhage (OR, 1.81; 95% CI, 1.10-2.98; P = .02) in high-risk medically ill patients on chemoprophylaxis.¹⁴ This results in a number needed to harm for major bleeding of 336, a value similar to the NNT for benefit reported by Wein et al.⁴ Heparin-induced thrombocytopenia, a potentially limb- and life-threatening complication of UFH or LMWH exposure, has an overall incidence of 0.3% to 0.7% in hospitalized patients on chemoprophylaxis.³ Finally, the most commonly used chemoprophylaxis medications are administered subcutaneously, resulting in injection site pain. Unsurprisingly, hospitalized patients refuse chemoprophylaxis more frequently than any other medication.¹⁵

The negative implications of inappropriate chemoprophylaxis extend beyond direct harms to patients. Poor stratification and overuse results in unnecessary healthcare costs. One single-center retrospective review demonstrated that, after integration of chemoprophylaxis into hospital order sets, 76% of patients received unnecessary administration of chemoprophylaxis, resulting in an annualized expenditure of $77,652.¹⁶ This does not take into account costs associated with major bleeds.

Unfortunately, the pendulum has shifted from an era of underprescribing chemoprophylaxis to hospitalized medical patients to one of overprescribing. Data published in 2018 suggest that providers overuse chemoprophylaxis in low-risk medical patients at more than double the rate of underusing it in high-risk patients (57% vs 21%).¹⁷

Several national societies, including the often cited American College of Chest Physicians (ACCP) and American Society of Hematology (ASH), provide guidance on the use of VTE chemoprophylaxis in acutely ill medical inpatients.^3,18 The ASH guidelines conditionally recommend VTE chemoprophylaxis rather than no chemoprophylaxis.¹⁸ However, the guidelines do not provide guidance on a risk-stratified approach and disclose that this recommendation is supported by a low certainty in the evidence of the net health benefit gained.¹⁸ Guidelines from ACCP lean towards individualized care and recommend against the use of VTE chemoprophylaxis for hospitalized acutely ill, low-risk medical patients.³

Clinicians should risk stratify using validated RAMs when making a patient-centered treatment plan on admission. The table outlines the most common RAMs with evidence for use in acute medically ill hospitalized patients. Although RAMs have limitations (eg, lack of prospective validation and complexity), the ACCP guidelines advocate for their use.³

Given that immobility independently increases risk for VTE, early mobilization is a simple and cost-effective way to potentially prevent VTE in low-risk patients. In addition to this potential benefit, early mobilization shortens the length of hospital stay, improves functional status and rates of delirium in hospitalized elderly patients, and hastens postoperative recovery after major surgeries.¹⁹

• Incorporate a patient-centered, risk-stratified approach to identify low-risk patients. This can be done manually or with use of RAMS embedded in the electronic health record.
• Do not prescribe chemoprophylaxis to low-risk hospitalized medical patients.
• Emphasize the importance of early mobilization in hospitalized patients.

In regard to the case, the hospitalist should use a RAM developed for the nonsurgical, non–critically ill patient to determine her need for chemoprophylaxis. Based on the clinical data presented, the three RAMs available would classify the patient as low risk for developing an in-hospital VTE. She should not receive chemoprophylaxis given the lack of data demonstrating benefit in this population. To mitigate the potential risk of bleeding, heparin-induced thrombocytopenia, and painful injections, the hospitalist should discontinue heparin. The hospitalist should advocate for early mobilization and minimize the duration of hospital stay as appropriate.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason^™”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by emailing [email protected].

Inspired by the ABIM Foundation’s Choosing Wisel y ^® campaign, the “Things We Do for No Reason ^™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A hospitalist admits a 68-year-old woman for community-acquired pneumonia with a past medical history of hypertension, gastroesophageal reflux disease, and osteoarthritis. Her hospitalist consults physical therapy to maximize mobility; continues her home medications including pantoprazole, hydrochlorothiazide, and acetaminophen; and initiates antimicrobial therapy with ceftriaxone and azithromycin. The hospital admission order set requires administration of subcutaneous unfractionated heparin for venous thromboembolism chemoprophylaxis.

Venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), ranks among the leading preventable causes of morbidity and mortality in hospitalized patients.¹ DVTs can rapidly progress to a PE, which account for 5% to 10% of in-hospital deaths.¹ The negative sequelae of in-hospital VTE, including prolonged hospital stay, increased healthcare costs, and greater risks associated with pharmacologic treatment, add $9 to $18.2 billion in US healthcare expenditures each year.² Various risk-assessment models (RAMs) identify medical patients at high risk for developing VTE based on the presence of risk factors including acute heart failure, prior history of VTE, and reduced mobility.³ Since hospitalization may itself increase the risk for VTE, medical patients often receive universal chemoprophylaxis with anticoagulants such as unfractionated heparin (UFH), low-molecular-weight heparin (LMWH), or fondaparinux.³ A meta-analysis of randomized controlled trials (RCTs) published by Wein et al supports the use of VTE chemoprophylaxis in high-risk patients.⁴ It showed statistically significant reductions in rates of PE in high-risk hospitalized medical patients with UFH (risk ratio [RR], 0.64; 95% CI, 0.50-0.82) or LMWH chemoprophylaxis (RR, 0.37; 95% CI, 0.21-0.64), compared with controls.

In recognition of the magnitude of the problem, national organizations have emphasized routine chemoprophylaxis for prevention of in-hospital VTE as a top-priority measure for patient safety.^5,6 The Joint Commission includes chemoprophylaxis as a quality core metric and failure to adhere to such standards compromises hospital accreditation.⁵ Since 2008, the Centers for Medicare & Medicaid Services no longer reimburses hospitals for preventable VTE and requires institutions to document the rationale for omitting chemoprophylaxis if not commenced on hospital admission.⁶

In order to understand why chemoprophylaxis fails to benefit low-risk medical patients, it is necessary to critically examine the benefits identified in trials of high-risk patients. Although RCTs and meta-analysis of chemoprophylaxis have consistently demonstrated a reduction in VTE, prevention of asymptomatic VTE identified on screening with ultrasound or venography accounts for more than 90% of the composite outcome in the three key trials.^7-9 Hospitalists do not routinely screen for asymptomatic VTE, and incorporation of these events into composite VTE outcomes inflates the magnitude of benefit gained by chemoprophylaxis. Importantly, the standard of care does not include screening for asymptomatic DVTs, and studies have estimated that only 10% to 15% of asymptomatic DVTs progress to a symptomatic VTE.¹⁰

A meta-analysis of trials evaluating unselected general medical patients (ie, not those with specific high-risk conditions such as acute myocardial infarction) did not show a reduction in symptomatic VTE with chemoprophylaxis (odds ratio [OR], 0.59; 95% CI, 0.29-1.23).¹¹ In the meta-analysis by Wein et al, which did include patients with specific high-risk conditions, chemoprophylaxis produced a small absolute risk reduction, resulting in a number needed to treat (NNT) of 345 to prevent one PE.⁴ This demonstrates that, even in high-risk patients, the magnitude of benefit is small. Population-level data also question the benefit of chemoprophylaxis. Flanders et al stratified 35 Michigan hospitals into high-, moderate-, and low-performance tertiles, with performance based on the rate of chemoprophylaxis use on admission for general medical patients at high-risk for VTE. The authors found no significant difference in the rate of VTE at 90 days among tertiles.¹² These findings question the usefulness of universal chemoprophylaxis when applied in a real-world setting.

The high rates of VTE in the absence of chemoprophylaxis reported in historic trials may overestimate the contemporary risk. A 2019 multicenter, observational study examined the rate of hospital-acquired DVT for 1,170 low- and high-risk patients with acute medical illness admitted to the internal medicine ward.¹³ Of them, 250 (21%) underwent prophylaxis with parenteral anticoagulants (mean Padua Prediction Score, 4.5). The remaining 920 (79%) were not treated with prophylaxis (mean Padua Prediction Score, 2.5). All patients underwent ultrasound at admission and discharge. The average length of stay was 13 days, and just three patients (0.3%) experienced in-hospital DVT, two of whom were receiving chemoprophylaxis. Only one (0.09%) DVT was symptomatic.

It should be emphasized that any evidence favoring chemoprophylaxis comes from studies of patients at high-risk of VTE. No data show benefit for low-risk patients. Therefore, any risk of chemoprophylaxis likely outweighs the benefits in low-risk patients. Importantly, the risks are underappreciated. A 2014 meta-analysis reported an increased risk of major hemorrhage (OR, 1.81; 95% CI, 1.10-2.98; P = .02) in high-risk medically ill patients on chemoprophylaxis.¹⁴ This results in a number needed to harm for major bleeding of 336, a value similar to the NNT for benefit reported by Wein et al.⁴ Heparin-induced thrombocytopenia, a potentially limb- and life-threatening complication of UFH or LMWH exposure, has an overall incidence of 0.3% to 0.7% in hospitalized patients on chemoprophylaxis.³ Finally, the most commonly used chemoprophylaxis medications are administered subcutaneously, resulting in injection site pain. Unsurprisingly, hospitalized patients refuse chemoprophylaxis more frequently than any other medication.¹⁵

The negative implications of inappropriate chemoprophylaxis extend beyond direct harms to patients. Poor stratification and overuse results in unnecessary healthcare costs. One single-center retrospective review demonstrated that, after integration of chemoprophylaxis into hospital order sets, 76% of patients received unnecessary administration of chemoprophylaxis, resulting in an annualized expenditure of $77,652.¹⁶ This does not take into account costs associated with major bleeds.

Unfortunately, the pendulum has shifted from an era of underprescribing chemoprophylaxis to hospitalized medical patients to one of overprescribing. Data published in 2018 suggest that providers overuse chemoprophylaxis in low-risk medical patients at more than double the rate of underusing it in high-risk patients (57% vs 21%).¹⁷

Several national societies, including the often cited American College of Chest Physicians (ACCP) and American Society of Hematology (ASH), provide guidance on the use of VTE chemoprophylaxis in acutely ill medical inpatients.^3,18 The ASH guidelines conditionally recommend VTE chemoprophylaxis rather than no chemoprophylaxis.¹⁸ However, the guidelines do not provide guidance on a risk-stratified approach and disclose that this recommendation is supported by a low certainty in the evidence of the net health benefit gained.¹⁸ Guidelines from ACCP lean towards individualized care and recommend against the use of VTE chemoprophylaxis for hospitalized acutely ill, low-risk medical patients.³

Clinicians should risk stratify using validated RAMs when making a patient-centered treatment plan on admission. The table outlines the most common RAMs with evidence for use in acute medically ill hospitalized patients. Although RAMs have limitations (eg, lack of prospective validation and complexity), the ACCP guidelines advocate for their use.³

Given that immobility independently increases risk for VTE, early mobilization is a simple and cost-effective way to potentially prevent VTE in low-risk patients. In addition to this potential benefit, early mobilization shortens the length of hospital stay, improves functional status and rates of delirium in hospitalized elderly patients, and hastens postoperative recovery after major surgeries.¹⁹

• Incorporate a patient-centered, risk-stratified approach to identify low-risk patients. This can be done manually or with use of RAMS embedded in the electronic health record.
• Do not prescribe chemoprophylaxis to low-risk hospitalized medical patients.
• Emphasize the importance of early mobilization in hospitalized patients.

In regard to the case, the hospitalist should use a RAM developed for the nonsurgical, non–critically ill patient to determine her need for chemoprophylaxis. Based on the clinical data presented, the three RAMs available would classify the patient as low risk for developing an in-hospital VTE. She should not receive chemoprophylaxis given the lack of data demonstrating benefit in this population. To mitigate the potential risk of bleeding, heparin-induced thrombocytopenia, and painful injections, the hospitalist should discontinue heparin. The hospitalist should advocate for early mobilization and minimize the duration of hospital stay as appropriate.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason^™”? Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by emailing [email protected].

There is a growing appreciation of gender disparities in career advancement in medicine. By 2004, approximately 50% of medical school graduates were women, yet considerable differences persist between genders in compensation, faculty rank, and leadership positions.^1-3 According to the Association of American Medical Colleges (AAMC), women account for only 25% of full professors, 18% of department chairs, and 18% of medical school deans.¹ Women are also underrepresented in other areas of leadership such as division directors, professional society leadership, and hospital executives.^4-6

Specialties that are predominantly women, including pediatrics, are not immune to gender disparities. Women represent 71% of pediatric residents¹ and currently constitute two-thirds of active pediatricians in the United States.⁷ However, there is a disproportionately low number of women ascending the pediatric academic ladder, with only 35% of full professors² and 28% of department chairs being women.¹ Pediatrics also was noted to have the fifth-largest gender pay gap across 40 specialties.³ These disparities can contribute to burnout, poorer patient outcomes, and decreased advancement of women known as the “leaky pipeline.”^1,8,9

There is some evidence that gender disparities may be improving among younger professionals with increasing percentages of women as leaders and decreasing pay gaps.^10,11 These potential positive trends provide hope that fields in medicine early in their development may demonstrate fewer gender disparities. One of the youngest fields of medicine is pediatric hospital medicine (PHM), which officially became a recognized pediatric subspecialty in 2017.¹² There is no literature to date describing gender disparities in PHM. We aimed to explore the gender distribution of university-based PHM program leadership and to compare this gender distribution with that seen in the broader field of PHM.

This study was Institutional Review Board–approved as non–human subjects research through University of Chicago, Chicago, Illinois. From January to March 2020, the authors performed web-based searches for PHM division directors or program leaders in the United States. Because there is no single database of PHM programs in the United States, we used the AAMC list of Liaison Committee on Medical Education (LCME)–accredited US medical schools; medical schools in Puerto Rico were not included, nor were pending and provisional institutions. If an institution had multiple practice sites for its students, the primary site for third-year medical student clerkship rotations was included. If a medical school had multiple branches, each with its own primary inpatient pediatrics site, these sites were included. If there was no PHM division director, a program leader (lead hospitalist) was substituted and counted as long as the role was formally designated. This leadership role is herein referred to under the umbrella term of “division director.”

We searched medical school web pages, affiliated hospital web pages, and Google. All program leadership information (divisional and fellowship, if present) was confirmed through direct communication with the program, most commonly with division directors, and included name, gender, title, and presence of associate/assistant leader, gender, and title. Associate division directors were only included if it was a formal leadership position. Associate directors of research, quality, etc, were not included due to the limited number of formal positions noted on further review. Of note, the terms “associate” and “assistant” are referring to leadership positions and not academic ranks.

Fellowship leadership was included if affiliated with a US medical school in the primary list. Medical schools with multiple PHM fellowships were included as separate observations. The leadership was confirmed using the methods described above and cross-referenced through the PHM Fellowship Program website. PHM fellowship programs starting in 2020 were included if leadership was determined.

All leadership positions were verified by two authors, and all authors reviewed the master list to identify errors.

To determine the overall gender breakdown in the specialty, we used three estimates: 2019 American Board of Pediatrics (ABP) PHM Board Certification Exam applicants, the 2019 American Academy of Pediatrics Section on Hospital Medicine membership, and a random sample of all PHM faculty in 25% of the programs included in this study.⁴

Descriptive statistics using 95% confidence intervals for proportions were used. Differences between proportions were evaluated using a two-proportion z test with the null hypothesis that the two proportions are the same and significance set at P < .05. 

Of the 150 AAMC LCME–accredited medical school departments of pediatrics evaluated, a total of 142 programs were included; eight programs were excluded due to not providing inpatient pediatric services.

Division Leadership

The proportion of women PHM division directors was 55% (95% CI, 47%-63%) in this sample of 146 leaders from 142 programs (4 programs had coleaders). In the 113 programs with standalone PHM divisions or sections, the proportion of women division directors was 56% (95% CI, 47%-64%). In the 29 hospitalist groups that were not standalone (ie, embedded in another division), the proportion of women leaders was similar at 52% (95% CI, 34%-69%). In 24 programs with 27 formally designated associate directors (1 program had 3 associate directors and 1 program had 2), 81% of associate directors were women (95% CI, 63%-92%).

Fellowship Leadership

A total of 51 PHM fellowship programs had 53 directors (2 had codirectors), and 66% of the fellowship directors were women (95% CI, 53%-77%). A total of 31 programs had 34 assistant directors (3 programs had 2 assistants), and 82% of the assistant fellowship directors were women (95% CI, 66%-92%).

Comparison With the Field at Large

The inaugural ABP PHM board certification exam in 2019 had 1,627 applicants with 70% women (95% CI, 68%-73%) (Suzanne Woods, MD, email communication, December 4, 2019). The American Academy of Pediatrics Section on Hospital Medicine, the largest PHM-specific organization, has 2,299 practicing physician members with 71% women (95% CI, 69%-73%) (Niccole Alexander, email communication, November 25, 2019). Our random sample of 25% of university-based PHM programs contained 1,063 faculty members with 72% women (95% CI, 69%-75%).

The Table provides P values for comparisons of the proportion of women in each of the above-described leadership roles compared to the most conservative estimate of women in the field from the estimates given above (ie, 70%). Compared with the field at large, women appear to be underrepresented as division directors (70% vs 55%; P < .001) but not as fellowship directors (70% vs 66%; P = .5). There is a higher proportion of women in all associate/assistant director roles, compared with the population (82% vs 70%; P = .04).

We found a significant difference between the proportion of women as PHM division directors (55%) when compared with the proportion of women physicians in PHM (70%), which suggests that women are underrepresented in clinical leadership at university-based pediatric hospitalist programs. Similar findings are described in other specialties, including notably adult hospital medicine.⁴ Burden et al found that only 16% of hospital medicine program leaders were women despite an equal number of women and men in the field. PHM has a much larger proportion of women, compared with that of hospital medicine, and yet women are still underrepresented as program leaders.

We found no disparities between the proportion of women as PHM fellowship directors and the field at large. These results are similar to those of other studies, which showed a higher number of women in educational leadership roles and lower representation in roles with influence over policy and allocation of resources.^13,14 Although the proportion of women in educational roles itself is not a concern, there is evidence that these positions may be undervalued by some institutions, which provide these positions with lower salaries and fewer opportunities for career advancement.^13,14

Interestingly, women are well-represented in associate/assistant director roles at both the division and fellowship leader level when comparing the distribution in those roles with that of the PHM field at large. This finding suggests that the pipeline of women is robust and potentially may indicate positive change. Alternatively, this finding may reflect a previously described phenomenon of the “sticky floor” in which women are “stuck” in these supportive roles and do not necessarily advance to higher-impact positions.¹⁵ We found a statistically significant higher proportion of women in the combined group of all associate/assistant directors compared with the overall population, which raises the concern that supportive leadership roles may represent “women’s work.”¹⁶ Future studies are needed to track whether these women truly advance or whether women are overrepresented in supportive leadership positions at the expense of primary leadership positions.

Adequate representation of women alone is not sufficient to achieve gender equity in medicine. We need to understand why there is a lower representation of women in leadership positions. Some barriers have already been described, including gender bias in promotions,¹⁷ higher demands outside of work,¹⁸ and lower pay,³ though none are specific to PHM. A further qualitative exploration of PHM leadership would help describe any barriers women in PHM specifically may be facing in their career trajectory. In addition, more information is needed to explore the experience of women with intersectional identities in PHM, especially since they may experience increased bias and discrimination.¹⁹

Limitations of this study include the lack of a centralized list of PHM programs and data on PHM workforce. Our three estimates for the proportion of women in PHM were similar at 70%-71%; however, these are only proxies for the true gender distribution of PHM physicians, which is unknown. PHM leadership targets of close to 70% women would be reflective of the field at large; however, institutional variation may exist, and ideally leadership should be diverse and reflective of its faculty members. Our study only describes university-based PHM programs and, therefore, is not necessarily generalizable to nonuniversity programs. Further studies are needed to evaluate any potential differences based on program type. In our study, gender was used in binary terms; however, we acknowledge that gender exists on a spectrum.

As a specialty early in development with a robust pipeline of women, PHM is in a unique position to lead the way in gender equity. However, women appear to be underrepresented as division directors at university-based PHM programs. Achieving proportional representation of women leaders is imperative for tapping into the full potential of the community and ensuring that the goals of the field are representative of the population.

Special thanks to Lucille Lester, MD, who asked the question that started this road to discovery.

There is a growing appreciation of gender disparities in career advancement in medicine. By 2004, approximately 50% of medical school graduates were women, yet considerable differences persist between genders in compensation, faculty rank, and leadership positions.^1-3 According to the Association of American Medical Colleges (AAMC), women account for only 25% of full professors, 18% of department chairs, and 18% of medical school deans.¹ Women are also underrepresented in other areas of leadership such as division directors, professional society leadership, and hospital executives.^4-6

Specialties that are predominantly women, including pediatrics, are not immune to gender disparities. Women represent 71% of pediatric residents¹ and currently constitute two-thirds of active pediatricians in the United States.⁷ However, there is a disproportionately low number of women ascending the pediatric academic ladder, with only 35% of full professors² and 28% of department chairs being women.¹ Pediatrics also was noted to have the fifth-largest gender pay gap across 40 specialties.³ These disparities can contribute to burnout, poorer patient outcomes, and decreased advancement of women known as the “leaky pipeline.”^1,8,9

There is some evidence that gender disparities may be improving among younger professionals with increasing percentages of women as leaders and decreasing pay gaps.^10,11 These potential positive trends provide hope that fields in medicine early in their development may demonstrate fewer gender disparities. One of the youngest fields of medicine is pediatric hospital medicine (PHM), which officially became a recognized pediatric subspecialty in 2017.¹² There is no literature to date describing gender disparities in PHM. We aimed to explore the gender distribution of university-based PHM program leadership and to compare this gender distribution with that seen in the broader field of PHM.

This study was Institutional Review Board–approved as non–human subjects research through University of Chicago, Chicago, Illinois. From January to March 2020, the authors performed web-based searches for PHM division directors or program leaders in the United States. Because there is no single database of PHM programs in the United States, we used the AAMC list of Liaison Committee on Medical Education (LCME)–accredited US medical schools; medical schools in Puerto Rico were not included, nor were pending and provisional institutions. If an institution had multiple practice sites for its students, the primary site for third-year medical student clerkship rotations was included. If a medical school had multiple branches, each with its own primary inpatient pediatrics site, these sites were included. If there was no PHM division director, a program leader (lead hospitalist) was substituted and counted as long as the role was formally designated. This leadership role is herein referred to under the umbrella term of “division director.”

We searched medical school web pages, affiliated hospital web pages, and Google. All program leadership information (divisional and fellowship, if present) was confirmed through direct communication with the program, most commonly with division directors, and included name, gender, title, and presence of associate/assistant leader, gender, and title. Associate division directors were only included if it was a formal leadership position. Associate directors of research, quality, etc, were not included due to the limited number of formal positions noted on further review. Of note, the terms “associate” and “assistant” are referring to leadership positions and not academic ranks.

Fellowship leadership was included if affiliated with a US medical school in the primary list. Medical schools with multiple PHM fellowships were included as separate observations. The leadership was confirmed using the methods described above and cross-referenced through the PHM Fellowship Program website. PHM fellowship programs starting in 2020 were included if leadership was determined.

All leadership positions were verified by two authors, and all authors reviewed the master list to identify errors.

To determine the overall gender breakdown in the specialty, we used three estimates: 2019 American Board of Pediatrics (ABP) PHM Board Certification Exam applicants, the 2019 American Academy of Pediatrics Section on Hospital Medicine membership, and a random sample of all PHM faculty in 25% of the programs included in this study.⁴

Descriptive statistics using 95% confidence intervals for proportions were used. Differences between proportions were evaluated using a two-proportion z test with the null hypothesis that the two proportions are the same and significance set at P < .05. 

Of the 150 AAMC LCME–accredited medical school departments of pediatrics evaluated, a total of 142 programs were included; eight programs were excluded due to not providing inpatient pediatric services.

Division Leadership

The proportion of women PHM division directors was 55% (95% CI, 47%-63%) in this sample of 146 leaders from 142 programs (4 programs had coleaders). In the 113 programs with standalone PHM divisions or sections, the proportion of women division directors was 56% (95% CI, 47%-64%). In the 29 hospitalist groups that were not standalone (ie, embedded in another division), the proportion of women leaders was similar at 52% (95% CI, 34%-69%). In 24 programs with 27 formally designated associate directors (1 program had 3 associate directors and 1 program had 2), 81% of associate directors were women (95% CI, 63%-92%).

Fellowship Leadership

A total of 51 PHM fellowship programs had 53 directors (2 had codirectors), and 66% of the fellowship directors were women (95% CI, 53%-77%). A total of 31 programs had 34 assistant directors (3 programs had 2 assistants), and 82% of the assistant fellowship directors were women (95% CI, 66%-92%).

Comparison With the Field at Large

The inaugural ABP PHM board certification exam in 2019 had 1,627 applicants with 70% women (95% CI, 68%-73%) (Suzanne Woods, MD, email communication, December 4, 2019). The American Academy of Pediatrics Section on Hospital Medicine, the largest PHM-specific organization, has 2,299 practicing physician members with 71% women (95% CI, 69%-73%) (Niccole Alexander, email communication, November 25, 2019). Our random sample of 25% of university-based PHM programs contained 1,063 faculty members with 72% women (95% CI, 69%-75%).

The Table provides P values for comparisons of the proportion of women in each of the above-described leadership roles compared to the most conservative estimate of women in the field from the estimates given above (ie, 70%). Compared with the field at large, women appear to be underrepresented as division directors (70% vs 55%; P < .001) but not as fellowship directors (70% vs 66%; P = .5). There is a higher proportion of women in all associate/assistant director roles, compared with the population (82% vs 70%; P = .04).

We found a significant difference between the proportion of women as PHM division directors (55%) when compared with the proportion of women physicians in PHM (70%), which suggests that women are underrepresented in clinical leadership at university-based pediatric hospitalist programs. Similar findings are described in other specialties, including notably adult hospital medicine.⁴ Burden et al found that only 16% of hospital medicine program leaders were women despite an equal number of women and men in the field. PHM has a much larger proportion of women, compared with that of hospital medicine, and yet women are still underrepresented as program leaders.

We found no disparities between the proportion of women as PHM fellowship directors and the field at large. These results are similar to those of other studies, which showed a higher number of women in educational leadership roles and lower representation in roles with influence over policy and allocation of resources.^13,14 Although the proportion of women in educational roles itself is not a concern, there is evidence that these positions may be undervalued by some institutions, which provide these positions with lower salaries and fewer opportunities for career advancement.^13,14

Interestingly, women are well-represented in associate/assistant director roles at both the division and fellowship leader level when comparing the distribution in those roles with that of the PHM field at large. This finding suggests that the pipeline of women is robust and potentially may indicate positive change. Alternatively, this finding may reflect a previously described phenomenon of the “sticky floor” in which women are “stuck” in these supportive roles and do not necessarily advance to higher-impact positions.¹⁵ We found a statistically significant higher proportion of women in the combined group of all associate/assistant directors compared with the overall population, which raises the concern that supportive leadership roles may represent “women’s work.”¹⁶ Future studies are needed to track whether these women truly advance or whether women are overrepresented in supportive leadership positions at the expense of primary leadership positions.

Adequate representation of women alone is not sufficient to achieve gender equity in medicine. We need to understand why there is a lower representation of women in leadership positions. Some barriers have already been described, including gender bias in promotions,¹⁷ higher demands outside of work,¹⁸ and lower pay,³ though none are specific to PHM. A further qualitative exploration of PHM leadership would help describe any barriers women in PHM specifically may be facing in their career trajectory. In addition, more information is needed to explore the experience of women with intersectional identities in PHM, especially since they may experience increased bias and discrimination.¹⁹

Limitations of this study include the lack of a centralized list of PHM programs and data on PHM workforce. Our three estimates for the proportion of women in PHM were similar at 70%-71%; however, these are only proxies for the true gender distribution of PHM physicians, which is unknown. PHM leadership targets of close to 70% women would be reflective of the field at large; however, institutional variation may exist, and ideally leadership should be diverse and reflective of its faculty members. Our study only describes university-based PHM programs and, therefore, is not necessarily generalizable to nonuniversity programs. Further studies are needed to evaluate any potential differences based on program type. In our study, gender was used in binary terms; however, we acknowledge that gender exists on a spectrum.

As a specialty early in development with a robust pipeline of women, PHM is in a unique position to lead the way in gender equity. However, women appear to be underrepresented as division directors at university-based PHM programs. Achieving proportional representation of women leaders is imperative for tapping into the full potential of the community and ensuring that the goals of the field are representative of the population.

Special thanks to Lucille Lester, MD, who asked the question that started this road to discovery.

There is a growing appreciation of gender disparities in career advancement in medicine. By 2004, approximately 50% of medical school graduates were women, yet considerable differences persist between genders in compensation, faculty rank, and leadership positions.^1-3 According to the Association of American Medical Colleges (AAMC), women account for only 25% of full professors, 18% of department chairs, and 18% of medical school deans.¹ Women are also underrepresented in other areas of leadership such as division directors, professional society leadership, and hospital executives.^4-6

Specialties that are predominantly women, including pediatrics, are not immune to gender disparities. Women represent 71% of pediatric residents¹ and currently constitute two-thirds of active pediatricians in the United States.⁷ However, there is a disproportionately low number of women ascending the pediatric academic ladder, with only 35% of full professors² and 28% of department chairs being women.¹ Pediatrics also was noted to have the fifth-largest gender pay gap across 40 specialties.³ These disparities can contribute to burnout, poorer patient outcomes, and decreased advancement of women known as the “leaky pipeline.”^1,8,9

There is some evidence that gender disparities may be improving among younger professionals with increasing percentages of women as leaders and decreasing pay gaps.^10,11 These potential positive trends provide hope that fields in medicine early in their development may demonstrate fewer gender disparities. One of the youngest fields of medicine is pediatric hospital medicine (PHM), which officially became a recognized pediatric subspecialty in 2017.¹² There is no literature to date describing gender disparities in PHM. We aimed to explore the gender distribution of university-based PHM program leadership and to compare this gender distribution with that seen in the broader field of PHM.

This study was Institutional Review Board–approved as non–human subjects research through University of Chicago, Chicago, Illinois. From January to March 2020, the authors performed web-based searches for PHM division directors or program leaders in the United States. Because there is no single database of PHM programs in the United States, we used the AAMC list of Liaison Committee on Medical Education (LCME)–accredited US medical schools; medical schools in Puerto Rico were not included, nor were pending and provisional institutions. If an institution had multiple practice sites for its students, the primary site for third-year medical student clerkship rotations was included. If a medical school had multiple branches, each with its own primary inpatient pediatrics site, these sites were included. If there was no PHM division director, a program leader (lead hospitalist) was substituted and counted as long as the role was formally designated. This leadership role is herein referred to under the umbrella term of “division director.”

We searched medical school web pages, affiliated hospital web pages, and Google. All program leadership information (divisional and fellowship, if present) was confirmed through direct communication with the program, most commonly with division directors, and included name, gender, title, and presence of associate/assistant leader, gender, and title. Associate division directors were only included if it was a formal leadership position. Associate directors of research, quality, etc, were not included due to the limited number of formal positions noted on further review. Of note, the terms “associate” and “assistant” are referring to leadership positions and not academic ranks.

Fellowship leadership was included if affiliated with a US medical school in the primary list. Medical schools with multiple PHM fellowships were included as separate observations. The leadership was confirmed using the methods described above and cross-referenced through the PHM Fellowship Program website. PHM fellowship programs starting in 2020 were included if leadership was determined.

All leadership positions were verified by two authors, and all authors reviewed the master list to identify errors.

To determine the overall gender breakdown in the specialty, we used three estimates: 2019 American Board of Pediatrics (ABP) PHM Board Certification Exam applicants, the 2019 American Academy of Pediatrics Section on Hospital Medicine membership, and a random sample of all PHM faculty in 25% of the programs included in this study.⁴

Descriptive statistics using 95% confidence intervals for proportions were used. Differences between proportions were evaluated using a two-proportion z test with the null hypothesis that the two proportions are the same and significance set at P < .05. 

Of the 150 AAMC LCME–accredited medical school departments of pediatrics evaluated, a total of 142 programs were included; eight programs were excluded due to not providing inpatient pediatric services.

Division Leadership

The proportion of women PHM division directors was 55% (95% CI, 47%-63%) in this sample of 146 leaders from 142 programs (4 programs had coleaders). In the 113 programs with standalone PHM divisions or sections, the proportion of women division directors was 56% (95% CI, 47%-64%). In the 29 hospitalist groups that were not standalone (ie, embedded in another division), the proportion of women leaders was similar at 52% (95% CI, 34%-69%). In 24 programs with 27 formally designated associate directors (1 program had 3 associate directors and 1 program had 2), 81% of associate directors were women (95% CI, 63%-92%).

Fellowship Leadership

A total of 51 PHM fellowship programs had 53 directors (2 had codirectors), and 66% of the fellowship directors were women (95% CI, 53%-77%). A total of 31 programs had 34 assistant directors (3 programs had 2 assistants), and 82% of the assistant fellowship directors were women (95% CI, 66%-92%).

Comparison With the Field at Large

The inaugural ABP PHM board certification exam in 2019 had 1,627 applicants with 70% women (95% CI, 68%-73%) (Suzanne Woods, MD, email communication, December 4, 2019). The American Academy of Pediatrics Section on Hospital Medicine, the largest PHM-specific organization, has 2,299 practicing physician members with 71% women (95% CI, 69%-73%) (Niccole Alexander, email communication, November 25, 2019). Our random sample of 25% of university-based PHM programs contained 1,063 faculty members with 72% women (95% CI, 69%-75%).

The Table provides P values for comparisons of the proportion of women in each of the above-described leadership roles compared to the most conservative estimate of women in the field from the estimates given above (ie, 70%). Compared with the field at large, women appear to be underrepresented as division directors (70% vs 55%; P < .001) but not as fellowship directors (70% vs 66%; P = .5). There is a higher proportion of women in all associate/assistant director roles, compared with the population (82% vs 70%; P = .04).

We found a significant difference between the proportion of women as PHM division directors (55%) when compared with the proportion of women physicians in PHM (70%), which suggests that women are underrepresented in clinical leadership at university-based pediatric hospitalist programs. Similar findings are described in other specialties, including notably adult hospital medicine.⁴ Burden et al found that only 16% of hospital medicine program leaders were women despite an equal number of women and men in the field. PHM has a much larger proportion of women, compared with that of hospital medicine, and yet women are still underrepresented as program leaders.

We found no disparities between the proportion of women as PHM fellowship directors and the field at large. These results are similar to those of other studies, which showed a higher number of women in educational leadership roles and lower representation in roles with influence over policy and allocation of resources.^13,14 Although the proportion of women in educational roles itself is not a concern, there is evidence that these positions may be undervalued by some institutions, which provide these positions with lower salaries and fewer opportunities for career advancement.^13,14

Interestingly, women are well-represented in associate/assistant director roles at both the division and fellowship leader level when comparing the distribution in those roles with that of the PHM field at large. This finding suggests that the pipeline of women is robust and potentially may indicate positive change. Alternatively, this finding may reflect a previously described phenomenon of the “sticky floor” in which women are “stuck” in these supportive roles and do not necessarily advance to higher-impact positions.¹⁵ We found a statistically significant higher proportion of women in the combined group of all associate/assistant directors compared with the overall population, which raises the concern that supportive leadership roles may represent “women’s work.”¹⁶ Future studies are needed to track whether these women truly advance or whether women are overrepresented in supportive leadership positions at the expense of primary leadership positions.

Adequate representation of women alone is not sufficient to achieve gender equity in medicine. We need to understand why there is a lower representation of women in leadership positions. Some barriers have already been described, including gender bias in promotions,¹⁷ higher demands outside of work,¹⁸ and lower pay,³ though none are specific to PHM. A further qualitative exploration of PHM leadership would help describe any barriers women in PHM specifically may be facing in their career trajectory. In addition, more information is needed to explore the experience of women with intersectional identities in PHM, especially since they may experience increased bias and discrimination.¹⁹

Limitations of this study include the lack of a centralized list of PHM programs and data on PHM workforce. Our three estimates for the proportion of women in PHM were similar at 70%-71%; however, these are only proxies for the true gender distribution of PHM physicians, which is unknown. PHM leadership targets of close to 70% women would be reflective of the field at large; however, institutional variation may exist, and ideally leadership should be diverse and reflective of its faculty members. Our study only describes university-based PHM programs and, therefore, is not necessarily generalizable to nonuniversity programs. Further studies are needed to evaluate any potential differences based on program type. In our study, gender was used in binary terms; however, we acknowledge that gender exists on a spectrum.

As a specialty early in development with a robust pipeline of women, PHM is in a unique position to lead the way in gender equity. However, women appear to be underrepresented as division directors at university-based PHM programs. Achieving proportional representation of women leaders is imperative for tapping into the full potential of the community and ensuring that the goals of the field are representative of the population.

Special thanks to Lucille Lester, MD, who asked the question that started this road to discovery.

Preparing patients for hospital discharge requires multiple tasks that cross professional boundaries. Clinician’s roles may be ambiguous, and responsibility for a safe high-quality discharge is often diffused among the team rather than being defined as the core responsibility of a single member.^1-8 Without a shared understanding of patient resources and tasks involved in anticipatory planning, lapses in discharge preparation can occur, which places patients at increased risk for harm after hospitalization.^3-7 As a result, organizations like the Centers for Medicare & Medicaid Services (CMS) have called for team-based patient-centered discharge planning.⁸ Yet to develop more effective team-based discharge planning interventions, a more nuanced understanding of how healthcare teams work together is needed.^2,3,9

Shared mental models (SMMs) provide a useful theoretical framework and measurement approach for examining how interprofessional teams coordinate complex tasks like hospital discharge.^10-13 SMMs represent the team members’ collective understanding and organized knowledge of key elements needed for teams to perform effectively.^9-11 Validated questionnaires can be used to measure two key properties of SMMs: the degree to which team members have a similar understanding of the situation at hand (team SMM convergence) and to what extent this understanding is aligned with the patient (team-­patient SMM convergence).^10,11 Researchers have found that teams with higher-quality SMMs have a better understanding of what is happening and why, have clearer expectations of their roles and tasks, and can better predict what might happen next.^10,12 As a result, these teams more effectively coordinate actions and adapt to task demands even in cases of high complexity, uncertainty, and stress.^10-13 Prior studies examining healthcare teams in emergency departments,^14-16 critical care units,^17,18 and operating rooms¹⁹ suggest high-quality SMMs are needed to safely care for patients.¹³ Yet there has been limited evaluation of SMMs in general internal medicine, much less during hospital discharge.^9,13 The purpose of this study was to examine SMMs for a critical task of the inpatient team: developing a shared understanding of the patient’s readiness for hospital discharge.^20,21

Design

We used a cross-sectional survey design at a single Midwestern community hospital to determine inpatient care teams’ SMMs of patient hospital discharge readiness. This study is part of a larger mixed-methods evaluation of interprofessional hospital discharge teamwork in older adult patients at risk for a poor transition to home.⁹ Data were collected using questionnaires from patients and their team (nurse, coordinator, and physician) within 4 hours of the patient leaving the hospital. First, we measured the teams’ assessment, team convergence, and team-­patient convergence on patient readiness for discharge from the hospital. Then, after identifying relevant potential predictors from the literature, we developed regression models to predict the teams’ assessment, team convergence, and team-patient convergence of discharge readiness based on these variables. Our local institutional review board approved this study.

Sample and Participants

We used a convenience sampling approach to identify eligible discharge events consisting of the patient and care team.⁹ We focused on patients at high-risk for poor hospital-to-home transitions.^3,22 Eligible events included older patients (≥65 years) who were discharged home without home health or hospice services and admitted with a primary diagnosis of heart failure, acute myocardial infarction, hip replacement, knee replacement, pneumonia, or chronic obstructive pulmonary disease. Patient exclusion criteria included inability to complete study forms because of mental incapacity or a language barrier. Discharge team member inclusion criteria included the bedside nurse, attending physician, and coordinator (a unit-dedicated discharge nurse or social worker) caring for the patient participant at the time of hospital discharge. Each discharge team was unique: The same three individuals could not be included as a “team” for more than one discharge event, although individual members could be included as a part of other teams with a different set of individuals. Appendix A provides an enrollment flowchart.

Conceptual Framework

We applied the SMM conceptual framework to the context of hospital discharge. As shown in the Figure, SMMs are examined at the team level and contain the critical knowledge held by the team to be effective.^15,16 From a patient-centered perspective, patients are considered the expert on how ready they feel to be discharged home.^20,23,24 In this case, the SMM content is the discharge team members’ shared assessment of how ready the patient is for hospital discharge (Figure, B).¹⁰ Convergence is the degree of agreement among individual mental models.^10-13 In this study we examined two types of convergence: (1) team convergence, or the team members degree of agreement on the patient’s readiness for discharge (Figure, C), and (2) team-patient convergence, or the degree to which the team’s SMM aligns with the patient’s mental model (Figure, D).^10-13

Measures and Variables

Readiness for Hospital Discharge Scales/Short Form

We used parallel clinician and patient versions of the Readiness for Hospital Discharge Scale/Short Form (RHDS/SF)^25-28 to determine the teams’ assessment of discharge readiness, team SMM convergence, and team-patient SMM convergence.

The RHDS/SF scales are 8-item validated instruments that use a Likert scale (0 for not ready to 10 for totally ready) to assess the individual clinician’s or patient’s perceptions of how ready the patient is to be discharged.^20,25,27 The RHDS/SF instruments include four dimensions conceptualized as crucial to patient readiness for discharge and important to anticipatory planning: (1) Personal Status, physical-emotional state of the patient before discharge; (2) Knowledge, perceived adequacy of information needed to respond to common posthospitalization concerns/problems; (3) Coping Ability, perceived ability to self-manage health care needs; and (4) Expected Support, emotional-physical assistance available (Appendix B).^20,25,27 The RHDS/SF instruments’ results are calculated as a mean of item scores, with higher individual scores indicating the rater assessed the patient as being more ready for hospital discharge.²⁰ The RHDS/SF scales have undergone rigorous psychometric testing and are linked to patient outcomes (eg, readmissions, emergency room visits, patient coping difficulties after discharge, and patient-rated quality of preparation for posthospital care).^20,25-28 For example, predictive validity assessments for adult medical-surgical patients found lower Nurse-RHDS/SF scores are associated with a six- to ninefold increase in 30-day readmission risk.²⁰

Contextual Variables

We reviewed the literature to identify potential patient and system factors associated with adverse transitional care outcomes^1-8 and/or higher quality SMMs in other settings.^10-19 For example, patient characteristics included age, principal diagnosis, length of stay, number of comorbidities, and cognition impairment (using the Short Portable Mini Mental Status Questionnaire²⁹).^2,22,30 Examples of system factor include teamwork and communication quality^1-6 on day of discharge, as well as educational background and experience of clinicians on the team.^31-33 We adapted a validated survey using 7-point Likert scale questions to determine teamwork quality and communication quality during individual patent discharges.³³Appendix C provides descriptions of all variables.

Data Collection

Patient recruitment occurred from February to October 2017 in a single community hospital in Iowa.⁹ We identified potentially eligible events in collaboration with the unit charge nurses. Patients were screened 24 to 48 hours prior to anticipated day of discharge; those interested/eligible underwent informed consent procedures.⁹ We collected data from the patient and their corresponding bedside nurse, coordinator, and attending physician on the day of discharge. After the discharge order was placed and care instructions were provided, the patient completed a demographic survey, Short Portable Mini Mental Status Questionnaire, and the Patient-RHDS/SF. Individual team members completed a survey with the demographic information, their respective versions of RHDS/SF, and day-of-discharge teamwork-related questions. On average, the survey took clinicians less than 5 minutes to complete. We performed a chart review to determine additional patient characteristics such as principal diagnosis, length of stay, and number of comorbidities.

Data Analysis

Team Assessment of Patient Discharge Readiness

The teams’ shared assessment (SMM content) was determined by averaging the members’ individual scores on the Clinician-RHDS/SF.³⁴ Discharge events with higher team assessments indicated the team perceived the patient as being readier for hospital discharge. Guided by prior research, we examined the RHDS/SF scores as a continuous variable and as a four-level categorical variable of readiness: low (<7), moderate (7-7.9), high (8-8.9), and very high (9-10).²⁰

Team SMM Convergence

To determine the teams’ convergence on patient discharge readiness, we calculated an adjusted interrater agreement index (r*_wg(j))^35,36 for each team using the individual clinicians’ scores on the RHDS/SF. These convergence values were categorized into four agreement levels: low agreement (<0.7), moderate agreement (0.7-0.79), high agreement (0.8-0.89), and very high agreement (0.9-1). See Appendix D for the r*_wg(j) equation.^35,36

Team-Patient SMM Convergence

To determine the team-patient SMM convergence, we subtracted the team’s assessment of patient discharge readiness from the Patient-RHDS/SF score. We used a one-unit change on the RHDS/SF (1 point on the 0-10 scale) as a meaningful difference between the patient’s self-assessment and teams’ assessment on readiness for hospital discharge. This definition for divergence aligns with prior RHDS psychometric testing studies^20,27 and research examining convergence between patient and nurse assessments.²⁸ For example, Weiss and colleagues²⁷ found a 1-point decrease in the RN-RHDS/SF item mean was associated with a 45% increase in likelihood of postdischarge utilization (hospital readmission and emergency room department visits). Therefore, we defined convergence of team-patient SMMs (or similar patient and team scores) as those with an absolute difference score less than 1 point, whereas teams with low team-patient SMM convergence (or divergent patient and team scores) were defined as having an absolute difference score greater than 1 point.

Prediction Models

For the exploratory aim, we first examined the bivariate relationship between the outcome variables (discharge teams’ assessment of patient readiness, team convergence, and team-patient convergence) and the identified contextual variables. We also checked for potential collinearity among the explanatory variables. Then we used a linear stepwise regression procedure to identify factors associated with each continuous outcome variable. Due to the small sample size, we performed separate backward stepwise regression selection analyses for the three outcomes of interest. The candidate explanatory variables were evaluated using P < .20 for model entry. Final models were evaluated using leave-one-out cross validation. STATA (v.15.1, StataCorp; 2017) was used for analysis.

Sample

A total of 64 discharge events were included in this study. All discharge teams had a unique composition including 64 patients and varying combinations of 56 individual nurses (n = 27), physicians (n = 23), and coordinators (n = 6). Each event had three team members (ie, a nurse, a coordinator, and a physician) with no missing data. The majority of the 64 patient participants were White, retired, had a high school education, and lived in their own home with only one other person (Table 1).

Interprofessional Teams’ SMM of Readiness for Hospital Discharge

While the majority of teams perceived patients had high readiness for hospital discharge (mean, 8.5 out of 10; SD, 0.91), patients scores were nearly a full point lower (mean, 7.7; SD, 1.6; Table 2). The largest difference across categories was in the low-readiness category with 27% of patient scores falling into this category vs only 9.4% of discharge team mean scores. The mean SMM convergence of team perception of patients’ readiness for discharge was 0.90 (SD, 0.10); however, scores ranged from 0.66 (low agreement) to 1 (complete agreement). The average SMM team-patient convergence, or the discrepancy between the discharge team mean scores and the patient total scores across domains, was 1.16 (SD, 0.82). Of the 64 discharge events, 42.2% had similar team-patient perceptions of readiness for discharge, 9.4% had the patient reporting higher readiness for discharge than the team, and 48.4% had a team assessment rating of higher readiness for discharge than the patient’s self-assessment.

Prediction Models

In the exploratory analysis, we created individual linear regression models to predict the teams’ assessment, team convergence, and team-patient convergence for readiness of hospital discharge (Table 3; Appendix E). Factors associated with the teams’ assessment of discharge readiness included whether the patient was married and had less cognitive impairment, both of which were positively related to a higher-­rated readiness among teams. An important system factor was higher quality of communication among team members, which was positively associated with teams’ assessment of patient discharge readiness. In contrast, only patient factors—married patients and those with a principal diagnosis of heart failure—were associated with more convergent team SMMs. Team-patient convergence was positively associated with two patient factors: marital status (married) and fewer comorbidities. However, team-patient convergence was also associated with two system factors: teams with a bachelor’s level–trained nurse (compared with a nurse with an associate degree ) and teams reporting a higher quality of teamwork on day of discharge.

Our study applied novel approaches to explore the interprofessional teams’ understanding of discharge readiness, a concept known to be an important predictor of patient outcomes after discharge, including readmission.^20,28 We found that discharge teams frequently had poor quality SMMs of hospital discharge readiness. Despite having a discharge order and receiving home care instructions, one in four patients reported low readiness for hospital discharge. Additionally, discharge teams frequently overestimated patient’s readiness for hospital discharge. Misalignment on patient readiness for discharge occurred both within the discharge team (ie, low team convergence) and between patients and their care teams (ie, low team-patient convergence). The potential importance of this disagreement is substantiated by prior work suggesting that divergence in readiness ratings between nurses and patients are associated with postdischarge coping difficulties.²⁸

Previous readiness for discharge has been measured from the perspective of the patient,^20,21,27,28 nurse,^20,25-28 and physician,³⁷ yet rarely has the teams’ perspective been examined. We add to this literature by measuring the team’s perspective, as well as agreement between team and patient, on the individual patient’s readiness for discharge. Notably, we found that higher-quality communication is positively related to teams’ assessment of discharge readiness, with teams that reported higher quality teamwork having more convergent team-patient SMMs. Our results support many qualitative studies identifying communication and teamwork as major factors in teams’ effectiveness in discharge planning.^1-7,9 However, given the small sample size in this study, additional research is needed to further understand these relationships, as well as link SMMs to patient outcomes such as hospital readmission.

In an attempt to improve discharge planning, hospitals are increasingly assessing readiness for discharge as a low-intensity, low-cost intervention.^26,27 Yet, recent evidence suggests that readiness assessments alone have minimal impact on reducing hospital readmissions.²⁶ To be successful, these assessments likely depend on quality interprofessional communication and ensuring the patient’s voice is incorporated into the discharge decision process.²⁶ However, there have been few ways to effectively evaluate these types of team interventions.⁹ Measuring SMM properties holds promise for identifying specific team mechanisms that may influence the effectiveness and fidelity of interventions for team-based discharge planning. As our findings indicate, SMMs provide a theoretical and methodological basis for evaluating if readiness for discharge was team based (convergence among team members) and patient centered (convergence among team assessment and patient self-assessment). Researchers and improvement scientists can use the approach outlined to evaluate team-based patient-centered interventions for hospital discharge planning.⁹

This study provides a unique contribution to the growing work in the team science of SMMs.^9,10 We rigorously evaluated SMMs of key stakeholders (patients and their interprofessional team) in “real-time” clinical practice using a patient-centered assessment linked with postdischarge outcomes.^20,27,28 However, it is still unknown how much convergence is needed (and with whom) to safely discharge patients.¹³ Prior studies suggest highly convergent SMMs increase team performance when they are also accurate.^10-13 Convergence alone should not be sought because this may reflect groupthink or clinical inertia.^10,15 To improve discharge team performance over time,^10‑13 it is important to assess not only patient’s readiness on the day of discharge but also how prepared the patient actually was for the recovery period following acute care. In the larger mixed-methods study, we found that teams’ with more convergent SMMs on teamwork quality were associated with patient’s reported quality of transition 30-days after discharge.⁹ Together, these findings further highlight the importance of aligning patient and interprofessional team members perspectives during the discharge planning, as well as providing clinicians with regular feedback about patient’s postdischarge experiences and outcomes.

To optimize team performance, the discharge planning process must be considered from an interprofessional team perspective as it functions in real-world practice settings. There are increasing pressures to discharge patients “quicker and sicker,” to simplify and standardize clinical process, and to provide patient-centered care.^3,5-8 Without thoughtful interventions to facilitate communication during discharge planning, these pressures likely reinforce inaccurate assumptions regarding the work of fellow team members and force teams to think “fast” instead of “slow.”^38-40 One approach to overcome such barriers is to focus on building a high-quality interprofessional SMM around discharge readiness. For example, the RHDS/SF questions could be integrated into the electronic medical records, displayed on dashboards, and discussed regularly during discharge rounds. In particular, to strengthen the team’s SMM and quality of teamwork, together the staff can ask three practical questions (Appendix F): (1) Do we think the patient is ready for discharge? (2) To what extent do we all agree the patient is ready for discharge? (3) Does our assessment of discharge readiness match the patient’s? During this high-risk transition point, asking these questions might allow the team to move from thinking fast to thinking slowly so they can more effectively identify heuristics they may be using inaccurately, prevent blind spots, and move toward high reliability.^{10,13,18,38-40}

This study has limitations. First, events were recruited from patients with any of only six conditions at a single hospital. Other settings, patient condition types, or team compositions of other clinicians may differ in results. Second, in this study the SMM content was focused on readiness for hospital discharge among four key stakeholders. It is possible other SMM content needs to be shared among the interprofessional discharge team (eg, caregivers’ perspectives,^2,6-8 resource availability,^3-6 clinicians’ roles^4,9) or additional members should be included (eg, physical therapists, nursing assistants, home health consultants, or primary care clinicians). Although this study focused on a patient-centered outcome (Patient-RHDS/SF), we did not examine other important outcomes such as hospital readmission. Additionally, due to the small sample size, these results have limited generalizability and should be interpreted with caution. Last, we limited data collection to the day of hospital discharge; future studies might consider assessing discharge readiness throughout hospitalization.

Readying patients for hospital discharge is a time-sensitivehigh-risk task requiring multiple healthcare professionals to concurrently assess patient needs, formulate an anticipatory care plan, provide education, and arrange for postdischarge needs.^20,21 Despite this, few studies have analyzed teamwork aspects to understand how these transitions could be improved.⁹ By piloting SMM measurement and describing factors that affect SMMs, we provide a step toward identifying and evaluating strategies to assist interprofessional care teams in preparing patients for a safe, high-quality, patient-centered hospital discharge.

This work was presented at the Midwest Nursing Research Society’s 2018 Annual Research Conference in Cleveland, Ohio, as well as at AcademyHealth’s 2019 Annual Research Meeting in Washington, District of Columbia.

Preparing patients for hospital discharge requires multiple tasks that cross professional boundaries. Clinician’s roles may be ambiguous, and responsibility for a safe high-quality discharge is often diffused among the team rather than being defined as the core responsibility of a single member.^1-8 Without a shared understanding of patient resources and tasks involved in anticipatory planning, lapses in discharge preparation can occur, which places patients at increased risk for harm after hospitalization.^3-7 As a result, organizations like the Centers for Medicare & Medicaid Services (CMS) have called for team-based patient-centered discharge planning.⁸ Yet to develop more effective team-based discharge planning interventions, a more nuanced understanding of how healthcare teams work together is needed.^2,3,9

Shared mental models (SMMs) provide a useful theoretical framework and measurement approach for examining how interprofessional teams coordinate complex tasks like hospital discharge.^10-13 SMMs represent the team members’ collective understanding and organized knowledge of key elements needed for teams to perform effectively.^9-11 Validated questionnaires can be used to measure two key properties of SMMs: the degree to which team members have a similar understanding of the situation at hand (team SMM convergence) and to what extent this understanding is aligned with the patient (team-­patient SMM convergence).^10,11 Researchers have found that teams with higher-quality SMMs have a better understanding of what is happening and why, have clearer expectations of their roles and tasks, and can better predict what might happen next.^10,12 As a result, these teams more effectively coordinate actions and adapt to task demands even in cases of high complexity, uncertainty, and stress.^10-13 Prior studies examining healthcare teams in emergency departments,^14-16 critical care units,^17,18 and operating rooms¹⁹ suggest high-quality SMMs are needed to safely care for patients.¹³ Yet there has been limited evaluation of SMMs in general internal medicine, much less during hospital discharge.^9,13 The purpose of this study was to examine SMMs for a critical task of the inpatient team: developing a shared understanding of the patient’s readiness for hospital discharge.^20,21

Design

We used a cross-sectional survey design at a single Midwestern community hospital to determine inpatient care teams’ SMMs of patient hospital discharge readiness. This study is part of a larger mixed-methods evaluation of interprofessional hospital discharge teamwork in older adult patients at risk for a poor transition to home.⁹ Data were collected using questionnaires from patients and their team (nurse, coordinator, and physician) within 4 hours of the patient leaving the hospital. First, we measured the teams’ assessment, team convergence, and team-­patient convergence on patient readiness for discharge from the hospital. Then, after identifying relevant potential predictors from the literature, we developed regression models to predict the teams’ assessment, team convergence, and team-patient convergence of discharge readiness based on these variables. Our local institutional review board approved this study.

Sample and Participants

We used a convenience sampling approach to identify eligible discharge events consisting of the patient and care team.⁹ We focused on patients at high-risk for poor hospital-to-home transitions.^3,22 Eligible events included older patients (≥65 years) who were discharged home without home health or hospice services and admitted with a primary diagnosis of heart failure, acute myocardial infarction, hip replacement, knee replacement, pneumonia, or chronic obstructive pulmonary disease. Patient exclusion criteria included inability to complete study forms because of mental incapacity or a language barrier. Discharge team member inclusion criteria included the bedside nurse, attending physician, and coordinator (a unit-dedicated discharge nurse or social worker) caring for the patient participant at the time of hospital discharge. Each discharge team was unique: The same three individuals could not be included as a “team” for more than one discharge event, although individual members could be included as a part of other teams with a different set of individuals. Appendix A provides an enrollment flowchart.

Conceptual Framework

We applied the SMM conceptual framework to the context of hospital discharge. As shown in the Figure, SMMs are examined at the team level and contain the critical knowledge held by the team to be effective.^15,16 From a patient-centered perspective, patients are considered the expert on how ready they feel to be discharged home.^20,23,24 In this case, the SMM content is the discharge team members’ shared assessment of how ready the patient is for hospital discharge (Figure, B).¹⁰ Convergence is the degree of agreement among individual mental models.^10-13 In this study we examined two types of convergence: (1) team convergence, or the team members degree of agreement on the patient’s readiness for discharge (Figure, C), and (2) team-patient convergence, or the degree to which the team’s SMM aligns with the patient’s mental model (Figure, D).^10-13

Measures and Variables

Readiness for Hospital Discharge Scales/Short Form

We used parallel clinician and patient versions of the Readiness for Hospital Discharge Scale/Short Form (RHDS/SF)^25-28 to determine the teams’ assessment of discharge readiness, team SMM convergence, and team-patient SMM convergence.

The RHDS/SF scales are 8-item validated instruments that use a Likert scale (0 for not ready to 10 for totally ready) to assess the individual clinician’s or patient’s perceptions of how ready the patient is to be discharged.^20,25,27 The RHDS/SF instruments include four dimensions conceptualized as crucial to patient readiness for discharge and important to anticipatory planning: (1) Personal Status, physical-emotional state of the patient before discharge; (2) Knowledge, perceived adequacy of information needed to respond to common posthospitalization concerns/problems; (3) Coping Ability, perceived ability to self-manage health care needs; and (4) Expected Support, emotional-physical assistance available (Appendix B).^20,25,27 The RHDS/SF instruments’ results are calculated as a mean of item scores, with higher individual scores indicating the rater assessed the patient as being more ready for hospital discharge.²⁰ The RHDS/SF scales have undergone rigorous psychometric testing and are linked to patient outcomes (eg, readmissions, emergency room visits, patient coping difficulties after discharge, and patient-rated quality of preparation for posthospital care).^20,25-28 For example, predictive validity assessments for adult medical-surgical patients found lower Nurse-RHDS/SF scores are associated with a six- to ninefold increase in 30-day readmission risk.²⁰

Contextual Variables

We reviewed the literature to identify potential patient and system factors associated with adverse transitional care outcomes^1-8 and/or higher quality SMMs in other settings.^10-19 For example, patient characteristics included age, principal diagnosis, length of stay, number of comorbidities, and cognition impairment (using the Short Portable Mini Mental Status Questionnaire²⁹).^2,22,30 Examples of system factor include teamwork and communication quality^1-6 on day of discharge, as well as educational background and experience of clinicians on the team.^31-33 We adapted a validated survey using 7-point Likert scale questions to determine teamwork quality and communication quality during individual patent discharges.³³Appendix C provides descriptions of all variables.

Data Collection

Patient recruitment occurred from February to October 2017 in a single community hospital in Iowa.⁹ We identified potentially eligible events in collaboration with the unit charge nurses. Patients were screened 24 to 48 hours prior to anticipated day of discharge; those interested/eligible underwent informed consent procedures.⁹ We collected data from the patient and their corresponding bedside nurse, coordinator, and attending physician on the day of discharge. After the discharge order was placed and care instructions were provided, the patient completed a demographic survey, Short Portable Mini Mental Status Questionnaire, and the Patient-RHDS/SF. Individual team members completed a survey with the demographic information, their respective versions of RHDS/SF, and day-of-discharge teamwork-related questions. On average, the survey took clinicians less than 5 minutes to complete. We performed a chart review to determine additional patient characteristics such as principal diagnosis, length of stay, and number of comorbidities.

Data Analysis

Team Assessment of Patient Discharge Readiness

The teams’ shared assessment (SMM content) was determined by averaging the members’ individual scores on the Clinician-RHDS/SF.³⁴ Discharge events with higher team assessments indicated the team perceived the patient as being readier for hospital discharge. Guided by prior research, we examined the RHDS/SF scores as a continuous variable and as a four-level categorical variable of readiness: low (<7), moderate (7-7.9), high (8-8.9), and very high (9-10).²⁰

Team SMM Convergence

To determine the teams’ convergence on patient discharge readiness, we calculated an adjusted interrater agreement index (r*_wg(j))^35,36 for each team using the individual clinicians’ scores on the RHDS/SF. These convergence values were categorized into four agreement levels: low agreement (<0.7), moderate agreement (0.7-0.79), high agreement (0.8-0.89), and very high agreement (0.9-1). See Appendix D for the r*_wg(j) equation.^35,36

Team-Patient SMM Convergence

To determine the team-patient SMM convergence, we subtracted the team’s assessment of patient discharge readiness from the Patient-RHDS/SF score. We used a one-unit change on the RHDS/SF (1 point on the 0-10 scale) as a meaningful difference between the patient’s self-assessment and teams’ assessment on readiness for hospital discharge. This definition for divergence aligns with prior RHDS psychometric testing studies^20,27 and research examining convergence between patient and nurse assessments.²⁸ For example, Weiss and colleagues²⁷ found a 1-point decrease in the RN-RHDS/SF item mean was associated with a 45% increase in likelihood of postdischarge utilization (hospital readmission and emergency room department visits). Therefore, we defined convergence of team-patient SMMs (or similar patient and team scores) as those with an absolute difference score less than 1 point, whereas teams with low team-patient SMM convergence (or divergent patient and team scores) were defined as having an absolute difference score greater than 1 point.

Prediction Models

For the exploratory aim, we first examined the bivariate relationship between the outcome variables (discharge teams’ assessment of patient readiness, team convergence, and team-patient convergence) and the identified contextual variables. We also checked for potential collinearity among the explanatory variables. Then we used a linear stepwise regression procedure to identify factors associated with each continuous outcome variable. Due to the small sample size, we performed separate backward stepwise regression selection analyses for the three outcomes of interest. The candidate explanatory variables were evaluated using P < .20 for model entry. Final models were evaluated using leave-one-out cross validation. STATA (v.15.1, StataCorp; 2017) was used for analysis.

Sample

A total of 64 discharge events were included in this study. All discharge teams had a unique composition including 64 patients and varying combinations of 56 individual nurses (n = 27), physicians (n = 23), and coordinators (n = 6). Each event had three team members (ie, a nurse, a coordinator, and a physician) with no missing data. The majority of the 64 patient participants were White, retired, had a high school education, and lived in their own home with only one other person (Table 1).

Interprofessional Teams’ SMM of Readiness for Hospital Discharge

While the majority of teams perceived patients had high readiness for hospital discharge (mean, 8.5 out of 10; SD, 0.91), patients scores were nearly a full point lower (mean, 7.7; SD, 1.6; Table 2). The largest difference across categories was in the low-readiness category with 27% of patient scores falling into this category vs only 9.4% of discharge team mean scores. The mean SMM convergence of team perception of patients’ readiness for discharge was 0.90 (SD, 0.10); however, scores ranged from 0.66 (low agreement) to 1 (complete agreement). The average SMM team-patient convergence, or the discrepancy between the discharge team mean scores and the patient total scores across domains, was 1.16 (SD, 0.82). Of the 64 discharge events, 42.2% had similar team-patient perceptions of readiness for discharge, 9.4% had the patient reporting higher readiness for discharge than the team, and 48.4% had a team assessment rating of higher readiness for discharge than the patient’s self-assessment.

Prediction Models

In the exploratory analysis, we created individual linear regression models to predict the teams’ assessment, team convergence, and team-patient convergence for readiness of hospital discharge (Table 3; Appendix E). Factors associated with the teams’ assessment of discharge readiness included whether the patient was married and had less cognitive impairment, both of which were positively related to a higher-­rated readiness among teams. An important system factor was higher quality of communication among team members, which was positively associated with teams’ assessment of patient discharge readiness. In contrast, only patient factors—married patients and those with a principal diagnosis of heart failure—were associated with more convergent team SMMs. Team-patient convergence was positively associated with two patient factors: marital status (married) and fewer comorbidities. However, team-patient convergence was also associated with two system factors: teams with a bachelor’s level–trained nurse (compared with a nurse with an associate degree ) and teams reporting a higher quality of teamwork on day of discharge.

Our study applied novel approaches to explore the interprofessional teams’ understanding of discharge readiness, a concept known to be an important predictor of patient outcomes after discharge, including readmission.^20,28 We found that discharge teams frequently had poor quality SMMs of hospital discharge readiness. Despite having a discharge order and receiving home care instructions, one in four patients reported low readiness for hospital discharge. Additionally, discharge teams frequently overestimated patient’s readiness for hospital discharge. Misalignment on patient readiness for discharge occurred both within the discharge team (ie, low team convergence) and between patients and their care teams (ie, low team-patient convergence). The potential importance of this disagreement is substantiated by prior work suggesting that divergence in readiness ratings between nurses and patients are associated with postdischarge coping difficulties.²⁸

Previous readiness for discharge has been measured from the perspective of the patient,^20,21,27,28 nurse,^20,25-28 and physician,³⁷ yet rarely has the teams’ perspective been examined. We add to this literature by measuring the team’s perspective, as well as agreement between team and patient, on the individual patient’s readiness for discharge. Notably, we found that higher-quality communication is positively related to teams’ assessment of discharge readiness, with teams that reported higher quality teamwork having more convergent team-patient SMMs. Our results support many qualitative studies identifying communication and teamwork as major factors in teams’ effectiveness in discharge planning.^1-7,9 However, given the small sample size in this study, additional research is needed to further understand these relationships, as well as link SMMs to patient outcomes such as hospital readmission.

In an attempt to improve discharge planning, hospitals are increasingly assessing readiness for discharge as a low-intensity, low-cost intervention.^26,27 Yet, recent evidence suggests that readiness assessments alone have minimal impact on reducing hospital readmissions.²⁶ To be successful, these assessments likely depend on quality interprofessional communication and ensuring the patient’s voice is incorporated into the discharge decision process.²⁶ However, there have been few ways to effectively evaluate these types of team interventions.⁹ Measuring SMM properties holds promise for identifying specific team mechanisms that may influence the effectiveness and fidelity of interventions for team-based discharge planning. As our findings indicate, SMMs provide a theoretical and methodological basis for evaluating if readiness for discharge was team based (convergence among team members) and patient centered (convergence among team assessment and patient self-assessment). Researchers and improvement scientists can use the approach outlined to evaluate team-based patient-centered interventions for hospital discharge planning.⁹

This study provides a unique contribution to the growing work in the team science of SMMs.^9,10 We rigorously evaluated SMMs of key stakeholders (patients and their interprofessional team) in “real-time” clinical practice using a patient-centered assessment linked with postdischarge outcomes.^20,27,28 However, it is still unknown how much convergence is needed (and with whom) to safely discharge patients.¹³ Prior studies suggest highly convergent SMMs increase team performance when they are also accurate.^10-13 Convergence alone should not be sought because this may reflect groupthink or clinical inertia.^10,15 To improve discharge team performance over time,^10‑13 it is important to assess not only patient’s readiness on the day of discharge but also how prepared the patient actually was for the recovery period following acute care. In the larger mixed-methods study, we found that teams’ with more convergent SMMs on teamwork quality were associated with patient’s reported quality of transition 30-days after discharge.⁹ Together, these findings further highlight the importance of aligning patient and interprofessional team members perspectives during the discharge planning, as well as providing clinicians with regular feedback about patient’s postdischarge experiences and outcomes.

To optimize team performance, the discharge planning process must be considered from an interprofessional team perspective as it functions in real-world practice settings. There are increasing pressures to discharge patients “quicker and sicker,” to simplify and standardize clinical process, and to provide patient-centered care.^3,5-8 Without thoughtful interventions to facilitate communication during discharge planning, these pressures likely reinforce inaccurate assumptions regarding the work of fellow team members and force teams to think “fast” instead of “slow.”^38-40 One approach to overcome such barriers is to focus on building a high-quality interprofessional SMM around discharge readiness. For example, the RHDS/SF questions could be integrated into the electronic medical records, displayed on dashboards, and discussed regularly during discharge rounds. In particular, to strengthen the team’s SMM and quality of teamwork, together the staff can ask three practical questions (Appendix F): (1) Do we think the patient is ready for discharge? (2) To what extent do we all agree the patient is ready for discharge? (3) Does our assessment of discharge readiness match the patient’s? During this high-risk transition point, asking these questions might allow the team to move from thinking fast to thinking slowly so they can more effectively identify heuristics they may be using inaccurately, prevent blind spots, and move toward high reliability.^{10,13,18,38-40}

This study has limitations. First, events were recruited from patients with any of only six conditions at a single hospital. Other settings, patient condition types, or team compositions of other clinicians may differ in results. Second, in this study the SMM content was focused on readiness for hospital discharge among four key stakeholders. It is possible other SMM content needs to be shared among the interprofessional discharge team (eg, caregivers’ perspectives,^2,6-8 resource availability,^3-6 clinicians’ roles^4,9) or additional members should be included (eg, physical therapists, nursing assistants, home health consultants, or primary care clinicians). Although this study focused on a patient-centered outcome (Patient-RHDS/SF), we did not examine other important outcomes such as hospital readmission. Additionally, due to the small sample size, these results have limited generalizability and should be interpreted with caution. Last, we limited data collection to the day of hospital discharge; future studies might consider assessing discharge readiness throughout hospitalization.

Readying patients for hospital discharge is a time-sensitivehigh-risk task requiring multiple healthcare professionals to concurrently assess patient needs, formulate an anticipatory care plan, provide education, and arrange for postdischarge needs.^20,21 Despite this, few studies have analyzed teamwork aspects to understand how these transitions could be improved.⁹ By piloting SMM measurement and describing factors that affect SMMs, we provide a step toward identifying and evaluating strategies to assist interprofessional care teams in preparing patients for a safe, high-quality, patient-centered hospital discharge.

This work was presented at the Midwest Nursing Research Society’s 2018 Annual Research Conference in Cleveland, Ohio, as well as at AcademyHealth’s 2019 Annual Research Meeting in Washington, District of Columbia.

Preparing patients for hospital discharge requires multiple tasks that cross professional boundaries. Clinician’s roles may be ambiguous, and responsibility for a safe high-quality discharge is often diffused among the team rather than being defined as the core responsibility of a single member.^1-8 Without a shared understanding of patient resources and tasks involved in anticipatory planning, lapses in discharge preparation can occur, which places patients at increased risk for harm after hospitalization.^3-7 As a result, organizations like the Centers for Medicare & Medicaid Services (CMS) have called for team-based patient-centered discharge planning.⁸ Yet to develop more effective team-based discharge planning interventions, a more nuanced understanding of how healthcare teams work together is needed.^2,3,9

Shared mental models (SMMs) provide a useful theoretical framework and measurement approach for examining how interprofessional teams coordinate complex tasks like hospital discharge.^10-13 SMMs represent the team members’ collective understanding and organized knowledge of key elements needed for teams to perform effectively.^9-11 Validated questionnaires can be used to measure two key properties of SMMs: the degree to which team members have a similar understanding of the situation at hand (team SMM convergence) and to what extent this understanding is aligned with the patient (team-­patient SMM convergence).^10,11 Researchers have found that teams with higher-quality SMMs have a better understanding of what is happening and why, have clearer expectations of their roles and tasks, and can better predict what might happen next.^10,12 As a result, these teams more effectively coordinate actions and adapt to task demands even in cases of high complexity, uncertainty, and stress.^10-13 Prior studies examining healthcare teams in emergency departments,^14-16 critical care units,^17,18 and operating rooms¹⁹ suggest high-quality SMMs are needed to safely care for patients.¹³ Yet there has been limited evaluation of SMMs in general internal medicine, much less during hospital discharge.^9,13 The purpose of this study was to examine SMMs for a critical task of the inpatient team: developing a shared understanding of the patient’s readiness for hospital discharge.^20,21

Design

We used a cross-sectional survey design at a single Midwestern community hospital to determine inpatient care teams’ SMMs of patient hospital discharge readiness. This study is part of a larger mixed-methods evaluation of interprofessional hospital discharge teamwork in older adult patients at risk for a poor transition to home.⁹ Data were collected using questionnaires from patients and their team (nurse, coordinator, and physician) within 4 hours of the patient leaving the hospital. First, we measured the teams’ assessment, team convergence, and team-­patient convergence on patient readiness for discharge from the hospital. Then, after identifying relevant potential predictors from the literature, we developed regression models to predict the teams’ assessment, team convergence, and team-patient convergence of discharge readiness based on these variables. Our local institutional review board approved this study.

Sample and Participants

We used a convenience sampling approach to identify eligible discharge events consisting of the patient and care team.⁹ We focused on patients at high-risk for poor hospital-to-home transitions.^3,22 Eligible events included older patients (≥65 years) who were discharged home without home health or hospice services and admitted with a primary diagnosis of heart failure, acute myocardial infarction, hip replacement, knee replacement, pneumonia, or chronic obstructive pulmonary disease. Patient exclusion criteria included inability to complete study forms because of mental incapacity or a language barrier. Discharge team member inclusion criteria included the bedside nurse, attending physician, and coordinator (a unit-dedicated discharge nurse or social worker) caring for the patient participant at the time of hospital discharge. Each discharge team was unique: The same three individuals could not be included as a “team” for more than one discharge event, although individual members could be included as a part of other teams with a different set of individuals. Appendix A provides an enrollment flowchart.

Conceptual Framework

We applied the SMM conceptual framework to the context of hospital discharge. As shown in the Figure, SMMs are examined at the team level and contain the critical knowledge held by the team to be effective.^15,16 From a patient-centered perspective, patients are considered the expert on how ready they feel to be discharged home.^20,23,24 In this case, the SMM content is the discharge team members’ shared assessment of how ready the patient is for hospital discharge (Figure, B).¹⁰ Convergence is the degree of agreement among individual mental models.^10-13 In this study we examined two types of convergence: (1) team convergence, or the team members degree of agreement on the patient’s readiness for discharge (Figure, C), and (2) team-patient convergence, or the degree to which the team’s SMM aligns with the patient’s mental model (Figure, D).^10-13

Measures and Variables

Readiness for Hospital Discharge Scales/Short Form

We used parallel clinician and patient versions of the Readiness for Hospital Discharge Scale/Short Form (RHDS/SF)^25-28 to determine the teams’ assessment of discharge readiness, team SMM convergence, and team-patient SMM convergence.

The RHDS/SF scales are 8-item validated instruments that use a Likert scale (0 for not ready to 10 for totally ready) to assess the individual clinician’s or patient’s perceptions of how ready the patient is to be discharged.^20,25,27 The RHDS/SF instruments include four dimensions conceptualized as crucial to patient readiness for discharge and important to anticipatory planning: (1) Personal Status, physical-emotional state of the patient before discharge; (2) Knowledge, perceived adequacy of information needed to respond to common posthospitalization concerns/problems; (3) Coping Ability, perceived ability to self-manage health care needs; and (4) Expected Support, emotional-physical assistance available (Appendix B).^20,25,27 The RHDS/SF instruments’ results are calculated as a mean of item scores, with higher individual scores indicating the rater assessed the patient as being more ready for hospital discharge.²⁰ The RHDS/SF scales have undergone rigorous psychometric testing and are linked to patient outcomes (eg, readmissions, emergency room visits, patient coping difficulties after discharge, and patient-rated quality of preparation for posthospital care).^20,25-28 For example, predictive validity assessments for adult medical-surgical patients found lower Nurse-RHDS/SF scores are associated with a six- to ninefold increase in 30-day readmission risk.²⁰

Contextual Variables

We reviewed the literature to identify potential patient and system factors associated with adverse transitional care outcomes^1-8 and/or higher quality SMMs in other settings.^10-19 For example, patient characteristics included age, principal diagnosis, length of stay, number of comorbidities, and cognition impairment (using the Short Portable Mini Mental Status Questionnaire²⁹).^2,22,30 Examples of system factor include teamwork and communication quality^1-6 on day of discharge, as well as educational background and experience of clinicians on the team.^31-33 We adapted a validated survey using 7-point Likert scale questions to determine teamwork quality and communication quality during individual patent discharges.³³Appendix C provides descriptions of all variables.

Data Collection

Patient recruitment occurred from February to October 2017 in a single community hospital in Iowa.⁹ We identified potentially eligible events in collaboration with the unit charge nurses. Patients were screened 24 to 48 hours prior to anticipated day of discharge; those interested/eligible underwent informed consent procedures.⁹ We collected data from the patient and their corresponding bedside nurse, coordinator, and attending physician on the day of discharge. After the discharge order was placed and care instructions were provided, the patient completed a demographic survey, Short Portable Mini Mental Status Questionnaire, and the Patient-RHDS/SF. Individual team members completed a survey with the demographic information, their respective versions of RHDS/SF, and day-of-discharge teamwork-related questions. On average, the survey took clinicians less than 5 minutes to complete. We performed a chart review to determine additional patient characteristics such as principal diagnosis, length of stay, and number of comorbidities.

Data Analysis

Team Assessment of Patient Discharge Readiness

The teams’ shared assessment (SMM content) was determined by averaging the members’ individual scores on the Clinician-RHDS/SF.³⁴ Discharge events with higher team assessments indicated the team perceived the patient as being readier for hospital discharge. Guided by prior research, we examined the RHDS/SF scores as a continuous variable and as a four-level categorical variable of readiness: low (<7), moderate (7-7.9), high (8-8.9), and very high (9-10).²⁰

Team SMM Convergence

To determine the teams’ convergence on patient discharge readiness, we calculated an adjusted interrater agreement index (r*_wg(j))^35,36 for each team using the individual clinicians’ scores on the RHDS/SF. These convergence values were categorized into four agreement levels: low agreement (<0.7), moderate agreement (0.7-0.79), high agreement (0.8-0.89), and very high agreement (0.9-1). See Appendix D for the r*_wg(j) equation.^35,36

Team-Patient SMM Convergence

To determine the team-patient SMM convergence, we subtracted the team’s assessment of patient discharge readiness from the Patient-RHDS/SF score. We used a one-unit change on the RHDS/SF (1 point on the 0-10 scale) as a meaningful difference between the patient’s self-assessment and teams’ assessment on readiness for hospital discharge. This definition for divergence aligns with prior RHDS psychometric testing studies^20,27 and research examining convergence between patient and nurse assessments.²⁸ For example, Weiss and colleagues²⁷ found a 1-point decrease in the RN-RHDS/SF item mean was associated with a 45% increase in likelihood of postdischarge utilization (hospital readmission and emergency room department visits). Therefore, we defined convergence of team-patient SMMs (or similar patient and team scores) as those with an absolute difference score less than 1 point, whereas teams with low team-patient SMM convergence (or divergent patient and team scores) were defined as having an absolute difference score greater than 1 point.

Prediction Models

For the exploratory aim, we first examined the bivariate relationship between the outcome variables (discharge teams’ assessment of patient readiness, team convergence, and team-patient convergence) and the identified contextual variables. We also checked for potential collinearity among the explanatory variables. Then we used a linear stepwise regression procedure to identify factors associated with each continuous outcome variable. Due to the small sample size, we performed separate backward stepwise regression selection analyses for the three outcomes of interest. The candidate explanatory variables were evaluated using P < .20 for model entry. Final models were evaluated using leave-one-out cross validation. STATA (v.15.1, StataCorp; 2017) was used for analysis.

Sample

A total of 64 discharge events were included in this study. All discharge teams had a unique composition including 64 patients and varying combinations of 56 individual nurses (n = 27), physicians (n = 23), and coordinators (n = 6). Each event had three team members (ie, a nurse, a coordinator, and a physician) with no missing data. The majority of the 64 patient participants were White, retired, had a high school education, and lived in their own home with only one other person (Table 1).

Interprofessional Teams’ SMM of Readiness for Hospital Discharge

While the majority of teams perceived patients had high readiness for hospital discharge (mean, 8.5 out of 10; SD, 0.91), patients scores were nearly a full point lower (mean, 7.7; SD, 1.6; Table 2). The largest difference across categories was in the low-readiness category with 27% of patient scores falling into this category vs only 9.4% of discharge team mean scores. The mean SMM convergence of team perception of patients’ readiness for discharge was 0.90 (SD, 0.10); however, scores ranged from 0.66 (low agreement) to 1 (complete agreement). The average SMM team-patient convergence, or the discrepancy between the discharge team mean scores and the patient total scores across domains, was 1.16 (SD, 0.82). Of the 64 discharge events, 42.2% had similar team-patient perceptions of readiness for discharge, 9.4% had the patient reporting higher readiness for discharge than the team, and 48.4% had a team assessment rating of higher readiness for discharge than the patient’s self-assessment.

Prediction Models

In the exploratory analysis, we created individual linear regression models to predict the teams’ assessment, team convergence, and team-patient convergence for readiness of hospital discharge (Table 3; Appendix E). Factors associated with the teams’ assessment of discharge readiness included whether the patient was married and had less cognitive impairment, both of which were positively related to a higher-­rated readiness among teams. An important system factor was higher quality of communication among team members, which was positively associated with teams’ assessment of patient discharge readiness. In contrast, only patient factors—married patients and those with a principal diagnosis of heart failure—were associated with more convergent team SMMs. Team-patient convergence was positively associated with two patient factors: marital status (married) and fewer comorbidities. However, team-patient convergence was also associated with two system factors: teams with a bachelor’s level–trained nurse (compared with a nurse with an associate degree ) and teams reporting a higher quality of teamwork on day of discharge.

Our study applied novel approaches to explore the interprofessional teams’ understanding of discharge readiness, a concept known to be an important predictor of patient outcomes after discharge, including readmission.^20,28 We found that discharge teams frequently had poor quality SMMs of hospital discharge readiness. Despite having a discharge order and receiving home care instructions, one in four patients reported low readiness for hospital discharge. Additionally, discharge teams frequently overestimated patient’s readiness for hospital discharge. Misalignment on patient readiness for discharge occurred both within the discharge team (ie, low team convergence) and between patients and their care teams (ie, low team-patient convergence). The potential importance of this disagreement is substantiated by prior work suggesting that divergence in readiness ratings between nurses and patients are associated with postdischarge coping difficulties.²⁸

Previous readiness for discharge has been measured from the perspective of the patient,^20,21,27,28 nurse,^20,25-28 and physician,³⁷ yet rarely has the teams’ perspective been examined. We add to this literature by measuring the team’s perspective, as well as agreement between team and patient, on the individual patient’s readiness for discharge. Notably, we found that higher-quality communication is positively related to teams’ assessment of discharge readiness, with teams that reported higher quality teamwork having more convergent team-patient SMMs. Our results support many qualitative studies identifying communication and teamwork as major factors in teams’ effectiveness in discharge planning.^1-7,9 However, given the small sample size in this study, additional research is needed to further understand these relationships, as well as link SMMs to patient outcomes such as hospital readmission.

In an attempt to improve discharge planning, hospitals are increasingly assessing readiness for discharge as a low-intensity, low-cost intervention.^26,27 Yet, recent evidence suggests that readiness assessments alone have minimal impact on reducing hospital readmissions.²⁶ To be successful, these assessments likely depend on quality interprofessional communication and ensuring the patient’s voice is incorporated into the discharge decision process.²⁶ However, there have been few ways to effectively evaluate these types of team interventions.⁹ Measuring SMM properties holds promise for identifying specific team mechanisms that may influence the effectiveness and fidelity of interventions for team-based discharge planning. As our findings indicate, SMMs provide a theoretical and methodological basis for evaluating if readiness for discharge was team based (convergence among team members) and patient centered (convergence among team assessment and patient self-assessment). Researchers and improvement scientists can use the approach outlined to evaluate team-based patient-centered interventions for hospital discharge planning.⁹

This study provides a unique contribution to the growing work in the team science of SMMs.^9,10 We rigorously evaluated SMMs of key stakeholders (patients and their interprofessional team) in “real-time” clinical practice using a patient-centered assessment linked with postdischarge outcomes.^20,27,28 However, it is still unknown how much convergence is needed (and with whom) to safely discharge patients.¹³ Prior studies suggest highly convergent SMMs increase team performance when they are also accurate.^10-13 Convergence alone should not be sought because this may reflect groupthink or clinical inertia.^10,15 To improve discharge team performance over time,^10‑13 it is important to assess not only patient’s readiness on the day of discharge but also how prepared the patient actually was for the recovery period following acute care. In the larger mixed-methods study, we found that teams’ with more convergent SMMs on teamwork quality were associated with patient’s reported quality of transition 30-days after discharge.⁹ Together, these findings further highlight the importance of aligning patient and interprofessional team members perspectives during the discharge planning, as well as providing clinicians with regular feedback about patient’s postdischarge experiences and outcomes.

To optimize team performance, the discharge planning process must be considered from an interprofessional team perspective as it functions in real-world practice settings. There are increasing pressures to discharge patients “quicker and sicker,” to simplify and standardize clinical process, and to provide patient-centered care.^3,5-8 Without thoughtful interventions to facilitate communication during discharge planning, these pressures likely reinforce inaccurate assumptions regarding the work of fellow team members and force teams to think “fast” instead of “slow.”^38-40 One approach to overcome such barriers is to focus on building a high-quality interprofessional SMM around discharge readiness. For example, the RHDS/SF questions could be integrated into the electronic medical records, displayed on dashboards, and discussed regularly during discharge rounds. In particular, to strengthen the team’s SMM and quality of teamwork, together the staff can ask three practical questions (Appendix F): (1) Do we think the patient is ready for discharge? (2) To what extent do we all agree the patient is ready for discharge? (3) Does our assessment of discharge readiness match the patient’s? During this high-risk transition point, asking these questions might allow the team to move from thinking fast to thinking slowly so they can more effectively identify heuristics they may be using inaccurately, prevent blind spots, and move toward high reliability.^{10,13,18,38-40}

This study has limitations. First, events were recruited from patients with any of only six conditions at a single hospital. Other settings, patient condition types, or team compositions of other clinicians may differ in results. Second, in this study the SMM content was focused on readiness for hospital discharge among four key stakeholders. It is possible other SMM content needs to be shared among the interprofessional discharge team (eg, caregivers’ perspectives,^2,6-8 resource availability,^3-6 clinicians’ roles^4,9) or additional members should be included (eg, physical therapists, nursing assistants, home health consultants, or primary care clinicians). Although this study focused on a patient-centered outcome (Patient-RHDS/SF), we did not examine other important outcomes such as hospital readmission. Additionally, due to the small sample size, these results have limited generalizability and should be interpreted with caution. Last, we limited data collection to the day of hospital discharge; future studies might consider assessing discharge readiness throughout hospitalization.

Readying patients for hospital discharge is a time-sensitivehigh-risk task requiring multiple healthcare professionals to concurrently assess patient needs, formulate an anticipatory care plan, provide education, and arrange for postdischarge needs.^20,21 Despite this, few studies have analyzed teamwork aspects to understand how these transitions could be improved.⁹ By piloting SMM measurement and describing factors that affect SMMs, we provide a step toward identifying and evaluating strategies to assist interprofessional care teams in preparing patients for a safe, high-quality, patient-centered hospital discharge.

This work was presented at the Midwest Nursing Research Society’s 2018 Annual Research Conference in Cleveland, Ohio, as well as at AcademyHealth’s 2019 Annual Research Meeting in Washington, District of Columbia.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Leptospirosis (LS) is considered the most common and widespread zoonotic disease in the world. Numerous outbreaks have occurred in the past 10 years. Due to its technically difficult diagnosis, LS is severely underrecognized, underdiagnosed, and therefore, underreported.^1,2 The Centers for Disease Control and Prevention (CDC) estimate 100 to 150 cases of LS are identified annually in the US, with about 50% of those cases occurring in Puerto Rico (PR).³ Specifically in PR, about 15 to 100 cases of suspected LS were reported annually between 2000 and 2009, with 59 cases and 1 death reported in 2010. The data are thought to be severely underreported due to a lack of widespread diagnostic testing availability in PR and no formal veterinary and environmental surveillance programs to monitor the incidence of animal cases and actual circulating serovars.⁴

A recent systematic review of 80 studies from 34 countries on morbidity and mortality of LS revealed that the global incidence and mortality is about 1.03 million cases and 58,900 deaths every year. Almost half of the reported deaths were adult males aged 20 to 49 years.⁵ Although mild cases of LS are not associated with an elevated mortality, icteric LS with renal failure (Weil disease) carries a mortality rate of 10%.⁶ In patients who develop hemorrhagic pneumonitis, mortality may be as high as 50 to 70%.⁷ Therefore, it is pivotal that clinicians recognize the disease early, that novel modalities of treatment continue to be developed, and that their impact on patient morbidity and mortality are studied and documented.

Case Presentation

A 43-year-old man with a medical history of schizophrenia presented to the emergency department at the US Department of Veterans Affairs (VA) Caribbean Healthcare System in San Juan, PR, after experiencing 1 week of intermittent fever, myalgia, and general weakness. Emergency medical services had found him disheveled and in a rodent-infested swamp area several days before admission. Initial vital signs were within normal limits.

On physical examination, the patient was afebrile, without acute distress, but he had diffuse jaundice and mild epigastric tenderness without evidence of peritoneal irritation. His complete blood count was remarkable for leukocytosis with left shifting, adequate hemoglobin levels but with 9 × 10³ U/L platelets. The complete metabolic panel demonstrated an aspartate aminotransferase level of 564 U/L, alanine transaminase level of 462 U/L, total bilirubin of 12 mg/dL, which 10.2 mg/dL were direct bilirubin, and an alkaline phosphate of 345 U/L. Lipase levels were measured at 626 U/L. Marked coagulopathy also was present. The toxicology panel, including acetaminophen and salicylate acid levels, did not reveal the presence of any of the tested substances, and chest imaging did not demonstrate any infiltrates.

An abdominal ultrasound was negative for acute cholestatic pathologies, such as cholelithiasis, cholecystitis, or choledocholithiasis. Nonetheless, a noncontrast abdominopelvic computed tomography was remarkable for peripancreatic fat stranding, which raised suspicion for a diagnosis of pancreatitis.

Once the patient was transferred to the intensive care unit, he developed several episodes of hematemesis, leading to hemodynamical instability and severe respiratory distress. Due to anticipated respiratory failure and need for airway securement, endotracheal intubation was performed. Multiple packed red blood cells were transfused, and the patient was started in vasopressor support.

Diagnosis

A presumptive diagnosis of LS was made due to a considerable history of rodent exposure. The patient was started on broad-spectrum parenteral antibiotics, vancomycin 750 mg every 24 hours, metronidazole 500 mg every 8 hours, and ceftriaxone 2 g IV daily for adequate coverage against Leptospira spp. Despite 72 hours of antibiotic treatment, the patient’s clinical state deteriorated. He required high dosages of norepinephrine (1.5 mcg/kg/min) and vasopressin (0.03 U/min) to maintain adequate organ perfusion. Despite lung protective settings with low tidal volume and a high positive end-expiratory pressure, there was difficulty maintaining adequate oxygenation. Chest imaging was remarkable for bilateral infiltrates concerning for acute respiratory distress syndrome (ARDS).

The coagulopathy and cholestasis continued to worsen, and the renal failure progressed from nonoliguric to anuric. Because of this progression, the patient was started on continuous renal replacement therapy (CRRT) by hemodialysis. Within 24 hours of initiating CRRT, the patient’s clinical status improved dramatically. Vasopressor support was weaned, the coagulopathy resolved, and the cholestasis was improving. The patient’s respiratory status improved in such a manner that he was extubated by the seventh day after being placed on mechanical ventilation. The urine and blood samples sent for identification of Leptospira spp. through polymerase chain reaction (PCR) returned positive by the ninth day of admission. While on CRRT, the patient’s renal function eventually returned to baseline, and he was discharged 12 days after admission.

Discussion

The spirochetes of the genus Leptospira include both saprophytic and pathogenic species. These pathogenic Leptospira spp. have adapted to a grand variety of zoonotic hosts, the most important being rodents. They serve as vectors for the contraction of the disease by humans. Initial infection in rodents by Leptospira spp. causes a systemic illness followed by a persistent colonization of renal tubules from which they are excreted in the urine and into the environment. Humans, in turn, are an incidental host unable to induce a carrier state for the transmission of the pathogenic organism.¹ The time from exposure to onset of symptoms, or incubation phase, averages 7 to 12 days but may range from 3 to 30 days.⁸

LS has been described as having 2 discernable but often coexisting phases. The first, an acute febrile bacteremic phase, has been noted to last about 9 days in about 85% of patients, although a minority have persistent fever from 2 weeks to > 30 days. A second phase, the immune or inflammatory phase, is characterized by a second fever spike preceded by 1 to 5 afebrile days in which there is presence of IgM antibodies and resolution of leptospiremia but positive urine cultures.⁹ Weil disease may present as the second phase of the disease or as a single, progressive illness from its first manifestation. It is characterized by a triad of jaundice, renal failure, and hemorrhage or coagulopathy.¹⁰ Weil disease is of great concern and importance due to its associated higher mortality than that found with the mildest form of the disease.

There are studies that advocate for RRT as an intricate part of the treatment regimen in LS to remove the inflammatory cytokines produced as a reaction to the spirochete.¹¹ In tropical countries with a higher incidence of the disease, leptospirosis is an important cause of acute kidney injury (AKI), depending on multiple factors, including the AKI definition that is used.¹² Renal invasion by Leptospira spp. produces acute tubular necrosis (ATN) and cell edema during the first week and then could progress to acute interstitial nephritis (AIN) in 2 to 3 weeks. It is believed that the mechanism for the Leptospira spp. invasion of the tubules that results in damage is associated with the antigenic components in its outer membrane; the most important outer membrane protein expressed during infection is LipL32. This protein increases the production of proinflammatory proteins, such as inducible nitric oxide synthase, monocyte chemotactic protein-1 (CCL2/MCP-1), T cells, and tumor necrosis factor.¹³

Although doxycycline has been recommended for the prophylaxis and treatment of mild LS, the preferred agent and the conferred benefits of antibiotic treatment overall for the severe form of the disease has been controversial. Traditionally, penicillin G sodium has been recommended as the first-line antibiotic treatment for moderate-to-severe LS.¹⁴ Nonetheless, there has been an increasing pattern of penicillin resistance among Leptospira spp. that has prompted the study and use of alternative agents.

An open-label, randomized comparison of parenteral cefotaxime, penicillin G sodium, and doxycycline for the treatment of suspected severe leptospirosis conducted by Suputtamongkol and colleagues showed no difference in mortality, defervescence, or time to resolution of abnormal laboratory findings.¹⁵ Current CDC recommendations include the use of parenteral penicillin 1.5 MU every 6 hours as the drug of choice, with ceftriaxone 1 g administered IV every 24 hours equally as effective.³

In addition to antimicrobial therapy, supportive care has shifted to include hemodialysis in those patients who develop AKI as part of the disease. Andrade and colleagues conducted a study of 33 patients with LS in Brazil that was set to compare the impact of door-to-dialysis time and dosage of hemodialysis on mortality. In patients with a quicker door-to-dialysis time and daily hemodialysis sessions, there was a 50% (16.7% vs 66.7%) absolute mortality reduction when compared with those with delayed initiation and alternate-day hemodialysis sessions.¹¹ A follow-up prospective study compared the use of traditional sustained low-efficiency dialysis (SLED) with the use of extended SLED via hemodiafiltration in patients with LS presenting with ARDS and AKI. Although hemodiafiltration resulted in a relative decrease in serum levels of interleukin (IL)-17, IL-7, and CCL2/MCP-1, there was no significant difference in mortality.¹⁶ The most important prognostic factor in severe LS presenting with AKI and relating to RRT is a shorter door-to-dialysis time and increased dose, not the mode of dialysis clearance. Nonetheless, both RRT methods resulted in a progressive decrease in inflammatory mediators that have been associated with ATN and AIN in the context of LS.¹⁶ The authors argue that using CRRT instead of SLED via hemodiafiltration could have accentuated the effects of the reduction that inflammatory mediators may have on mortality in patients with severe LS.

Conclusions

LS continues to be of interest due to its current status as the most common zoonotic disease and its widespread prevalence throughout the globe. Novel treatment modalities for LS, specifically for Weil disease, continue to be developed with the goal of reducing the current mortality rate associated with the disease.

In endemic areas, prompt recognition is essential to initiate the recommended therapy. Parenteral antibiotics, such as penicillin G sodium and ceftriaxone, continue to be the mainstay of treatment and constitute the current CDC recommendations. Nonetheless, early initiation of CRRT has been shown to greatly reduce the mortality associated with Weil disease and, when available, should be considered in these patients.

Our patient failed to improve while receiving parenteral antibiotics alone but showed marked improvement after being placed on CRRT. Furthermore, initiation of CRRT resulted in near-complete resolution of his organ dysfunction and eventual discharge from the hospital. This case serves to further support the use of early CRRT as part of the standard of care in severe LS.

Leptospirosis (LS) is considered the most common and widespread zoonotic disease in the world. Numerous outbreaks have occurred in the past 10 years. Due to its technically difficult diagnosis, LS is severely underrecognized, underdiagnosed, and therefore, underreported.^1,2 The Centers for Disease Control and Prevention (CDC) estimate 100 to 150 cases of LS are identified annually in the US, with about 50% of those cases occurring in Puerto Rico (PR).³ Specifically in PR, about 15 to 100 cases of suspected LS were reported annually between 2000 and 2009, with 59 cases and 1 death reported in 2010. The data are thought to be severely underreported due to a lack of widespread diagnostic testing availability in PR and no formal veterinary and environmental surveillance programs to monitor the incidence of animal cases and actual circulating serovars.⁴

A recent systematic review of 80 studies from 34 countries on morbidity and mortality of LS revealed that the global incidence and mortality is about 1.03 million cases and 58,900 deaths every year. Almost half of the reported deaths were adult males aged 20 to 49 years.⁵ Although mild cases of LS are not associated with an elevated mortality, icteric LS with renal failure (Weil disease) carries a mortality rate of 10%.⁶ In patients who develop hemorrhagic pneumonitis, mortality may be as high as 50 to 70%.⁷ Therefore, it is pivotal that clinicians recognize the disease early, that novel modalities of treatment continue to be developed, and that their impact on patient morbidity and mortality are studied and documented.

Case Presentation

A 43-year-old man with a medical history of schizophrenia presented to the emergency department at the US Department of Veterans Affairs (VA) Caribbean Healthcare System in San Juan, PR, after experiencing 1 week of intermittent fever, myalgia, and general weakness. Emergency medical services had found him disheveled and in a rodent-infested swamp area several days before admission. Initial vital signs were within normal limits.

On physical examination, the patient was afebrile, without acute distress, but he had diffuse jaundice and mild epigastric tenderness without evidence of peritoneal irritation. His complete blood count was remarkable for leukocytosis with left shifting, adequate hemoglobin levels but with 9 × 10³ U/L platelets. The complete metabolic panel demonstrated an aspartate aminotransferase level of 564 U/L, alanine transaminase level of 462 U/L, total bilirubin of 12 mg/dL, which 10.2 mg/dL were direct bilirubin, and an alkaline phosphate of 345 U/L. Lipase levels were measured at 626 U/L. Marked coagulopathy also was present. The toxicology panel, including acetaminophen and salicylate acid levels, did not reveal the presence of any of the tested substances, and chest imaging did not demonstrate any infiltrates.

An abdominal ultrasound was negative for acute cholestatic pathologies, such as cholelithiasis, cholecystitis, or choledocholithiasis. Nonetheless, a noncontrast abdominopelvic computed tomography was remarkable for peripancreatic fat stranding, which raised suspicion for a diagnosis of pancreatitis.

Once the patient was transferred to the intensive care unit, he developed several episodes of hematemesis, leading to hemodynamical instability and severe respiratory distress. Due to anticipated respiratory failure and need for airway securement, endotracheal intubation was performed. Multiple packed red blood cells were transfused, and the patient was started in vasopressor support.

Diagnosis

A presumptive diagnosis of LS was made due to a considerable history of rodent exposure. The patient was started on broad-spectrum parenteral antibiotics, vancomycin 750 mg every 24 hours, metronidazole 500 mg every 8 hours, and ceftriaxone 2 g IV daily for adequate coverage against Leptospira spp. Despite 72 hours of antibiotic treatment, the patient’s clinical state deteriorated. He required high dosages of norepinephrine (1.5 mcg/kg/min) and vasopressin (0.03 U/min) to maintain adequate organ perfusion. Despite lung protective settings with low tidal volume and a high positive end-expiratory pressure, there was difficulty maintaining adequate oxygenation. Chest imaging was remarkable for bilateral infiltrates concerning for acute respiratory distress syndrome (ARDS).

The coagulopathy and cholestasis continued to worsen, and the renal failure progressed from nonoliguric to anuric. Because of this progression, the patient was started on continuous renal replacement therapy (CRRT) by hemodialysis. Within 24 hours of initiating CRRT, the patient’s clinical status improved dramatically. Vasopressor support was weaned, the coagulopathy resolved, and the cholestasis was improving. The patient’s respiratory status improved in such a manner that he was extubated by the seventh day after being placed on mechanical ventilation. The urine and blood samples sent for identification of Leptospira spp. through polymerase chain reaction (PCR) returned positive by the ninth day of admission. While on CRRT, the patient’s renal function eventually returned to baseline, and he was discharged 12 days after admission.

Discussion

The spirochetes of the genus Leptospira include both saprophytic and pathogenic species. These pathogenic Leptospira spp. have adapted to a grand variety of zoonotic hosts, the most important being rodents. They serve as vectors for the contraction of the disease by humans. Initial infection in rodents by Leptospira spp. causes a systemic illness followed by a persistent colonization of renal tubules from which they are excreted in the urine and into the environment. Humans, in turn, are an incidental host unable to induce a carrier state for the transmission of the pathogenic organism.¹ The time from exposure to onset of symptoms, or incubation phase, averages 7 to 12 days but may range from 3 to 30 days.⁸

LS has been described as having 2 discernable but often coexisting phases. The first, an acute febrile bacteremic phase, has been noted to last about 9 days in about 85% of patients, although a minority have persistent fever from 2 weeks to > 30 days. A second phase, the immune or inflammatory phase, is characterized by a second fever spike preceded by 1 to 5 afebrile days in which there is presence of IgM antibodies and resolution of leptospiremia but positive urine cultures.⁹ Weil disease may present as the second phase of the disease or as a single, progressive illness from its first manifestation. It is characterized by a triad of jaundice, renal failure, and hemorrhage or coagulopathy.¹⁰ Weil disease is of great concern and importance due to its associated higher mortality than that found with the mildest form of the disease.

There are studies that advocate for RRT as an intricate part of the treatment regimen in LS to remove the inflammatory cytokines produced as a reaction to the spirochete.¹¹ In tropical countries with a higher incidence of the disease, leptospirosis is an important cause of acute kidney injury (AKI), depending on multiple factors, including the AKI definition that is used.¹² Renal invasion by Leptospira spp. produces acute tubular necrosis (ATN) and cell edema during the first week and then could progress to acute interstitial nephritis (AIN) in 2 to 3 weeks. It is believed that the mechanism for the Leptospira spp. invasion of the tubules that results in damage is associated with the antigenic components in its outer membrane; the most important outer membrane protein expressed during infection is LipL32. This protein increases the production of proinflammatory proteins, such as inducible nitric oxide synthase, monocyte chemotactic protein-1 (CCL2/MCP-1), T cells, and tumor necrosis factor.¹³

Although doxycycline has been recommended for the prophylaxis and treatment of mild LS, the preferred agent and the conferred benefits of antibiotic treatment overall for the severe form of the disease has been controversial. Traditionally, penicillin G sodium has been recommended as the first-line antibiotic treatment for moderate-to-severe LS.¹⁴ Nonetheless, there has been an increasing pattern of penicillin resistance among Leptospira spp. that has prompted the study and use of alternative agents.

An open-label, randomized comparison of parenteral cefotaxime, penicillin G sodium, and doxycycline for the treatment of suspected severe leptospirosis conducted by Suputtamongkol and colleagues showed no difference in mortality, defervescence, or time to resolution of abnormal laboratory findings.¹⁵ Current CDC recommendations include the use of parenteral penicillin 1.5 MU every 6 hours as the drug of choice, with ceftriaxone 1 g administered IV every 24 hours equally as effective.³

In addition to antimicrobial therapy, supportive care has shifted to include hemodialysis in those patients who develop AKI as part of the disease. Andrade and colleagues conducted a study of 33 patients with LS in Brazil that was set to compare the impact of door-to-dialysis time and dosage of hemodialysis on mortality. In patients with a quicker door-to-dialysis time and daily hemodialysis sessions, there was a 50% (16.7% vs 66.7%) absolute mortality reduction when compared with those with delayed initiation and alternate-day hemodialysis sessions.¹¹ A follow-up prospective study compared the use of traditional sustained low-efficiency dialysis (SLED) with the use of extended SLED via hemodiafiltration in patients with LS presenting with ARDS and AKI. Although hemodiafiltration resulted in a relative decrease in serum levels of interleukin (IL)-17, IL-7, and CCL2/MCP-1, there was no significant difference in mortality.¹⁶ The most important prognostic factor in severe LS presenting with AKI and relating to RRT is a shorter door-to-dialysis time and increased dose, not the mode of dialysis clearance. Nonetheless, both RRT methods resulted in a progressive decrease in inflammatory mediators that have been associated with ATN and AIN in the context of LS.¹⁶ The authors argue that using CRRT instead of SLED via hemodiafiltration could have accentuated the effects of the reduction that inflammatory mediators may have on mortality in patients with severe LS.

Conclusions

LS continues to be of interest due to its current status as the most common zoonotic disease and its widespread prevalence throughout the globe. Novel treatment modalities for LS, specifically for Weil disease, continue to be developed with the goal of reducing the current mortality rate associated with the disease.

In endemic areas, prompt recognition is essential to initiate the recommended therapy. Parenteral antibiotics, such as penicillin G sodium and ceftriaxone, continue to be the mainstay of treatment and constitute the current CDC recommendations. Nonetheless, early initiation of CRRT has been shown to greatly reduce the mortality associated with Weil disease and, when available, should be considered in these patients.

Our patient failed to improve while receiving parenteral antibiotics alone but showed marked improvement after being placed on CRRT. Furthermore, initiation of CRRT resulted in near-complete resolution of his organ dysfunction and eventual discharge from the hospital. This case serves to further support the use of early CRRT as part of the standard of care in severe LS.

Leptospirosis (LS) is considered the most common and widespread zoonotic disease in the world. Numerous outbreaks have occurred in the past 10 years. Due to its technically difficult diagnosis, LS is severely underrecognized, underdiagnosed, and therefore, underreported.^1,2 The Centers for Disease Control and Prevention (CDC) estimate 100 to 150 cases of LS are identified annually in the US, with about 50% of those cases occurring in Puerto Rico (PR).³ Specifically in PR, about 15 to 100 cases of suspected LS were reported annually between 2000 and 2009, with 59 cases and 1 death reported in 2010. The data are thought to be severely underreported due to a lack of widespread diagnostic testing availability in PR and no formal veterinary and environmental surveillance programs to monitor the incidence of animal cases and actual circulating serovars.⁴

A recent systematic review of 80 studies from 34 countries on morbidity and mortality of LS revealed that the global incidence and mortality is about 1.03 million cases and 58,900 deaths every year. Almost half of the reported deaths were adult males aged 20 to 49 years.⁵ Although mild cases of LS are not associated with an elevated mortality, icteric LS with renal failure (Weil disease) carries a mortality rate of 10%.⁶ In patients who develop hemorrhagic pneumonitis, mortality may be as high as 50 to 70%.⁷ Therefore, it is pivotal that clinicians recognize the disease early, that novel modalities of treatment continue to be developed, and that their impact on patient morbidity and mortality are studied and documented.

Case Presentation

A 43-year-old man with a medical history of schizophrenia presented to the emergency department at the US Department of Veterans Affairs (VA) Caribbean Healthcare System in San Juan, PR, after experiencing 1 week of intermittent fever, myalgia, and general weakness. Emergency medical services had found him disheveled and in a rodent-infested swamp area several days before admission. Initial vital signs were within normal limits.

On physical examination, the patient was afebrile, without acute distress, but he had diffuse jaundice and mild epigastric tenderness without evidence of peritoneal irritation. His complete blood count was remarkable for leukocytosis with left shifting, adequate hemoglobin levels but with 9 × 10³ U/L platelets. The complete metabolic panel demonstrated an aspartate aminotransferase level of 564 U/L, alanine transaminase level of 462 U/L, total bilirubin of 12 mg/dL, which 10.2 mg/dL were direct bilirubin, and an alkaline phosphate of 345 U/L. Lipase levels were measured at 626 U/L. Marked coagulopathy also was present. The toxicology panel, including acetaminophen and salicylate acid levels, did not reveal the presence of any of the tested substances, and chest imaging did not demonstrate any infiltrates.

An abdominal ultrasound was negative for acute cholestatic pathologies, such as cholelithiasis, cholecystitis, or choledocholithiasis. Nonetheless, a noncontrast abdominopelvic computed tomography was remarkable for peripancreatic fat stranding, which raised suspicion for a diagnosis of pancreatitis.

Once the patient was transferred to the intensive care unit, he developed several episodes of hematemesis, leading to hemodynamical instability and severe respiratory distress. Due to anticipated respiratory failure and need for airway securement, endotracheal intubation was performed. Multiple packed red blood cells were transfused, and the patient was started in vasopressor support.

Diagnosis

A presumptive diagnosis of LS was made due to a considerable history of rodent exposure. The patient was started on broad-spectrum parenteral antibiotics, vancomycin 750 mg every 24 hours, metronidazole 500 mg every 8 hours, and ceftriaxone 2 g IV daily for adequate coverage against Leptospira spp. Despite 72 hours of antibiotic treatment, the patient’s clinical state deteriorated. He required high dosages of norepinephrine (1.5 mcg/kg/min) and vasopressin (0.03 U/min) to maintain adequate organ perfusion. Despite lung protective settings with low tidal volume and a high positive end-expiratory pressure, there was difficulty maintaining adequate oxygenation. Chest imaging was remarkable for bilateral infiltrates concerning for acute respiratory distress syndrome (ARDS).

The coagulopathy and cholestasis continued to worsen, and the renal failure progressed from nonoliguric to anuric. Because of this progression, the patient was started on continuous renal replacement therapy (CRRT) by hemodialysis. Within 24 hours of initiating CRRT, the patient’s clinical status improved dramatically. Vasopressor support was weaned, the coagulopathy resolved, and the cholestasis was improving. The patient’s respiratory status improved in such a manner that he was extubated by the seventh day after being placed on mechanical ventilation. The urine and blood samples sent for identification of Leptospira spp. through polymerase chain reaction (PCR) returned positive by the ninth day of admission. While on CRRT, the patient’s renal function eventually returned to baseline, and he was discharged 12 days after admission.

Discussion

The spirochetes of the genus Leptospira include both saprophytic and pathogenic species. These pathogenic Leptospira spp. have adapted to a grand variety of zoonotic hosts, the most important being rodents. They serve as vectors for the contraction of the disease by humans. Initial infection in rodents by Leptospira spp. causes a systemic illness followed by a persistent colonization of renal tubules from which they are excreted in the urine and into the environment. Humans, in turn, are an incidental host unable to induce a carrier state for the transmission of the pathogenic organism.¹ The time from exposure to onset of symptoms, or incubation phase, averages 7 to 12 days but may range from 3 to 30 days.⁸

LS has been described as having 2 discernable but often coexisting phases. The first, an acute febrile bacteremic phase, has been noted to last about 9 days in about 85% of patients, although a minority have persistent fever from 2 weeks to > 30 days. A second phase, the immune or inflammatory phase, is characterized by a second fever spike preceded by 1 to 5 afebrile days in which there is presence of IgM antibodies and resolution of leptospiremia but positive urine cultures.⁹ Weil disease may present as the second phase of the disease or as a single, progressive illness from its first manifestation. It is characterized by a triad of jaundice, renal failure, and hemorrhage or coagulopathy.¹⁰ Weil disease is of great concern and importance due to its associated higher mortality than that found with the mildest form of the disease.

There are studies that advocate for RRT as an intricate part of the treatment regimen in LS to remove the inflammatory cytokines produced as a reaction to the spirochete.¹¹ In tropical countries with a higher incidence of the disease, leptospirosis is an important cause of acute kidney injury (AKI), depending on multiple factors, including the AKI definition that is used.¹² Renal invasion by Leptospira spp. produces acute tubular necrosis (ATN) and cell edema during the first week and then could progress to acute interstitial nephritis (AIN) in 2 to 3 weeks. It is believed that the mechanism for the Leptospira spp. invasion of the tubules that results in damage is associated with the antigenic components in its outer membrane; the most important outer membrane protein expressed during infection is LipL32. This protein increases the production of proinflammatory proteins, such as inducible nitric oxide synthase, monocyte chemotactic protein-1 (CCL2/MCP-1), T cells, and tumor necrosis factor.¹³

Although doxycycline has been recommended for the prophylaxis and treatment of mild LS, the preferred agent and the conferred benefits of antibiotic treatment overall for the severe form of the disease has been controversial. Traditionally, penicillin G sodium has been recommended as the first-line antibiotic treatment for moderate-to-severe LS.¹⁴ Nonetheless, there has been an increasing pattern of penicillin resistance among Leptospira spp. that has prompted the study and use of alternative agents.

An open-label, randomized comparison of parenteral cefotaxime, penicillin G sodium, and doxycycline for the treatment of suspected severe leptospirosis conducted by Suputtamongkol and colleagues showed no difference in mortality, defervescence, or time to resolution of abnormal laboratory findings.¹⁵ Current CDC recommendations include the use of parenteral penicillin 1.5 MU every 6 hours as the drug of choice, with ceftriaxone 1 g administered IV every 24 hours equally as effective.³

In addition to antimicrobial therapy, supportive care has shifted to include hemodialysis in those patients who develop AKI as part of the disease. Andrade and colleagues conducted a study of 33 patients with LS in Brazil that was set to compare the impact of door-to-dialysis time and dosage of hemodialysis on mortality. In patients with a quicker door-to-dialysis time and daily hemodialysis sessions, there was a 50% (16.7% vs 66.7%) absolute mortality reduction when compared with those with delayed initiation and alternate-day hemodialysis sessions.¹¹ A follow-up prospective study compared the use of traditional sustained low-efficiency dialysis (SLED) with the use of extended SLED via hemodiafiltration in patients with LS presenting with ARDS and AKI. Although hemodiafiltration resulted in a relative decrease in serum levels of interleukin (IL)-17, IL-7, and CCL2/MCP-1, there was no significant difference in mortality.¹⁶ The most important prognostic factor in severe LS presenting with AKI and relating to RRT is a shorter door-to-dialysis time and increased dose, not the mode of dialysis clearance. Nonetheless, both RRT methods resulted in a progressive decrease in inflammatory mediators that have been associated with ATN and AIN in the context of LS.¹⁶ The authors argue that using CRRT instead of SLED via hemodiafiltration could have accentuated the effects of the reduction that inflammatory mediators may have on mortality in patients with severe LS.

Conclusions

LS continues to be of interest due to its current status as the most common zoonotic disease and its widespread prevalence throughout the globe. Novel treatment modalities for LS, specifically for Weil disease, continue to be developed with the goal of reducing the current mortality rate associated with the disease.

In endemic areas, prompt recognition is essential to initiate the recommended therapy. Parenteral antibiotics, such as penicillin G sodium and ceftriaxone, continue to be the mainstay of treatment and constitute the current CDC recommendations. Nonetheless, early initiation of CRRT has been shown to greatly reduce the mortality associated with Weil disease and, when available, should be considered in these patients.

Our patient failed to improve while receiving parenteral antibiotics alone but showed marked improvement after being placed on CRRT. Furthermore, initiation of CRRT resulted in near-complete resolution of his organ dysfunction and eventual discharge from the hospital. This case serves to further support the use of early CRRT as part of the standard of care in severe LS.

From the University of Toronto (Dr. Basuita, Corey L. Kamen, and Dr. Soong) and Sinai Health System (Corey L. Kamen, Cheryl Ethier, and Dr. Soong), Toronto, Ontario, Canada. Co-first authors are Manpreet Basuita, MD, and Corey L. Kamen, BSc.

Abstract

• Objective: Routine laboratory testing is common among medical inpatients; however, when ordered inappropriately testing can represent low-value care. We examined the impact of an evidence-based intervention bundle on utilization.
• Participants/setting: This prospective cohort study took place at a tertiary academic medical center and included 6424 patients admitted to the general internal medicine service between April 2016 and March 2018.
• Intervention: An intervention bundle, whose first components were implemented in July 2016, included computer order entry restrictions on repetitive laboratory testing, education, and audit-feedback.
• Measures: Data were extracted from the hospital electronic health record. The primary outcome was the number of routine blood tests (complete blood count, creatinine, and electrolytes) ordered per inpatient day.
• Analysis: Descriptive statistics were calculated for demographic variables. We used statistical process control charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome.
• Results: The mean number of combined routine laboratory tests ordered per inpatient day decreased from 1.19 (SD, 0.21) tests to 1.11 (SD, 0.05), a relative reduction of 6.7% (P < 0.0001). Mean cost per case related to laboratory tests decreased from $17.24 in the pre-intervention period to $16.17 in the post-intervention period (relative reduction of 6.2%). This resulted in savings of $26,851 in the intervention year.
• Conclusion: A laboratory intervention bundle was associated with small reductions in testing and costs. A routine test performed less than once per inpatient day may not be clinically appropriate or possible.

Keywords: utilization; clinical costs; quality improvement; QI intervention; internal medicine; inpatient.

Routine laboratory blood testing is a commonly used diagnostic tool that physicians rely on to provide patient care. Although routine blood testing represents less than 5% of most hospital budgets, routine use and over-reliance on testing among physicians makes it a target of cost-reduction efforts.^1-3 A variety of interventions have been proposed to reduce inappropriate laboratory tests, with varying results.^1,4-6 Successful interventions include providing physicians with fee data associated with ordered laboratory tests, unbundling panels of tests, and multicomponent interventions.⁶ We conducted a multifaceted quality improvement study to promote and develop interventions to adopt appropriate blood test ordering practices.

Methods

Setting

This prospective cohort study took place at Mount Sinai Hospital, a 443-bed academic hospital affiliated with the University of Toronto, where more than 2400 learners rotate through annually. The study was approved by the Mount Sinai Hospital Research Ethics Board.

Participants

We included all inpatient admissions to the general internal medicine service between April 2016 and March 2018. Exclusion criteria included a length of stay (LOS) longer than 365 days and admission to a critical care unit. Patients with more than 1 admission were counted as separate hospital inpatient visits.

Intervention

Based on internal data, we targeted the top 3 most frequently ordered routine blood tests: complete blood count (CBC), creatinine, and electrolytes. Trainee interviews revealed that habit, bundled order sets, and fear of “missing something” contributed to inappropriate routine blood test ordering. Based on these root causes, we used the Model for Improvement to iteratively develop a multimodal intervention that began in July 2016.^7,8 This included a change to the computerized provider order entry (CPOE) to nudge clinicians to a restrictive ordering strategy by substituting the “Daily x3” frequency of blood test ordering with a “Daily x1” option on a pick list of order options. Clinicians could still order daily routine blood tests for any specified duration, but would have to do so by manually changing the default setting within the CPOE.

From July 2017 to March 2018, the research team educated residents on appropriate laboratory test ordering and provided audit and feedback data to the clinicians. Diagnostic uncertainty was addressed in teaching sessions. Attending physicians were surveyed on appropriate indications for daily laboratory testing for each of CBC, electrolytes, and creatinine. Appropriate indications (Figure 1) were displayed in visible clinical areas and incorporated into teaching sessions.⁹

Clinician teams received real-time performance data on their routine blood test ordering patterns compared with an institutional benchmark. Bar graphs of blood work ordering rates (sum of CBCs, creatinine, and electrolytes ordered for all patients on a given team divided by the total LOS for all patients) were distributed to each internal medicine team via email every 2 weeks (Figure 2).^1,10-12

Data Collection and Analysis

Data were extracted from the hospital electronic health record (EHR). The primary outcome was the number of routine blood tests (CBC, creatinine, and electrolytes) ordered per inpatient day. Descriptive statistics were calculated for demographic variables. We used statistical process control (SPC) charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome. SPC charts display process changes over time. Data are plotted in chronological order, with the central line representing the outcome mean, an upper line representing the upper control limit, and a lower line representing the lower control limit. The upper and lower limits were set at 3δ, which correspond to 3 standard deviations above and below the mean. Six successive points above or beyond the mean suggests “special cause variation,” indicating that observed results are unlikely due to secular trends. SPC charts are commonly used quality tools for process improvement as well as research.^13-16 These charts were created using QI Macros SPC software for Excel V. 2012.07 (KnowWare International, Denver, CO).

The direct cost of each laboratory test was acquired from the hospital laboratory department. The cost of each laboratory test (CBC = $7.54/test, electrolytes = $2.04/test, creatinine = $1.28/test, in Canadian dollars) was subsequently added together and multiplied by the pre- and post-intervention difference of total blood tests saved per inpatient day and then multiplied by 365 to arrive at an estimated cost savings per year.

Results

Over the study period, there were 6424 unique patient admissions on the general internal medicine service, with a median LOS of 3.5 days (Table).

The majority of inpatient visits had at least 1 test of CBC (80%; mean, 3.6 tests/visit), creatinine (79.3%; mean, 3.5 tests/visit), or electrolytes (81.6%; mean, 3.9 tests/visit) completed. In total, 56,767 laboratory tests were ordered.

Following the intervention, there was a reduction in both rates of routine blood test orders and their associated costs, with a shift below the mean. The mean number of tests ordered (combined CBC, creatinine, and electrolytes) per inpatient day decreased from 1.19 (SD, 0.21) in the pre-intervention period to 1.11 (SD, 0.05) in the post-intervention period (P < 0.0001), representing a 6.7% relative reduction (Figure 3). We observed a 6.2% relative reduction in costs per inpatient day, translating to a total savings of $26,851 over 1 year for the intervention period.

Discussion

Our study suggests that a multimodal intervention, including CPOE restrictions, resident education with posters, and audit and feedback strategies, can reduce lab test ordering on general internal medicine wards. This finding is similar to those of previous studies using a similar intervention, although different laboratory tests were targeted.^{1,2,5,6,10,17}

Our study found lower test result reductions than those reported by a previous study, which reported a relative reduction of 17% to 30%,¹⁸ and by another investigation that was conducted recently in a similar setting.¹⁷ In the latter study, reductions in laboratory testing were mostly found in nonroutine tests, and no significant improvements were noted in CBC, electrolytes, and creatine, the 3 tests we studied over the same duration.¹⁷ This may represent a ceiling effect to reducing laboratory testing, and efforts to reduce CBC, electrolytes, and creatinine testing beyond 0.3 to 0.4 tests per inpatient day (or combined 1.16 tests per inpatient day) may not be clinically appropriate or possible. This information can guide institutions to include other areas of overuse based on rates of utilization in order to maximize the benefits from a resource intensive intervention.

There are a number of limitations that merit discussion. First, observational studies do not demonstrate causation; however, to our knowledge, there were no other co-interventions that were being conducted during the study duration. One important note is that our project’s intervention began in July, at which point there are new internal medicine residents beginning their training. As the concept of resource allocation becomes more important, medical schools are spending more time educating students about Choosing Wisely, and, therefore, newer cohorts of residents may be more cognizant of appropriate blood testing. Second, this is a single-center study, limiting generalizability; however, we note that many other centers have reported similar findings. Another limitation is that we do not know whether there were any adverse clinical events associated with blood work ordering that was too restrictive, although informal tracking of STAT laboratory testing remained stable throughout the study period. It is important to ensure that blood work is ordered in moderation and tailored to patients using one’s clinical judgment.

Future Directions

We observed modest reductions in the quantity and costs associated with a quality improvement intervention aimed at reducing routine blood testing. A baseline rate of laboratory testing of less than 1 test per inpatient day may require including other target tests to drive down absolute utilization.

Corresponding author: Christine Soong, MD, MSc, 433-600 University Avenue, Toronto, Ontario, Canada M5G 1X5; [email protected].

Financial disclosures: None.

From the University of Toronto (Dr. Basuita, Corey L. Kamen, and Dr. Soong) and Sinai Health System (Corey L. Kamen, Cheryl Ethier, and Dr. Soong), Toronto, Ontario, Canada. Co-first authors are Manpreet Basuita, MD, and Corey L. Kamen, BSc.

Abstract

• Objective: Routine laboratory testing is common among medical inpatients; however, when ordered inappropriately testing can represent low-value care. We examined the impact of an evidence-based intervention bundle on utilization.
• Participants/setting: This prospective cohort study took place at a tertiary academic medical center and included 6424 patients admitted to the general internal medicine service between April 2016 and March 2018.
• Intervention: An intervention bundle, whose first components were implemented in July 2016, included computer order entry restrictions on repetitive laboratory testing, education, and audit-feedback.
• Measures: Data were extracted from the hospital electronic health record. The primary outcome was the number of routine blood tests (complete blood count, creatinine, and electrolytes) ordered per inpatient day.
• Analysis: Descriptive statistics were calculated for demographic variables. We used statistical process control charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome.
• Results: The mean number of combined routine laboratory tests ordered per inpatient day decreased from 1.19 (SD, 0.21) tests to 1.11 (SD, 0.05), a relative reduction of 6.7% (P < 0.0001). Mean cost per case related to laboratory tests decreased from $17.24 in the pre-intervention period to $16.17 in the post-intervention period (relative reduction of 6.2%). This resulted in savings of $26,851 in the intervention year.
• Conclusion: A laboratory intervention bundle was associated with small reductions in testing and costs. A routine test performed less than once per inpatient day may not be clinically appropriate or possible.

Keywords: utilization; clinical costs; quality improvement; QI intervention; internal medicine; inpatient.

Routine laboratory blood testing is a commonly used diagnostic tool that physicians rely on to provide patient care. Although routine blood testing represents less than 5% of most hospital budgets, routine use and over-reliance on testing among physicians makes it a target of cost-reduction efforts.^1-3 A variety of interventions have been proposed to reduce inappropriate laboratory tests, with varying results.^1,4-6 Successful interventions include providing physicians with fee data associated with ordered laboratory tests, unbundling panels of tests, and multicomponent interventions.⁶ We conducted a multifaceted quality improvement study to promote and develop interventions to adopt appropriate blood test ordering practices.

Methods

Setting

This prospective cohort study took place at Mount Sinai Hospital, a 443-bed academic hospital affiliated with the University of Toronto, where more than 2400 learners rotate through annually. The study was approved by the Mount Sinai Hospital Research Ethics Board.

Participants

We included all inpatient admissions to the general internal medicine service between April 2016 and March 2018. Exclusion criteria included a length of stay (LOS) longer than 365 days and admission to a critical care unit. Patients with more than 1 admission were counted as separate hospital inpatient visits.

Intervention

Based on internal data, we targeted the top 3 most frequently ordered routine blood tests: complete blood count (CBC), creatinine, and electrolytes. Trainee interviews revealed that habit, bundled order sets, and fear of “missing something” contributed to inappropriate routine blood test ordering. Based on these root causes, we used the Model for Improvement to iteratively develop a multimodal intervention that began in July 2016.^7,8 This included a change to the computerized provider order entry (CPOE) to nudge clinicians to a restrictive ordering strategy by substituting the “Daily x3” frequency of blood test ordering with a “Daily x1” option on a pick list of order options. Clinicians could still order daily routine blood tests for any specified duration, but would have to do so by manually changing the default setting within the CPOE.

From July 2017 to March 2018, the research team educated residents on appropriate laboratory test ordering and provided audit and feedback data to the clinicians. Diagnostic uncertainty was addressed in teaching sessions. Attending physicians were surveyed on appropriate indications for daily laboratory testing for each of CBC, electrolytes, and creatinine. Appropriate indications (Figure 1) were displayed in visible clinical areas and incorporated into teaching sessions.⁹

Clinician teams received real-time performance data on their routine blood test ordering patterns compared with an institutional benchmark. Bar graphs of blood work ordering rates (sum of CBCs, creatinine, and electrolytes ordered for all patients on a given team divided by the total LOS for all patients) were distributed to each internal medicine team via email every 2 weeks (Figure 2).^1,10-12

Data Collection and Analysis

Data were extracted from the hospital electronic health record (EHR). The primary outcome was the number of routine blood tests (CBC, creatinine, and electrolytes) ordered per inpatient day. Descriptive statistics were calculated for demographic variables. We used statistical process control (SPC) charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome. SPC charts display process changes over time. Data are plotted in chronological order, with the central line representing the outcome mean, an upper line representing the upper control limit, and a lower line representing the lower control limit. The upper and lower limits were set at 3δ, which correspond to 3 standard deviations above and below the mean. Six successive points above or beyond the mean suggests “special cause variation,” indicating that observed results are unlikely due to secular trends. SPC charts are commonly used quality tools for process improvement as well as research.^13-16 These charts were created using QI Macros SPC software for Excel V. 2012.07 (KnowWare International, Denver, CO).

The direct cost of each laboratory test was acquired from the hospital laboratory department. The cost of each laboratory test (CBC = $7.54/test, electrolytes = $2.04/test, creatinine = $1.28/test, in Canadian dollars) was subsequently added together and multiplied by the pre- and post-intervention difference of total blood tests saved per inpatient day and then multiplied by 365 to arrive at an estimated cost savings per year.

Results

Over the study period, there were 6424 unique patient admissions on the general internal medicine service, with a median LOS of 3.5 days (Table).

The majority of inpatient visits had at least 1 test of CBC (80%; mean, 3.6 tests/visit), creatinine (79.3%; mean, 3.5 tests/visit), or electrolytes (81.6%; mean, 3.9 tests/visit) completed. In total, 56,767 laboratory tests were ordered.

Following the intervention, there was a reduction in both rates of routine blood test orders and their associated costs, with a shift below the mean. The mean number of tests ordered (combined CBC, creatinine, and electrolytes) per inpatient day decreased from 1.19 (SD, 0.21) in the pre-intervention period to 1.11 (SD, 0.05) in the post-intervention period (P < 0.0001), representing a 6.7% relative reduction (Figure 3). We observed a 6.2% relative reduction in costs per inpatient day, translating to a total savings of $26,851 over 1 year for the intervention period.

Discussion

Our study suggests that a multimodal intervention, including CPOE restrictions, resident education with posters, and audit and feedback strategies, can reduce lab test ordering on general internal medicine wards. This finding is similar to those of previous studies using a similar intervention, although different laboratory tests were targeted.^{1,2,5,6,10,17}

Our study found lower test result reductions than those reported by a previous study, which reported a relative reduction of 17% to 30%,¹⁸ and by another investigation that was conducted recently in a similar setting.¹⁷ In the latter study, reductions in laboratory testing were mostly found in nonroutine tests, and no significant improvements were noted in CBC, electrolytes, and creatine, the 3 tests we studied over the same duration.¹⁷ This may represent a ceiling effect to reducing laboratory testing, and efforts to reduce CBC, electrolytes, and creatinine testing beyond 0.3 to 0.4 tests per inpatient day (or combined 1.16 tests per inpatient day) may not be clinically appropriate or possible. This information can guide institutions to include other areas of overuse based on rates of utilization in order to maximize the benefits from a resource intensive intervention.

There are a number of limitations that merit discussion. First, observational studies do not demonstrate causation; however, to our knowledge, there were no other co-interventions that were being conducted during the study duration. One important note is that our project’s intervention began in July, at which point there are new internal medicine residents beginning their training. As the concept of resource allocation becomes more important, medical schools are spending more time educating students about Choosing Wisely, and, therefore, newer cohorts of residents may be more cognizant of appropriate blood testing. Second, this is a single-center study, limiting generalizability; however, we note that many other centers have reported similar findings. Another limitation is that we do not know whether there were any adverse clinical events associated with blood work ordering that was too restrictive, although informal tracking of STAT laboratory testing remained stable throughout the study period. It is important to ensure that blood work is ordered in moderation and tailored to patients using one’s clinical judgment.

Future Directions

We observed modest reductions in the quantity and costs associated with a quality improvement intervention aimed at reducing routine blood testing. A baseline rate of laboratory testing of less than 1 test per inpatient day may require including other target tests to drive down absolute utilization.

Corresponding author: Christine Soong, MD, MSc, 433-600 University Avenue, Toronto, Ontario, Canada M5G 1X5; [email protected].

Financial disclosures: None.

From the University of Toronto (Dr. Basuita, Corey L. Kamen, and Dr. Soong) and Sinai Health System (Corey L. Kamen, Cheryl Ethier, and Dr. Soong), Toronto, Ontario, Canada. Co-first authors are Manpreet Basuita, MD, and Corey L. Kamen, BSc.

Abstract

• Objective: Routine laboratory testing is common among medical inpatients; however, when ordered inappropriately testing can represent low-value care. We examined the impact of an evidence-based intervention bundle on utilization.
• Participants/setting: This prospective cohort study took place at a tertiary academic medical center and included 6424 patients admitted to the general internal medicine service between April 2016 and March 2018.
• Intervention: An intervention bundle, whose first components were implemented in July 2016, included computer order entry restrictions on repetitive laboratory testing, education, and audit-feedback.
• Measures: Data were extracted from the hospital electronic health record. The primary outcome was the number of routine blood tests (complete blood count, creatinine, and electrolytes) ordered per inpatient day.
• Analysis: Descriptive statistics were calculated for demographic variables. We used statistical process control charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome.
• Results: The mean number of combined routine laboratory tests ordered per inpatient day decreased from 1.19 (SD, 0.21) tests to 1.11 (SD, 0.05), a relative reduction of 6.7% (P < 0.0001). Mean cost per case related to laboratory tests decreased from $17.24 in the pre-intervention period to $16.17 in the post-intervention period (relative reduction of 6.2%). This resulted in savings of $26,851 in the intervention year.
• Conclusion: A laboratory intervention bundle was associated with small reductions in testing and costs. A routine test performed less than once per inpatient day may not be clinically appropriate or possible.

Keywords: utilization; clinical costs; quality improvement; QI intervention; internal medicine; inpatient.

Routine laboratory blood testing is a commonly used diagnostic tool that physicians rely on to provide patient care. Although routine blood testing represents less than 5% of most hospital budgets, routine use and over-reliance on testing among physicians makes it a target of cost-reduction efforts.^1-3 A variety of interventions have been proposed to reduce inappropriate laboratory tests, with varying results.^1,4-6 Successful interventions include providing physicians with fee data associated with ordered laboratory tests, unbundling panels of tests, and multicomponent interventions.⁶ We conducted a multifaceted quality improvement study to promote and develop interventions to adopt appropriate blood test ordering practices.

Methods

Setting

This prospective cohort study took place at Mount Sinai Hospital, a 443-bed academic hospital affiliated with the University of Toronto, where more than 2400 learners rotate through annually. The study was approved by the Mount Sinai Hospital Research Ethics Board.

Participants

We included all inpatient admissions to the general internal medicine service between April 2016 and March 2018. Exclusion criteria included a length of stay (LOS) longer than 365 days and admission to a critical care unit. Patients with more than 1 admission were counted as separate hospital inpatient visits.

Intervention

Based on internal data, we targeted the top 3 most frequently ordered routine blood tests: complete blood count (CBC), creatinine, and electrolytes. Trainee interviews revealed that habit, bundled order sets, and fear of “missing something” contributed to inappropriate routine blood test ordering. Based on these root causes, we used the Model for Improvement to iteratively develop a multimodal intervention that began in July 2016.^7,8 This included a change to the computerized provider order entry (CPOE) to nudge clinicians to a restrictive ordering strategy by substituting the “Daily x3” frequency of blood test ordering with a “Daily x1” option on a pick list of order options. Clinicians could still order daily routine blood tests for any specified duration, but would have to do so by manually changing the default setting within the CPOE.

From July 2017 to March 2018, the research team educated residents on appropriate laboratory test ordering and provided audit and feedback data to the clinicians. Diagnostic uncertainty was addressed in teaching sessions. Attending physicians were surveyed on appropriate indications for daily laboratory testing for each of CBC, electrolytes, and creatinine. Appropriate indications (Figure 1) were displayed in visible clinical areas and incorporated into teaching sessions.⁹

Clinician teams received real-time performance data on their routine blood test ordering patterns compared with an institutional benchmark. Bar graphs of blood work ordering rates (sum of CBCs, creatinine, and electrolytes ordered for all patients on a given team divided by the total LOS for all patients) were distributed to each internal medicine team via email every 2 weeks (Figure 2).^1,10-12

Data Collection and Analysis

Data were extracted from the hospital electronic health record (EHR). The primary outcome was the number of routine blood tests (CBC, creatinine, and electrolytes) ordered per inpatient day. Descriptive statistics were calculated for demographic variables. We used statistical process control (SPC) charts to compare the baseline period (April 2016-June 2017) and the intervention period (July 2017-March 2018) for the primary outcome. SPC charts display process changes over time. Data are plotted in chronological order, with the central line representing the outcome mean, an upper line representing the upper control limit, and a lower line representing the lower control limit. The upper and lower limits were set at 3δ, which correspond to 3 standard deviations above and below the mean. Six successive points above or beyond the mean suggests “special cause variation,” indicating that observed results are unlikely due to secular trends. SPC charts are commonly used quality tools for process improvement as well as research.^13-16 These charts were created using QI Macros SPC software for Excel V. 2012.07 (KnowWare International, Denver, CO).

The direct cost of each laboratory test was acquired from the hospital laboratory department. The cost of each laboratory test (CBC = $7.54/test, electrolytes = $2.04/test, creatinine = $1.28/test, in Canadian dollars) was subsequently added together and multiplied by the pre- and post-intervention difference of total blood tests saved per inpatient day and then multiplied by 365 to arrive at an estimated cost savings per year.

Results

Over the study period, there were 6424 unique patient admissions on the general internal medicine service, with a median LOS of 3.5 days (Table).

The majority of inpatient visits had at least 1 test of CBC (80%; mean, 3.6 tests/visit), creatinine (79.3%; mean, 3.5 tests/visit), or electrolytes (81.6%; mean, 3.9 tests/visit) completed. In total, 56,767 laboratory tests were ordered.

Following the intervention, there was a reduction in both rates of routine blood test orders and their associated costs, with a shift below the mean. The mean number of tests ordered (combined CBC, creatinine, and electrolytes) per inpatient day decreased from 1.19 (SD, 0.21) in the pre-intervention period to 1.11 (SD, 0.05) in the post-intervention period (P < 0.0001), representing a 6.7% relative reduction (Figure 3). We observed a 6.2% relative reduction in costs per inpatient day, translating to a total savings of $26,851 over 1 year for the intervention period.

Discussion

Our study suggests that a multimodal intervention, including CPOE restrictions, resident education with posters, and audit and feedback strategies, can reduce lab test ordering on general internal medicine wards. This finding is similar to those of previous studies using a similar intervention, although different laboratory tests were targeted.^{1,2,5,6,10,17}

Our study found lower test result reductions than those reported by a previous study, which reported a relative reduction of 17% to 30%,¹⁸ and by another investigation that was conducted recently in a similar setting.¹⁷ In the latter study, reductions in laboratory testing were mostly found in nonroutine tests, and no significant improvements were noted in CBC, electrolytes, and creatine, the 3 tests we studied over the same duration.¹⁷ This may represent a ceiling effect to reducing laboratory testing, and efforts to reduce CBC, electrolytes, and creatinine testing beyond 0.3 to 0.4 tests per inpatient day (or combined 1.16 tests per inpatient day) may not be clinically appropriate or possible. This information can guide institutions to include other areas of overuse based on rates of utilization in order to maximize the benefits from a resource intensive intervention.

There are a number of limitations that merit discussion. First, observational studies do not demonstrate causation; however, to our knowledge, there were no other co-interventions that were being conducted during the study duration. One important note is that our project’s intervention began in July, at which point there are new internal medicine residents beginning their training. As the concept of resource allocation becomes more important, medical schools are spending more time educating students about Choosing Wisely, and, therefore, newer cohorts of residents may be more cognizant of appropriate blood testing. Second, this is a single-center study, limiting generalizability; however, we note that many other centers have reported similar findings. Another limitation is that we do not know whether there were any adverse clinical events associated with blood work ordering that was too restrictive, although informal tracking of STAT laboratory testing remained stable throughout the study period. It is important to ensure that blood work is ordered in moderation and tailored to patients using one’s clinical judgment.

Future Directions

We observed modest reductions in the quantity and costs associated with a quality improvement intervention aimed at reducing routine blood testing. A baseline rate of laboratory testing of less than 1 test per inpatient day may require including other target tests to drive down absolute utilization.

Corresponding author: Christine Soong, MD, MSc, 433-600 University Avenue, Toronto, Ontario, Canada M5G 1X5; [email protected].

Financial disclosures: None.

From the Senior Health Program, Providence Health & Services, Oregon, Portland, OR.

Abstract

Background: Approximately 51 million adults in the United States are 65 years of age or older, yet few geriatric-trained primary care providers (PCP) serve this population. The Age-Friendly Health System framework, consisting of evidence-based 4M care (Mobility, Medication, Mentation, and what Matters), encourages all PCPs to assess mobility in older adults.

Objective: To improve PCP knowledge, confidence, and clinical practice in assessing and managing fall risk.

Methods: A 1-week educational session focusing on mobility (part of a 4-week Geriatric Mini-Fellowship) for 6 selected PCPs from a large health care system was conducted to increase knowledge and ability to address fall risk in older adults. The week included learning and practicing a Fall Risk Management Plan (FRMP) algorithm, including planning for their own practice changes. Pre- and post-test surveys assessed changes in knowledge and confidence. Patient data were compared 12 months before and after training to evaluate PCP adoption of FRMP components.

Results: The training increased provider knowledge and confidence. The trained PCPs were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to assess orthostatic blood pressure in their 65+ patients after the mini-fellowship. In high-risk patients, they were 4.1 times more likely to discuss fall risk and 6.3 times more likely to assess orthostatic blood pressure than their nontrained peers. Changes in physical therapy referral rates were not observed.

Conclusions: In-depth, skills-based geriatric educational sessions improved PCPs’ knowledge and confidence and also improved their fall risk management practices for their older patients.

Keywords: geriatrics; guidelines; Age-Friendly Health System; 4M; workforce training; practice change; fellowship.

The US population is aging rapidly. People aged 85 years and older are the largest-growing segment of the US population, and this segment is expected to increase by 123% by 2040.¹ Caregiving needs increase with age as older adults develop more chronic conditions, such as hypertension, heart disease, arthritis, and dementia. However, even with increasing morbidity and dependence, a majority of older adults still live in the community rather than in institutional settings.² These older adults seek medical care more frequently than younger people, with about 22% of patients 75 years and older having 10 or more health care visits in the previous 12 months. By 2040, nearly a quarter of the US population is expected to be 65 or older, with many of these older adults seeking regular primary care from providers who do not have formal training in the care of a population with multiple complex, chronic health conditions and increased caregiving needs.¹

Despite this growing demand for health care professionals trained in the care of older adults, access to these types of clinicians is limited. In 2018, there were roughly 7000 certified geriatricians, with only 3600 of them practicing full-time.^3,4 Similarly, of 290,000 certified nurse practitioners (NPs), about 9% of them have geriatric certification.⁵ Geriatricians, medical doctors trained in the care of older adults, and geriatric-trained NPs are part of a cadre of a geriatric-trained workforce that provides unique expertise in caring for older adults with chronic and advanced illness. They know how to manage multiple, complex geriatric syndromes like falls, dementia, and polypharmacy; understand and maximize team-based care; and focus on caring for an older person with a goal-centered versus a disease-centered approach.⁶

Broadly, geriatric care includes a spectrum of adults, from those who are aging healthfully to those who are the frailest. Research has suggested that approximately 30% of older adults need care by a geriatric-trained clinician, with the oldest and frailest patients needing more clinician time for assessment and treatment, care coordination, and coaching of caregivers.⁷ With this assumption in mind, it is projected that by 2025, there will be a national shortage of 26,980 geriatricians, with the western United States disproportionately affected by this shortage.⁴Rather than lamenting this shortage, Tinetti recommends a new path forward: “Our mission should not be to train enough geriatricians to provide direct care, but rather to ensure that every clinician caring for older adults is competent in geriatric principles and practices.”⁸ Sometimes called ”geriatricizing,” the idea is to use existing geriatric providers as a small elite training force to infuse geriatric principles and skills across their colleagues in primary care and other disciplines.^8,9 Efforts of the American Geriatrics Society (AGS), with support from the John A. Hartford Foundation (JAHF), have been successful in developing geriatric training across multiple specialties, including surgery, orthopedics, and emergency medicine (www.americangeriatrics.org/programs/geriatrics-specialists-initiative).

The Age-Friendly Health System and 4M Model

To help augment this idea of equipping health care systems and their clinicians with more readily available geriatric knowledge, skills, and tools, the JAHF, along with the Institute for Healthcare Improvement (IHI), created the Age-Friendly Health System (AFHS) paradigm in 2015.¹⁰ Using the 4M model, the AFHS initiative established a set of evidence-based geriatric priorities and interventions meant to improve the care of older adults, reduce harm and duplication, and provide a framework for engaging leadership, clinical teams, and operational systems across inpatient and ambulatory settings.¹¹ Mobility, including fall risk screening and intervention, is 1 of the 4M foundational elements of the Age-Friendly model. In addition to Mobility, the 4M model also includes 3 other key geriatric domains: Mentation (dementia, depression, and delirium), Medication (high-risk medications, polypharmacy, and deprescribing), and What Matters (goals of care conversations and understanding quality of life for older patients).¹¹ The 4M initiative encourages adoption of a geriatric lens that looks across chronic conditions and accounts for the interplay among geriatric syndromes, such as falls, cognitive impairment, and frailty, in order to provide care better tailored to what the patient needs and desires.¹² IHI and JAHF have targeted the adoption of the 4M model by 20% of US health care systems by 2020.¹¹

Mini-Fellowship and Mobility Week

To bolster geriatric skills among community-based primary care providers (PCPs), we initiated a Geriatric Mini-Fellowship, a 4-week condensed curriculum taught over 6 months. Each week focuses on 1 of the age-friendly 4Ms, with the goal of increasing the knowledge, self-efficacy, skills, and competencies of the participating PCPs (called “fellow” hereafter) and at the same time, equipping each to become a champion of geriatric practice. This article focuses on the Mobility week, the second week of the mini-fellowship, and the effect of the week on the fellows’ practice changes.

To construct the Mobility week’s curriculum with a focus on the ambulatory setting, we relied upon national evidence-based work in fall risk management. The Centers for Disease Control and Prevention (CDC) has made fall risk screening and management in primary care a high priority. Using the clinical practice guidelines for managing fall risk developed by the American and British Geriatrics Societies (AGS/BGS), the CDC developed the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) toolkit.¹³ Foundational to the toolkit is the validated 12-item Stay Independent falls screening questionnaire (STEADI questionnaire).¹⁴ Patients who score 4 or higher (out of a total score of 14) on the questionnaire are considered at increased risk of falling. The CDC has developed a clinical algorithm that guides clinical teams through screening and assessment to help identify appropriate interventions to target specific risk factors. Research has clearly established that a multifactorial approach to fall risk intervention can be successful in reducing fall risk by as much as 25%.^15-17

The significant morbidity and mortality caused by falls make training nongeriatrician clinicians on how to better address fall risk imperative. More than 25% of older adults fall each year.¹⁸ These falls contribute to rising rates of fall-related deaths,¹⁹ emergency department (ED) visits,²⁰ and hospital readmissions.²¹ Initiatives like the AFHS focus on mobility and the CDC’s development of supporting clinical materials²² aim to improve primary care adoption of fall risk screening and intervention practices.^23,24 The epidemic of falls must compel all PCPs, not just those practicing geriatrics, to make discussing and addressing fall risk and falls a priority.

Methods

Setting

This project took place as part of a regional primary care effort in Oregon. Providence Health & Services-Oregon is part of a multi-state integrated health care system in the western United States whose PCPs serve more than 80,000 patients aged 65 years and older per year; these patients comprise 38% of the system’s office visits each year. Regionally, there are 47 family and internal medicine clinics employing roughly 290 providers (physicians, NPs, and physician assistants). The organization has only 4 PCPs trained in geriatrics and does not offer any geriatric clinical consultation services. Six PCPs from different clinics, representing both rural and urban settings, are chosen to participate in the geriatric mini-fellowship each year.

This project was conducted as a quality improvement initiative within the organization and did not constitute human subjects research. It was not conducted under the oversight of the Institutional Review Board.

Intervention

The mini-fellowship was taught in 4 1-week blocks between April and October 2018, with a curriculum designed to be interactive and practical. The faculty was intentionally interdisciplinary to teach and model team-based practice. Each week participants were excused from their clinical practice. Approximately 160 hours of continuing medical education credits were awarded for the full mini-fellowship. As part of each weekly session, a performance improvement project (PIP) focused on that week’s topic (1 of the 4Ms) was developed by the fellow and their team members to incorporate the mini-fellowship learnings into their clinic workflows. Fellows also had 2 hours per week of dedicated administration time for a year, outside the fellowship, to work on their PIP and 4M practice changes within their clinic.

Provider Education

The week for mobility training comprised 4 daylong sessions. The first 2 days were spent learning about the epidemiology of falls; risk factors for falling; how to conduct a thorough history and assessment of fall risk; and how to create a prioritized Fall Risk Management Plan (FRMP) to decrease a patient’s individual fall risk through tailored interventions. The FRMP was adapted from the CDC STEADI toolkit.¹³ Core faculty were 2 geriatric-trained providers (NP and physician) and a physical therapist (PT) specializing in fall prevention.

On the third day, fellows took part in a simulated fall risk clinic, in which older adults volunteered to be patient partners, providing an opportunity to apply learnings from days 1 and 2. The clinic included the fellow observing a PT complete a mobility assessment and a pharmacist conduct a high-risk medication review. The fellow synthesized the findings of the mobility assessment and medication review, as well as their own history and assessment, to create a summary of fall risk recommendations to discuss with their volunteer patient partner. The fellows were observed and evaluated in their skills by their patient partner, course faculty, and another fellow. The patient partners, and their assigned fellow, also participated in a 45-minute fall risk presentation, led by a nurse.

On the fourth day, the fellows were joined by select clinic partners, including nurses, pharmacists, and/or medical assistants. The session included discussions among each fellow’s clinical team regarding the current state of fall risk efforts at their clinic, an analysis of barriers, and identification of opportunities to improve workflows and screening rates. Each fellow took with them an action plan tailored to their clinic to improve fall risk management practices, starting with the fellow’s own practice.

Fall Risk Management Plan

The educational sessions introduced the fellows to the FRMP. The FRMP, adapted from the STEADI toolkit, includes a process for fall risk screening (Figure 1) and stratifying a patient’s risk based on their STEADI score in order to promote 3 priority assessments (gait evaluation with PT referral if appropriate; orthostatic blood pressure; and high-risk medication review; Figure 2). Initial actions based on these priority assessments were followed over time, with additional fall risk interventions added as clinically indicated.²⁵ The FRMP is intended to be used during routine office visits, Medicare annual wellness visits, or office visits focused on fall risk or related medical disorders (ie, fall risk visits.)

Providers and their teams were encouraged to spread out fall-related conversations with their patients over multiple visits, since many patients have multiple fall risk factors at play, in addition to other chronic medical issues, and since many interventions often require behavior changes on the part of the patient. Providers also had access to fall-related electronic health record (EHR) templates as well as a comprehensive, internal fall risk management website that included assessment tools, evidence-based resources, and patient handouts.

Assessment and Measurements

We assessed provider knowledge and comfort in their fall risk evaluation and management skills before and after the educational intervention using an 11-item multiple-choice questionnaire and a 4-item confidence questionnaire. The confidence questions used a 7-point Likert scale, with 0 indicating “no confidence” and 7 indicating ”lots of confidence.” The questions were administered via a paper survey. Qualitative comments were derived from evaluations completed at the end of the week.

The fellows’ practice of fall risk screening and management was studied from May 2018, at the completion of Mobility week, to May 2019 for the post-intervention period. A 1-year timeframe before May 2018 was used as the pre-intervention period. Eligible visit types, during which we assumed fall risk was discussed, were any office visits for patients 65+ completed by the patients’ PCPs that used fall risk as a reason for the visit or had a fall-related diagnosis code. Fall risk visits performed by other clinic providers were not counted.

Of those patients who had fall risk screenings completed and were determined to be high risk (STEADI score ≥ 4), data were analyzed to determine whether these patients had any fall-related follow-up visits to their PCP within 60 days of the STEADI screening. For these high-risk patients, data were studied to understand whether orthostatic blood pressure measurements were performed (as documented in a flowsheet) and whether a PT referral was placed. These data were compared with those from providers who practiced in clinics within the same system but who did not participate in the mini-fellowship. Data were obtained from the organization’s EHR. Additional data were measured to evaluate patterns of deprescribing of select high-risk medications, but these data are not included in this analysis.

Analysis

A paired-samples t test was used to measure changes in provider confidence levels. Data were aggregated across fellows, resulting in a mean. A chi-square test of independence was performed to examine the relationship between rates of FRMP adoption by select provider groups. Analysis included a pre- and post-intervention assessment of the fellows’ adoption of FRMP practices, as well as a comparison between the fellows’ practice patterns and those of a control group of PCPs in the organization’s other clinics who did not participate in the mini-fellowship (nontrained control group). Excluded from the control group were providers from the same clinic as the fellows; providers in clinics with a geriatric-trained provider on staff; and clinics outside of the Portland metro and Medford service areas. We used an alpha level of 0.05 for all statistical tests.

Data from 5 providers were included in the analysis of the FRMP adoption. The sixth provider changed practice settings from the clinic to the ED after completing the fellowship; her patient data were not included in the FRMP part of the analysis. EHR data included data on all visits of patients 65+, as well as data for just those 65+ patients who had been identified as being at high risk to fall based on a STEADI score of 4 or higher.

Results

Provider Questionnaire

All 6 providers responded to the pre-intervention and post-intervention tests. For the knowledge questions, fellows, as a composite, correctly answered 57% of the questions before the intervention and 79% after the intervention. Provider confidence level in delivering fall risk care was measured prior to the training (mean, 4.12 [SD, 0.62]) and at the end of the training (mean, 6.47 [SD, 0.45]), demonstrating a significant increase in confidence (t (5) = –10.46, P < 0.001).

Qualitative Comments

Providers also had the opportunity to provide comments on their experience during the Mobility week and at the end of 1 year. In general, the simulated interdisciplinary fall risk clinic was highly rated (“the highlight of the week”) as a practical strategy to embed learning principles. One fellow commented, “Putting the learning into practice helps solidify it in my brain.” Fellows also appreciated the opportunity to learn and meet with their clinic colleagues to begin work on a fall-risk focused PIP and to “have a framework for what to do for people who screen positive [for fall risk].”

FRMP Adoption

A comparison of the care the fellows provided to their patients 65+ in the 12 months pre- and post-training shows the fellows demonstrated significant changes in practice patterns. The fellows were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to check orthostatic blood pressure than prior to the mini-fellowship (Table 1).

The control providers also demonstrated significant increases in fall risk screening and discussion of fall risk between the pre- and post-intervention periods; however, the relative risk (RR) was between 1.10 and 1.13 for this group. For the control group, checking orthostatic blood pressure did not significantly change. In the 12 months after training (Table 2), the fellows were 4.2 times more likely to discuss fall risk and almost 5 times more likely to check orthostatic blood pressure than their nontrained peers for all of their patients 65+, regardless of their risk to fall.

As shown in Table 3, for those patients determined to be at high risk of falling (STEADI score ≥ 4), fellows showed statistically significant increases in fall risk visits (RR, 3.02) and assessment of orthostatic blood pressure (RR, 10.68) before and after the mini-fellowship. The control providers did not show any changes in practice patterns between the pre- and post-period among patients at high risk to fall.

Neither the fellows nor the control group showed changes in patterns of referral to PT. In comparing the 2 groups in the 12 months after training (Table 4), for their patients at risk of falling, the fellows were 4 times more likely to complete fall risk visits and over 6 times more likely to assess orthostatic blood pressure than their nontrained peers. Subgroup analysis of the 75+ population revealed similar trends and significance, but these results are not included here.

Discussion

This study aimed to improve not only providers’ knowledge and confidence in caring for older adults at increased risk to fall, but also their clinical practice in assessing and managing fall risk. In addition to improved knowledge and confidence, we found that the fellows increased their discussion of fall risk (through fall risk visits) and their assessment of orthostatic blood pressure for all of their patients, not just for those identified at increased risk to fall. This improvement held true for the fellows themselves before and after the intervention, but also as compared to their nontrained peers. These practice improvements for all of their 65+ patients, not just those identified as being at high risk to fall, are especially important, since studies indicate that early screening and intervention can help identify people at risk and prevent future falls.¹⁵

We were surprised that there were no significant differences in PT referrals made by the trained fellows, but this finding may have been confounded by the fact that the data included all PT referrals, regardless of diagnosis, not just those referrals that were fall-related. Furthermore, our baseline PT referral rates, at 39% for the intervention group and 42% for the control group, are higher than national data when looking at rehabilitation use by older adults.²⁶

In comparison to a study evaluating the occurrence of fall risk–related clinical practice in primary care before any fall-related educational intervention, orthostatics were checked less frequently in our study (10% versus 30%) and there were fewer PT referrals (42%–44% versus 53%).²⁷ However, the Phelan study took place in patients who had actually had a fall, rather than just having a higher risk for a fall, and was based on detailed chart review. Other studies^23,24 found higher rates of fall risk interventions, but did not break out PT referrals specifically.

In terms of the educational intervention itself, most studies of geriatric education interventions have measured changes in knowledge, confidence, or self-efficacy as they relate to geriatric competence,^28-30 and do not measure practice change as an outcome outside of intent to change or self-reported practice change.^31,32 In general, practice change or longer-term health care–related outcomes have not been studied. Additionally, a range of dosages of educational interventions has been studied, from 1-hour lunchtime presentations^23,32 to half-day²⁹ or several half-day workshops,²⁸ up to 160 hours over 10 months³⁰ or 5 weekends over 6 months.³¹ The duration of our entire intervention at 160 hours over 6 months would be considered on the upper end of dosing relative to these studies, with our Mobility week intervention comprising 32 hours during 1 week. In the Warshaw study, despite 107 1-hour sessions being taught to over 60 physicians in 16 practices over 4 years, only 2 practices ultimately initiated any practice change projects.³² We believe that only curricula that embed practice change skills and opportunities, at a significant enough dose, can actually impact practice change in a sustainable manner.

Knowledge and skill acquisition among individual providers does not take place to a sufficient degree in the current health care arena, which is focused on productivity and short visit times. Consistent with other studies, we included interdisciplinary members of the primary care team for part of the mini-fellowship, although other studies used models that train across disciplines for the entirety of the learning experience.^28-30,33 Our educational model was strengthened by including other professionals to provide some of the education and model the ideal geriatric team, including PT, occupational therapy, and pharmacy, for the week on mobility.

Most studies exploring interventions through geriatric educational initiatives are conducted within academic institutions, with a primary focus on physician faculty and, by extension, their teaching of residents and others.^34,35 We believe our integrated model, which is steeped in community-based primary care practices like Lam’s,³¹ offers the greatest outreach to large community-based care systems and their patients. Training providers to work with their teams to change their own practices first gives skills and expertise that help further establish them as geriatric champions within their practices, laying the groundwork for more widespread practice change at their clinics.

Limitations

In addition to the limitations described above relating to the capture of PT referrals, other limitations included the relatively short time period for follow-up data as well as the small size of the intervention group. However, we found value in the instructional depth that the small group size allowed.

While the nontrained providers did show some improvement during the same period, we believe the relative risk was not clinically significant. We suspect that the larger health system efforts to standardize screening of patients 65+ across all clinics as a core quality metric confounded these results. The data analysis also included only fall-related patient visits that occurred with a provider who was that patient’s PCP, which could have missed visits done by other PCP colleagues, RNs, or pharmacists in the same clinic, thus undercounting the true number of fall-related visits. Furthermore, counting of fall-related interventions relied upon providers documenting consistently in the EHR, which could also lead to under-represention of fall risk clinical efforts.

The data presented, while encouraging, do not reflect clinic-wide practice change patterns and are considered only proximate outcomes rather than more long-term or cost-related outcomes, as would be captured by fall-related utilization measures like emergency room visits and hospitalizations. We expect to evaluate the broader impact and these value-based outcomes in the future. All providers and teams were from the same health care system, which may not allow our results to transfer to other organizations or regions of clinical practice.

Summary

This study demonstrates that an intensive mini-fellowship model of geriatrics training improved both knowledge and confidence in the realm of fall risk assessment and intervention among PCPs who had not been formally trained in geriatrics. More importantly, the training improved the fall-related care of their patients at increased risk to fall, but also of all of their older patients, with improvements in care measured up to a year after the mini-fellowship. Although this article only describes the work done as part of the Mobility aim of the 4M AFHS model, we believe the entire mini-fellowship curriculum offers the opportunity to “geriatricize” clinicians and their teams in learning geriatric principles and skills that they can translate into their practice in a sustainable way, as Tinetti encourages.⁸ Future study to evaluate other process outcomes more precisely, such as PT, as well as cost- and value-based outcomes, and the influence of trained providers on their clinic partners, will further establish the value proposition of targeted, disseminated, intensive geriatrics training of primary care clinicians as a strategy of age-friendly health systems as they work to improve the care of their older adults.

Acknowledgment: We are grateful for the dedication and hard work of the 2018 Geriatric Mini-Fellowship fellows at Providence Health & Services-Oregon who made this article possible. Thanks to Drs. Stephanie Cha, Emily Puukka-Clark, Laurie Dutkiewicz, Cara Ellis, Deb Frost, Jordan Roth, and Subhechchha Shah for promoting the AFHS work within their Providence Medical Group clinics and to PMG leadership and the fellows’ clinical teams for supporting the fellows, the AFHS work, and their older patients.

Corresponding author: Colleen M. Casey, PhD, ANP-BC, Providence Health & Services, Senior Health Program, 4400 NE Halsey, 5th Floor, Portland, OR 97213; [email protected].

Financial disclosures: None.

From the Senior Health Program, Providence Health & Services, Oregon, Portland, OR.

Abstract

Background: Approximately 51 million adults in the United States are 65 years of age or older, yet few geriatric-trained primary care providers (PCP) serve this population. The Age-Friendly Health System framework, consisting of evidence-based 4M care (Mobility, Medication, Mentation, and what Matters), encourages all PCPs to assess mobility in older adults.

Objective: To improve PCP knowledge, confidence, and clinical practice in assessing and managing fall risk.

Methods: A 1-week educational session focusing on mobility (part of a 4-week Geriatric Mini-Fellowship) for 6 selected PCPs from a large health care system was conducted to increase knowledge and ability to address fall risk in older adults. The week included learning and practicing a Fall Risk Management Plan (FRMP) algorithm, including planning for their own practice changes. Pre- and post-test surveys assessed changes in knowledge and confidence. Patient data were compared 12 months before and after training to evaluate PCP adoption of FRMP components.

Results: The training increased provider knowledge and confidence. The trained PCPs were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to assess orthostatic blood pressure in their 65+ patients after the mini-fellowship. In high-risk patients, they were 4.1 times more likely to discuss fall risk and 6.3 times more likely to assess orthostatic blood pressure than their nontrained peers. Changes in physical therapy referral rates were not observed.

Conclusions: In-depth, skills-based geriatric educational sessions improved PCPs’ knowledge and confidence and also improved their fall risk management practices for their older patients.

Keywords: geriatrics; guidelines; Age-Friendly Health System; 4M; workforce training; practice change; fellowship.

The US population is aging rapidly. People aged 85 years and older are the largest-growing segment of the US population, and this segment is expected to increase by 123% by 2040.¹ Caregiving needs increase with age as older adults develop more chronic conditions, such as hypertension, heart disease, arthritis, and dementia. However, even with increasing morbidity and dependence, a majority of older adults still live in the community rather than in institutional settings.² These older adults seek medical care more frequently than younger people, with about 22% of patients 75 years and older having 10 or more health care visits in the previous 12 months. By 2040, nearly a quarter of the US population is expected to be 65 or older, with many of these older adults seeking regular primary care from providers who do not have formal training in the care of a population with multiple complex, chronic health conditions and increased caregiving needs.¹

Despite this growing demand for health care professionals trained in the care of older adults, access to these types of clinicians is limited. In 2018, there were roughly 7000 certified geriatricians, with only 3600 of them practicing full-time.^3,4 Similarly, of 290,000 certified nurse practitioners (NPs), about 9% of them have geriatric certification.⁵ Geriatricians, medical doctors trained in the care of older adults, and geriatric-trained NPs are part of a cadre of a geriatric-trained workforce that provides unique expertise in caring for older adults with chronic and advanced illness. They know how to manage multiple, complex geriatric syndromes like falls, dementia, and polypharmacy; understand and maximize team-based care; and focus on caring for an older person with a goal-centered versus a disease-centered approach.⁶

Broadly, geriatric care includes a spectrum of adults, from those who are aging healthfully to those who are the frailest. Research has suggested that approximately 30% of older adults need care by a geriatric-trained clinician, with the oldest and frailest patients needing more clinician time for assessment and treatment, care coordination, and coaching of caregivers.⁷ With this assumption in mind, it is projected that by 2025, there will be a national shortage of 26,980 geriatricians, with the western United States disproportionately affected by this shortage.⁴Rather than lamenting this shortage, Tinetti recommends a new path forward: “Our mission should not be to train enough geriatricians to provide direct care, but rather to ensure that every clinician caring for older adults is competent in geriatric principles and practices.”⁸ Sometimes called ”geriatricizing,” the idea is to use existing geriatric providers as a small elite training force to infuse geriatric principles and skills across their colleagues in primary care and other disciplines.^8,9 Efforts of the American Geriatrics Society (AGS), with support from the John A. Hartford Foundation (JAHF), have been successful in developing geriatric training across multiple specialties, including surgery, orthopedics, and emergency medicine (www.americangeriatrics.org/programs/geriatrics-specialists-initiative).

The Age-Friendly Health System and 4M Model

To help augment this idea of equipping health care systems and their clinicians with more readily available geriatric knowledge, skills, and tools, the JAHF, along with the Institute for Healthcare Improvement (IHI), created the Age-Friendly Health System (AFHS) paradigm in 2015.¹⁰ Using the 4M model, the AFHS initiative established a set of evidence-based geriatric priorities and interventions meant to improve the care of older adults, reduce harm and duplication, and provide a framework for engaging leadership, clinical teams, and operational systems across inpatient and ambulatory settings.¹¹ Mobility, including fall risk screening and intervention, is 1 of the 4M foundational elements of the Age-Friendly model. In addition to Mobility, the 4M model also includes 3 other key geriatric domains: Mentation (dementia, depression, and delirium), Medication (high-risk medications, polypharmacy, and deprescribing), and What Matters (goals of care conversations and understanding quality of life for older patients).¹¹ The 4M initiative encourages adoption of a geriatric lens that looks across chronic conditions and accounts for the interplay among geriatric syndromes, such as falls, cognitive impairment, and frailty, in order to provide care better tailored to what the patient needs and desires.¹² IHI and JAHF have targeted the adoption of the 4M model by 20% of US health care systems by 2020.¹¹

Mini-Fellowship and Mobility Week

To bolster geriatric skills among community-based primary care providers (PCPs), we initiated a Geriatric Mini-Fellowship, a 4-week condensed curriculum taught over 6 months. Each week focuses on 1 of the age-friendly 4Ms, with the goal of increasing the knowledge, self-efficacy, skills, and competencies of the participating PCPs (called “fellow” hereafter) and at the same time, equipping each to become a champion of geriatric practice. This article focuses on the Mobility week, the second week of the mini-fellowship, and the effect of the week on the fellows’ practice changes.

To construct the Mobility week’s curriculum with a focus on the ambulatory setting, we relied upon national evidence-based work in fall risk management. The Centers for Disease Control and Prevention (CDC) has made fall risk screening and management in primary care a high priority. Using the clinical practice guidelines for managing fall risk developed by the American and British Geriatrics Societies (AGS/BGS), the CDC developed the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) toolkit.¹³ Foundational to the toolkit is the validated 12-item Stay Independent falls screening questionnaire (STEADI questionnaire).¹⁴ Patients who score 4 or higher (out of a total score of 14) on the questionnaire are considered at increased risk of falling. The CDC has developed a clinical algorithm that guides clinical teams through screening and assessment to help identify appropriate interventions to target specific risk factors. Research has clearly established that a multifactorial approach to fall risk intervention can be successful in reducing fall risk by as much as 25%.^15-17

The significant morbidity and mortality caused by falls make training nongeriatrician clinicians on how to better address fall risk imperative. More than 25% of older adults fall each year.¹⁸ These falls contribute to rising rates of fall-related deaths,¹⁹ emergency department (ED) visits,²⁰ and hospital readmissions.²¹ Initiatives like the AFHS focus on mobility and the CDC’s development of supporting clinical materials²² aim to improve primary care adoption of fall risk screening and intervention practices.^23,24 The epidemic of falls must compel all PCPs, not just those practicing geriatrics, to make discussing and addressing fall risk and falls a priority.

Methods

Setting

This project took place as part of a regional primary care effort in Oregon. Providence Health & Services-Oregon is part of a multi-state integrated health care system in the western United States whose PCPs serve more than 80,000 patients aged 65 years and older per year; these patients comprise 38% of the system’s office visits each year. Regionally, there are 47 family and internal medicine clinics employing roughly 290 providers (physicians, NPs, and physician assistants). The organization has only 4 PCPs trained in geriatrics and does not offer any geriatric clinical consultation services. Six PCPs from different clinics, representing both rural and urban settings, are chosen to participate in the geriatric mini-fellowship each year.

This project was conducted as a quality improvement initiative within the organization and did not constitute human subjects research. It was not conducted under the oversight of the Institutional Review Board.

Intervention

The mini-fellowship was taught in 4 1-week blocks between April and October 2018, with a curriculum designed to be interactive and practical. The faculty was intentionally interdisciplinary to teach and model team-based practice. Each week participants were excused from their clinical practice. Approximately 160 hours of continuing medical education credits were awarded for the full mini-fellowship. As part of each weekly session, a performance improvement project (PIP) focused on that week’s topic (1 of the 4Ms) was developed by the fellow and their team members to incorporate the mini-fellowship learnings into their clinic workflows. Fellows also had 2 hours per week of dedicated administration time for a year, outside the fellowship, to work on their PIP and 4M practice changes within their clinic.

Provider Education

The week for mobility training comprised 4 daylong sessions. The first 2 days were spent learning about the epidemiology of falls; risk factors for falling; how to conduct a thorough history and assessment of fall risk; and how to create a prioritized Fall Risk Management Plan (FRMP) to decrease a patient’s individual fall risk through tailored interventions. The FRMP was adapted from the CDC STEADI toolkit.¹³ Core faculty were 2 geriatric-trained providers (NP and physician) and a physical therapist (PT) specializing in fall prevention.

On the third day, fellows took part in a simulated fall risk clinic, in which older adults volunteered to be patient partners, providing an opportunity to apply learnings from days 1 and 2. The clinic included the fellow observing a PT complete a mobility assessment and a pharmacist conduct a high-risk medication review. The fellow synthesized the findings of the mobility assessment and medication review, as well as their own history and assessment, to create a summary of fall risk recommendations to discuss with their volunteer patient partner. The fellows were observed and evaluated in their skills by their patient partner, course faculty, and another fellow. The patient partners, and their assigned fellow, also participated in a 45-minute fall risk presentation, led by a nurse.

On the fourth day, the fellows were joined by select clinic partners, including nurses, pharmacists, and/or medical assistants. The session included discussions among each fellow’s clinical team regarding the current state of fall risk efforts at their clinic, an analysis of barriers, and identification of opportunities to improve workflows and screening rates. Each fellow took with them an action plan tailored to their clinic to improve fall risk management practices, starting with the fellow’s own practice.

Fall Risk Management Plan

The educational sessions introduced the fellows to the FRMP. The FRMP, adapted from the STEADI toolkit, includes a process for fall risk screening (Figure 1) and stratifying a patient’s risk based on their STEADI score in order to promote 3 priority assessments (gait evaluation with PT referral if appropriate; orthostatic blood pressure; and high-risk medication review; Figure 2). Initial actions based on these priority assessments were followed over time, with additional fall risk interventions added as clinically indicated.²⁵ The FRMP is intended to be used during routine office visits, Medicare annual wellness visits, or office visits focused on fall risk or related medical disorders (ie, fall risk visits.)

Providers and their teams were encouraged to spread out fall-related conversations with their patients over multiple visits, since many patients have multiple fall risk factors at play, in addition to other chronic medical issues, and since many interventions often require behavior changes on the part of the patient. Providers also had access to fall-related electronic health record (EHR) templates as well as a comprehensive, internal fall risk management website that included assessment tools, evidence-based resources, and patient handouts.

Assessment and Measurements

We assessed provider knowledge and comfort in their fall risk evaluation and management skills before and after the educational intervention using an 11-item multiple-choice questionnaire and a 4-item confidence questionnaire. The confidence questions used a 7-point Likert scale, with 0 indicating “no confidence” and 7 indicating ”lots of confidence.” The questions were administered via a paper survey. Qualitative comments were derived from evaluations completed at the end of the week.

The fellows’ practice of fall risk screening and management was studied from May 2018, at the completion of Mobility week, to May 2019 for the post-intervention period. A 1-year timeframe before May 2018 was used as the pre-intervention period. Eligible visit types, during which we assumed fall risk was discussed, were any office visits for patients 65+ completed by the patients’ PCPs that used fall risk as a reason for the visit or had a fall-related diagnosis code. Fall risk visits performed by other clinic providers were not counted.

Of those patients who had fall risk screenings completed and were determined to be high risk (STEADI score ≥ 4), data were analyzed to determine whether these patients had any fall-related follow-up visits to their PCP within 60 days of the STEADI screening. For these high-risk patients, data were studied to understand whether orthostatic blood pressure measurements were performed (as documented in a flowsheet) and whether a PT referral was placed. These data were compared with those from providers who practiced in clinics within the same system but who did not participate in the mini-fellowship. Data were obtained from the organization’s EHR. Additional data were measured to evaluate patterns of deprescribing of select high-risk medications, but these data are not included in this analysis.

Analysis

A paired-samples t test was used to measure changes in provider confidence levels. Data were aggregated across fellows, resulting in a mean. A chi-square test of independence was performed to examine the relationship between rates of FRMP adoption by select provider groups. Analysis included a pre- and post-intervention assessment of the fellows’ adoption of FRMP practices, as well as a comparison between the fellows’ practice patterns and those of a control group of PCPs in the organization’s other clinics who did not participate in the mini-fellowship (nontrained control group). Excluded from the control group were providers from the same clinic as the fellows; providers in clinics with a geriatric-trained provider on staff; and clinics outside of the Portland metro and Medford service areas. We used an alpha level of 0.05 for all statistical tests.

Data from 5 providers were included in the analysis of the FRMP adoption. The sixth provider changed practice settings from the clinic to the ED after completing the fellowship; her patient data were not included in the FRMP part of the analysis. EHR data included data on all visits of patients 65+, as well as data for just those 65+ patients who had been identified as being at high risk to fall based on a STEADI score of 4 or higher.

Results

Provider Questionnaire

All 6 providers responded to the pre-intervention and post-intervention tests. For the knowledge questions, fellows, as a composite, correctly answered 57% of the questions before the intervention and 79% after the intervention. Provider confidence level in delivering fall risk care was measured prior to the training (mean, 4.12 [SD, 0.62]) and at the end of the training (mean, 6.47 [SD, 0.45]), demonstrating a significant increase in confidence (t (5) = –10.46, P < 0.001).

Qualitative Comments

Providers also had the opportunity to provide comments on their experience during the Mobility week and at the end of 1 year. In general, the simulated interdisciplinary fall risk clinic was highly rated (“the highlight of the week”) as a practical strategy to embed learning principles. One fellow commented, “Putting the learning into practice helps solidify it in my brain.” Fellows also appreciated the opportunity to learn and meet with their clinic colleagues to begin work on a fall-risk focused PIP and to “have a framework for what to do for people who screen positive [for fall risk].”

FRMP Adoption

A comparison of the care the fellows provided to their patients 65+ in the 12 months pre- and post-training shows the fellows demonstrated significant changes in practice patterns. The fellows were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to check orthostatic blood pressure than prior to the mini-fellowship (Table 1).

The control providers also demonstrated significant increases in fall risk screening and discussion of fall risk between the pre- and post-intervention periods; however, the relative risk (RR) was between 1.10 and 1.13 for this group. For the control group, checking orthostatic blood pressure did not significantly change. In the 12 months after training (Table 2), the fellows were 4.2 times more likely to discuss fall risk and almost 5 times more likely to check orthostatic blood pressure than their nontrained peers for all of their patients 65+, regardless of their risk to fall.

As shown in Table 3, for those patients determined to be at high risk of falling (STEADI score ≥ 4), fellows showed statistically significant increases in fall risk visits (RR, 3.02) and assessment of orthostatic blood pressure (RR, 10.68) before and after the mini-fellowship. The control providers did not show any changes in practice patterns between the pre- and post-period among patients at high risk to fall.

Neither the fellows nor the control group showed changes in patterns of referral to PT. In comparing the 2 groups in the 12 months after training (Table 4), for their patients at risk of falling, the fellows were 4 times more likely to complete fall risk visits and over 6 times more likely to assess orthostatic blood pressure than their nontrained peers. Subgroup analysis of the 75+ population revealed similar trends and significance, but these results are not included here.

Discussion

This study aimed to improve not only providers’ knowledge and confidence in caring for older adults at increased risk to fall, but also their clinical practice in assessing and managing fall risk. In addition to improved knowledge and confidence, we found that the fellows increased their discussion of fall risk (through fall risk visits) and their assessment of orthostatic blood pressure for all of their patients, not just for those identified at increased risk to fall. This improvement held true for the fellows themselves before and after the intervention, but also as compared to their nontrained peers. These practice improvements for all of their 65+ patients, not just those identified as being at high risk to fall, are especially important, since studies indicate that early screening and intervention can help identify people at risk and prevent future falls.¹⁵

We were surprised that there were no significant differences in PT referrals made by the trained fellows, but this finding may have been confounded by the fact that the data included all PT referrals, regardless of diagnosis, not just those referrals that were fall-related. Furthermore, our baseline PT referral rates, at 39% for the intervention group and 42% for the control group, are higher than national data when looking at rehabilitation use by older adults.²⁶

In comparison to a study evaluating the occurrence of fall risk–related clinical practice in primary care before any fall-related educational intervention, orthostatics were checked less frequently in our study (10% versus 30%) and there were fewer PT referrals (42%–44% versus 53%).²⁷ However, the Phelan study took place in patients who had actually had a fall, rather than just having a higher risk for a fall, and was based on detailed chart review. Other studies^23,24 found higher rates of fall risk interventions, but did not break out PT referrals specifically.

In terms of the educational intervention itself, most studies of geriatric education interventions have measured changes in knowledge, confidence, or self-efficacy as they relate to geriatric competence,^28-30 and do not measure practice change as an outcome outside of intent to change or self-reported practice change.^31,32 In general, practice change or longer-term health care–related outcomes have not been studied. Additionally, a range of dosages of educational interventions has been studied, from 1-hour lunchtime presentations^23,32 to half-day²⁹ or several half-day workshops,²⁸ up to 160 hours over 10 months³⁰ or 5 weekends over 6 months.³¹ The duration of our entire intervention at 160 hours over 6 months would be considered on the upper end of dosing relative to these studies, with our Mobility week intervention comprising 32 hours during 1 week. In the Warshaw study, despite 107 1-hour sessions being taught to over 60 physicians in 16 practices over 4 years, only 2 practices ultimately initiated any practice change projects.³² We believe that only curricula that embed practice change skills and opportunities, at a significant enough dose, can actually impact practice change in a sustainable manner.

Knowledge and skill acquisition among individual providers does not take place to a sufficient degree in the current health care arena, which is focused on productivity and short visit times. Consistent with other studies, we included interdisciplinary members of the primary care team for part of the mini-fellowship, although other studies used models that train across disciplines for the entirety of the learning experience.^28-30,33 Our educational model was strengthened by including other professionals to provide some of the education and model the ideal geriatric team, including PT, occupational therapy, and pharmacy, for the week on mobility.

Most studies exploring interventions through geriatric educational initiatives are conducted within academic institutions, with a primary focus on physician faculty and, by extension, their teaching of residents and others.^34,35 We believe our integrated model, which is steeped in community-based primary care practices like Lam’s,³¹ offers the greatest outreach to large community-based care systems and their patients. Training providers to work with their teams to change their own practices first gives skills and expertise that help further establish them as geriatric champions within their practices, laying the groundwork for more widespread practice change at their clinics.

Limitations

In addition to the limitations described above relating to the capture of PT referrals, other limitations included the relatively short time period for follow-up data as well as the small size of the intervention group. However, we found value in the instructional depth that the small group size allowed.

While the nontrained providers did show some improvement during the same period, we believe the relative risk was not clinically significant. We suspect that the larger health system efforts to standardize screening of patients 65+ across all clinics as a core quality metric confounded these results. The data analysis also included only fall-related patient visits that occurred with a provider who was that patient’s PCP, which could have missed visits done by other PCP colleagues, RNs, or pharmacists in the same clinic, thus undercounting the true number of fall-related visits. Furthermore, counting of fall-related interventions relied upon providers documenting consistently in the EHR, which could also lead to under-represention of fall risk clinical efforts.

The data presented, while encouraging, do not reflect clinic-wide practice change patterns and are considered only proximate outcomes rather than more long-term or cost-related outcomes, as would be captured by fall-related utilization measures like emergency room visits and hospitalizations. We expect to evaluate the broader impact and these value-based outcomes in the future. All providers and teams were from the same health care system, which may not allow our results to transfer to other organizations or regions of clinical practice.

Summary

This study demonstrates that an intensive mini-fellowship model of geriatrics training improved both knowledge and confidence in the realm of fall risk assessment and intervention among PCPs who had not been formally trained in geriatrics. More importantly, the training improved the fall-related care of their patients at increased risk to fall, but also of all of their older patients, with improvements in care measured up to a year after the mini-fellowship. Although this article only describes the work done as part of the Mobility aim of the 4M AFHS model, we believe the entire mini-fellowship curriculum offers the opportunity to “geriatricize” clinicians and their teams in learning geriatric principles and skills that they can translate into their practice in a sustainable way, as Tinetti encourages.⁸ Future study to evaluate other process outcomes more precisely, such as PT, as well as cost- and value-based outcomes, and the influence of trained providers on their clinic partners, will further establish the value proposition of targeted, disseminated, intensive geriatrics training of primary care clinicians as a strategy of age-friendly health systems as they work to improve the care of their older adults.

Acknowledgment: We are grateful for the dedication and hard work of the 2018 Geriatric Mini-Fellowship fellows at Providence Health & Services-Oregon who made this article possible. Thanks to Drs. Stephanie Cha, Emily Puukka-Clark, Laurie Dutkiewicz, Cara Ellis, Deb Frost, Jordan Roth, and Subhechchha Shah for promoting the AFHS work within their Providence Medical Group clinics and to PMG leadership and the fellows’ clinical teams for supporting the fellows, the AFHS work, and their older patients.

Corresponding author: Colleen M. Casey, PhD, ANP-BC, Providence Health & Services, Senior Health Program, 4400 NE Halsey, 5th Floor, Portland, OR 97213; [email protected].

Financial disclosures: None.

From the Senior Health Program, Providence Health & Services, Oregon, Portland, OR.

Abstract

Background: Approximately 51 million adults in the United States are 65 years of age or older, yet few geriatric-trained primary care providers (PCP) serve this population. The Age-Friendly Health System framework, consisting of evidence-based 4M care (Mobility, Medication, Mentation, and what Matters), encourages all PCPs to assess mobility in older adults.

Objective: To improve PCP knowledge, confidence, and clinical practice in assessing and managing fall risk.

Methods: A 1-week educational session focusing on mobility (part of a 4-week Geriatric Mini-Fellowship) for 6 selected PCPs from a large health care system was conducted to increase knowledge and ability to address fall risk in older adults. The week included learning and practicing a Fall Risk Management Plan (FRMP) algorithm, including planning for their own practice changes. Pre- and post-test surveys assessed changes in knowledge and confidence. Patient data were compared 12 months before and after training to evaluate PCP adoption of FRMP components.

Results: The training increased provider knowledge and confidence. The trained PCPs were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to assess orthostatic blood pressure in their 65+ patients after the mini-fellowship. In high-risk patients, they were 4.1 times more likely to discuss fall risk and 6.3 times more likely to assess orthostatic blood pressure than their nontrained peers. Changes in physical therapy referral rates were not observed.

Conclusions: In-depth, skills-based geriatric educational sessions improved PCPs’ knowledge and confidence and also improved their fall risk management practices for their older patients.

Keywords: geriatrics; guidelines; Age-Friendly Health System; 4M; workforce training; practice change; fellowship.

The US population is aging rapidly. People aged 85 years and older are the largest-growing segment of the US population, and this segment is expected to increase by 123% by 2040.¹ Caregiving needs increase with age as older adults develop more chronic conditions, such as hypertension, heart disease, arthritis, and dementia. However, even with increasing morbidity and dependence, a majority of older adults still live in the community rather than in institutional settings.² These older adults seek medical care more frequently than younger people, with about 22% of patients 75 years and older having 10 or more health care visits in the previous 12 months. By 2040, nearly a quarter of the US population is expected to be 65 or older, with many of these older adults seeking regular primary care from providers who do not have formal training in the care of a population with multiple complex, chronic health conditions and increased caregiving needs.¹

Despite this growing demand for health care professionals trained in the care of older adults, access to these types of clinicians is limited. In 2018, there were roughly 7000 certified geriatricians, with only 3600 of them practicing full-time.^3,4 Similarly, of 290,000 certified nurse practitioners (NPs), about 9% of them have geriatric certification.⁵ Geriatricians, medical doctors trained in the care of older adults, and geriatric-trained NPs are part of a cadre of a geriatric-trained workforce that provides unique expertise in caring for older adults with chronic and advanced illness. They know how to manage multiple, complex geriatric syndromes like falls, dementia, and polypharmacy; understand and maximize team-based care; and focus on caring for an older person with a goal-centered versus a disease-centered approach.⁶

Broadly, geriatric care includes a spectrum of adults, from those who are aging healthfully to those who are the frailest. Research has suggested that approximately 30% of older adults need care by a geriatric-trained clinician, with the oldest and frailest patients needing more clinician time for assessment and treatment, care coordination, and coaching of caregivers.⁷ With this assumption in mind, it is projected that by 2025, there will be a national shortage of 26,980 geriatricians, with the western United States disproportionately affected by this shortage.⁴Rather than lamenting this shortage, Tinetti recommends a new path forward: “Our mission should not be to train enough geriatricians to provide direct care, but rather to ensure that every clinician caring for older adults is competent in geriatric principles and practices.”⁸ Sometimes called ”geriatricizing,” the idea is to use existing geriatric providers as a small elite training force to infuse geriatric principles and skills across their colleagues in primary care and other disciplines.^8,9 Efforts of the American Geriatrics Society (AGS), with support from the John A. Hartford Foundation (JAHF), have been successful in developing geriatric training across multiple specialties, including surgery, orthopedics, and emergency medicine (www.americangeriatrics.org/programs/geriatrics-specialists-initiative).

The Age-Friendly Health System and 4M Model

To help augment this idea of equipping health care systems and their clinicians with more readily available geriatric knowledge, skills, and tools, the JAHF, along with the Institute for Healthcare Improvement (IHI), created the Age-Friendly Health System (AFHS) paradigm in 2015.¹⁰ Using the 4M model, the AFHS initiative established a set of evidence-based geriatric priorities and interventions meant to improve the care of older adults, reduce harm and duplication, and provide a framework for engaging leadership, clinical teams, and operational systems across inpatient and ambulatory settings.¹¹ Mobility, including fall risk screening and intervention, is 1 of the 4M foundational elements of the Age-Friendly model. In addition to Mobility, the 4M model also includes 3 other key geriatric domains: Mentation (dementia, depression, and delirium), Medication (high-risk medications, polypharmacy, and deprescribing), and What Matters (goals of care conversations and understanding quality of life for older patients).¹¹ The 4M initiative encourages adoption of a geriatric lens that looks across chronic conditions and accounts for the interplay among geriatric syndromes, such as falls, cognitive impairment, and frailty, in order to provide care better tailored to what the patient needs and desires.¹² IHI and JAHF have targeted the adoption of the 4M model by 20% of US health care systems by 2020.¹¹

Mini-Fellowship and Mobility Week

To bolster geriatric skills among community-based primary care providers (PCPs), we initiated a Geriatric Mini-Fellowship, a 4-week condensed curriculum taught over 6 months. Each week focuses on 1 of the age-friendly 4Ms, with the goal of increasing the knowledge, self-efficacy, skills, and competencies of the participating PCPs (called “fellow” hereafter) and at the same time, equipping each to become a champion of geriatric practice. This article focuses on the Mobility week, the second week of the mini-fellowship, and the effect of the week on the fellows’ practice changes.

To construct the Mobility week’s curriculum with a focus on the ambulatory setting, we relied upon national evidence-based work in fall risk management. The Centers for Disease Control and Prevention (CDC) has made fall risk screening and management in primary care a high priority. Using the clinical practice guidelines for managing fall risk developed by the American and British Geriatrics Societies (AGS/BGS), the CDC developed the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) toolkit.¹³ Foundational to the toolkit is the validated 12-item Stay Independent falls screening questionnaire (STEADI questionnaire).¹⁴ Patients who score 4 or higher (out of a total score of 14) on the questionnaire are considered at increased risk of falling. The CDC has developed a clinical algorithm that guides clinical teams through screening and assessment to help identify appropriate interventions to target specific risk factors. Research has clearly established that a multifactorial approach to fall risk intervention can be successful in reducing fall risk by as much as 25%.^15-17

The significant morbidity and mortality caused by falls make training nongeriatrician clinicians on how to better address fall risk imperative. More than 25% of older adults fall each year.¹⁸ These falls contribute to rising rates of fall-related deaths,¹⁹ emergency department (ED) visits,²⁰ and hospital readmissions.²¹ Initiatives like the AFHS focus on mobility and the CDC’s development of supporting clinical materials²² aim to improve primary care adoption of fall risk screening and intervention practices.^23,24 The epidemic of falls must compel all PCPs, not just those practicing geriatrics, to make discussing and addressing fall risk and falls a priority.

Methods

Setting

This project took place as part of a regional primary care effort in Oregon. Providence Health & Services-Oregon is part of a multi-state integrated health care system in the western United States whose PCPs serve more than 80,000 patients aged 65 years and older per year; these patients comprise 38% of the system’s office visits each year. Regionally, there are 47 family and internal medicine clinics employing roughly 290 providers (physicians, NPs, and physician assistants). The organization has only 4 PCPs trained in geriatrics and does not offer any geriatric clinical consultation services. Six PCPs from different clinics, representing both rural and urban settings, are chosen to participate in the geriatric mini-fellowship each year.

This project was conducted as a quality improvement initiative within the organization and did not constitute human subjects research. It was not conducted under the oversight of the Institutional Review Board.

Intervention

The mini-fellowship was taught in 4 1-week blocks between April and October 2018, with a curriculum designed to be interactive and practical. The faculty was intentionally interdisciplinary to teach and model team-based practice. Each week participants were excused from their clinical practice. Approximately 160 hours of continuing medical education credits were awarded for the full mini-fellowship. As part of each weekly session, a performance improvement project (PIP) focused on that week’s topic (1 of the 4Ms) was developed by the fellow and their team members to incorporate the mini-fellowship learnings into their clinic workflows. Fellows also had 2 hours per week of dedicated administration time for a year, outside the fellowship, to work on their PIP and 4M practice changes within their clinic.

Provider Education

The week for mobility training comprised 4 daylong sessions. The first 2 days were spent learning about the epidemiology of falls; risk factors for falling; how to conduct a thorough history and assessment of fall risk; and how to create a prioritized Fall Risk Management Plan (FRMP) to decrease a patient’s individual fall risk through tailored interventions. The FRMP was adapted from the CDC STEADI toolkit.¹³ Core faculty were 2 geriatric-trained providers (NP and physician) and a physical therapist (PT) specializing in fall prevention.

On the third day, fellows took part in a simulated fall risk clinic, in which older adults volunteered to be patient partners, providing an opportunity to apply learnings from days 1 and 2. The clinic included the fellow observing a PT complete a mobility assessment and a pharmacist conduct a high-risk medication review. The fellow synthesized the findings of the mobility assessment and medication review, as well as their own history and assessment, to create a summary of fall risk recommendations to discuss with their volunteer patient partner. The fellows were observed and evaluated in their skills by their patient partner, course faculty, and another fellow. The patient partners, and their assigned fellow, also participated in a 45-minute fall risk presentation, led by a nurse.

On the fourth day, the fellows were joined by select clinic partners, including nurses, pharmacists, and/or medical assistants. The session included discussions among each fellow’s clinical team regarding the current state of fall risk efforts at their clinic, an analysis of barriers, and identification of opportunities to improve workflows and screening rates. Each fellow took with them an action plan tailored to their clinic to improve fall risk management practices, starting with the fellow’s own practice.

Fall Risk Management Plan

The educational sessions introduced the fellows to the FRMP. The FRMP, adapted from the STEADI toolkit, includes a process for fall risk screening (Figure 1) and stratifying a patient’s risk based on their STEADI score in order to promote 3 priority assessments (gait evaluation with PT referral if appropriate; orthostatic blood pressure; and high-risk medication review; Figure 2). Initial actions based on these priority assessments were followed over time, with additional fall risk interventions added as clinically indicated.²⁵ The FRMP is intended to be used during routine office visits, Medicare annual wellness visits, or office visits focused on fall risk or related medical disorders (ie, fall risk visits.)

Providers and their teams were encouraged to spread out fall-related conversations with their patients over multiple visits, since many patients have multiple fall risk factors at play, in addition to other chronic medical issues, and since many interventions often require behavior changes on the part of the patient. Providers also had access to fall-related electronic health record (EHR) templates as well as a comprehensive, internal fall risk management website that included assessment tools, evidence-based resources, and patient handouts.

Assessment and Measurements

We assessed provider knowledge and comfort in their fall risk evaluation and management skills before and after the educational intervention using an 11-item multiple-choice questionnaire and a 4-item confidence questionnaire. The confidence questions used a 7-point Likert scale, with 0 indicating “no confidence” and 7 indicating ”lots of confidence.” The questions were administered via a paper survey. Qualitative comments were derived from evaluations completed at the end of the week.

The fellows’ practice of fall risk screening and management was studied from May 2018, at the completion of Mobility week, to May 2019 for the post-intervention period. A 1-year timeframe before May 2018 was used as the pre-intervention period. Eligible visit types, during which we assumed fall risk was discussed, were any office visits for patients 65+ completed by the patients’ PCPs that used fall risk as a reason for the visit or had a fall-related diagnosis code. Fall risk visits performed by other clinic providers were not counted.

Of those patients who had fall risk screenings completed and were determined to be high risk (STEADI score ≥ 4), data were analyzed to determine whether these patients had any fall-related follow-up visits to their PCP within 60 days of the STEADI screening. For these high-risk patients, data were studied to understand whether orthostatic blood pressure measurements were performed (as documented in a flowsheet) and whether a PT referral was placed. These data were compared with those from providers who practiced in clinics within the same system but who did not participate in the mini-fellowship. Data were obtained from the organization’s EHR. Additional data were measured to evaluate patterns of deprescribing of select high-risk medications, but these data are not included in this analysis.

Analysis

A paired-samples t test was used to measure changes in provider confidence levels. Data were aggregated across fellows, resulting in a mean. A chi-square test of independence was performed to examine the relationship between rates of FRMP adoption by select provider groups. Analysis included a pre- and post-intervention assessment of the fellows’ adoption of FRMP practices, as well as a comparison between the fellows’ practice patterns and those of a control group of PCPs in the organization’s other clinics who did not participate in the mini-fellowship (nontrained control group). Excluded from the control group were providers from the same clinic as the fellows; providers in clinics with a geriatric-trained provider on staff; and clinics outside of the Portland metro and Medford service areas. We used an alpha level of 0.05 for all statistical tests.

Data from 5 providers were included in the analysis of the FRMP adoption. The sixth provider changed practice settings from the clinic to the ED after completing the fellowship; her patient data were not included in the FRMP part of the analysis. EHR data included data on all visits of patients 65+, as well as data for just those 65+ patients who had been identified as being at high risk to fall based on a STEADI score of 4 or higher.

Results

Provider Questionnaire

All 6 providers responded to the pre-intervention and post-intervention tests. For the knowledge questions, fellows, as a composite, correctly answered 57% of the questions before the intervention and 79% after the intervention. Provider confidence level in delivering fall risk care was measured prior to the training (mean, 4.12 [SD, 0.62]) and at the end of the training (mean, 6.47 [SD, 0.45]), demonstrating a significant increase in confidence (t (5) = –10.46, P < 0.001).

Qualitative Comments

Providers also had the opportunity to provide comments on their experience during the Mobility week and at the end of 1 year. In general, the simulated interdisciplinary fall risk clinic was highly rated (“the highlight of the week”) as a practical strategy to embed learning principles. One fellow commented, “Putting the learning into practice helps solidify it in my brain.” Fellows also appreciated the opportunity to learn and meet with their clinic colleagues to begin work on a fall-risk focused PIP and to “have a framework for what to do for people who screen positive [for fall risk].”

FRMP Adoption

A comparison of the care the fellows provided to their patients 65+ in the 12 months pre- and post-training shows the fellows demonstrated significant changes in practice patterns. The fellows were 1.7 times more likely to screen for fall risk; 3.6 times more likely to discuss fall risk; and 5.8 times more likely to check orthostatic blood pressure than prior to the mini-fellowship (Table 1).

The control providers also demonstrated significant increases in fall risk screening and discussion of fall risk between the pre- and post-intervention periods; however, the relative risk (RR) was between 1.10 and 1.13 for this group. For the control group, checking orthostatic blood pressure did not significantly change. In the 12 months after training (Table 2), the fellows were 4.2 times more likely to discuss fall risk and almost 5 times more likely to check orthostatic blood pressure than their nontrained peers for all of their patients 65+, regardless of their risk to fall.

As shown in Table 3, for those patients determined to be at high risk of falling (STEADI score ≥ 4), fellows showed statistically significant increases in fall risk visits (RR, 3.02) and assessment of orthostatic blood pressure (RR, 10.68) before and after the mini-fellowship. The control providers did not show any changes in practice patterns between the pre- and post-period among patients at high risk to fall.

Neither the fellows nor the control group showed changes in patterns of referral to PT. In comparing the 2 groups in the 12 months after training (Table 4), for their patients at risk of falling, the fellows were 4 times more likely to complete fall risk visits and over 6 times more likely to assess orthostatic blood pressure than their nontrained peers. Subgroup analysis of the 75+ population revealed similar trends and significance, but these results are not included here.

Discussion

This study aimed to improve not only providers’ knowledge and confidence in caring for older adults at increased risk to fall, but also their clinical practice in assessing and managing fall risk. In addition to improved knowledge and confidence, we found that the fellows increased their discussion of fall risk (through fall risk visits) and their assessment of orthostatic blood pressure for all of their patients, not just for those identified at increased risk to fall. This improvement held true for the fellows themselves before and after the intervention, but also as compared to their nontrained peers. These practice improvements for all of their 65+ patients, not just those identified as being at high risk to fall, are especially important, since studies indicate that early screening and intervention can help identify people at risk and prevent future falls.¹⁵

We were surprised that there were no significant differences in PT referrals made by the trained fellows, but this finding may have been confounded by the fact that the data included all PT referrals, regardless of diagnosis, not just those referrals that were fall-related. Furthermore, our baseline PT referral rates, at 39% for the intervention group and 42% for the control group, are higher than national data when looking at rehabilitation use by older adults.²⁶

In comparison to a study evaluating the occurrence of fall risk–related clinical practice in primary care before any fall-related educational intervention, orthostatics were checked less frequently in our study (10% versus 30%) and there were fewer PT referrals (42%–44% versus 53%).²⁷ However, the Phelan study took place in patients who had actually had a fall, rather than just having a higher risk for a fall, and was based on detailed chart review. Other studies^23,24 found higher rates of fall risk interventions, but did not break out PT referrals specifically.

In terms of the educational intervention itself, most studies of geriatric education interventions have measured changes in knowledge, confidence, or self-efficacy as they relate to geriatric competence,^28-30 and do not measure practice change as an outcome outside of intent to change or self-reported practice change.^31,32 In general, practice change or longer-term health care–related outcomes have not been studied. Additionally, a range of dosages of educational interventions has been studied, from 1-hour lunchtime presentations^23,32 to half-day²⁹ or several half-day workshops,²⁸ up to 160 hours over 10 months³⁰ or 5 weekends over 6 months.³¹ The duration of our entire intervention at 160 hours over 6 months would be considered on the upper end of dosing relative to these studies, with our Mobility week intervention comprising 32 hours during 1 week. In the Warshaw study, despite 107 1-hour sessions being taught to over 60 physicians in 16 practices over 4 years, only 2 practices ultimately initiated any practice change projects.³² We believe that only curricula that embed practice change skills and opportunities, at a significant enough dose, can actually impact practice change in a sustainable manner.

Knowledge and skill acquisition among individual providers does not take place to a sufficient degree in the current health care arena, which is focused on productivity and short visit times. Consistent with other studies, we included interdisciplinary members of the primary care team for part of the mini-fellowship, although other studies used models that train across disciplines for the entirety of the learning experience.^28-30,33 Our educational model was strengthened by including other professionals to provide some of the education and model the ideal geriatric team, including PT, occupational therapy, and pharmacy, for the week on mobility.

Most studies exploring interventions through geriatric educational initiatives are conducted within academic institutions, with a primary focus on physician faculty and, by extension, their teaching of residents and others.^34,35 We believe our integrated model, which is steeped in community-based primary care practices like Lam’s,³¹ offers the greatest outreach to large community-based care systems and their patients. Training providers to work with their teams to change their own practices first gives skills and expertise that help further establish them as geriatric champions within their practices, laying the groundwork for more widespread practice change at their clinics.

Limitations

In addition to the limitations described above relating to the capture of PT referrals, other limitations included the relatively short time period for follow-up data as well as the small size of the intervention group. However, we found value in the instructional depth that the small group size allowed.

While the nontrained providers did show some improvement during the same period, we believe the relative risk was not clinically significant. We suspect that the larger health system efforts to standardize screening of patients 65+ across all clinics as a core quality metric confounded these results. The data analysis also included only fall-related patient visits that occurred with a provider who was that patient’s PCP, which could have missed visits done by other PCP colleagues, RNs, or pharmacists in the same clinic, thus undercounting the true number of fall-related visits. Furthermore, counting of fall-related interventions relied upon providers documenting consistently in the EHR, which could also lead to under-represention of fall risk clinical efforts.

The data presented, while encouraging, do not reflect clinic-wide practice change patterns and are considered only proximate outcomes rather than more long-term or cost-related outcomes, as would be captured by fall-related utilization measures like emergency room visits and hospitalizations. We expect to evaluate the broader impact and these value-based outcomes in the future. All providers and teams were from the same health care system, which may not allow our results to transfer to other organizations or regions of clinical practice.

Summary

This study demonstrates that an intensive mini-fellowship model of geriatrics training improved both knowledge and confidence in the realm of fall risk assessment and intervention among PCPs who had not been formally trained in geriatrics. More importantly, the training improved the fall-related care of their patients at increased risk to fall, but also of all of their older patients, with improvements in care measured up to a year after the mini-fellowship. Although this article only describes the work done as part of the Mobility aim of the 4M AFHS model, we believe the entire mini-fellowship curriculum offers the opportunity to “geriatricize” clinicians and their teams in learning geriatric principles and skills that they can translate into their practice in a sustainable way, as Tinetti encourages.⁸ Future study to evaluate other process outcomes more precisely, such as PT, as well as cost- and value-based outcomes, and the influence of trained providers on their clinic partners, will further establish the value proposition of targeted, disseminated, intensive geriatrics training of primary care clinicians as a strategy of age-friendly health systems as they work to improve the care of their older adults.

Acknowledgment: We are grateful for the dedication and hard work of the 2018 Geriatric Mini-Fellowship fellows at Providence Health & Services-Oregon who made this article possible. Thanks to Drs. Stephanie Cha, Emily Puukka-Clark, Laurie Dutkiewicz, Cara Ellis, Deb Frost, Jordan Roth, and Subhechchha Shah for promoting the AFHS work within their Providence Medical Group clinics and to PMG leadership and the fellows’ clinical teams for supporting the fellows, the AFHS work, and their older patients.

Corresponding author: Colleen M. Casey, PhD, ANP-BC, Providence Health & Services, Senior Health Program, 4400 NE Halsey, 5th Floor, Portland, OR 97213; [email protected].

Financial disclosures: None.

Early in the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, the World Health Organization issued guidance for coronavirus disease 2019 (COVID-19) management.¹ Based on a high intubation rate among 12 subjects with Middle Eastern respiratory syndrome, noninvasive ventilation (NIV) was discouraged.² While high-flow nasal oxygen (HFNO) was recognized as a reasonable strategy to avoid endotracheal intubation,¹ uncertainty regarding the potential of both therapies to aerosolize SARS-CoV-2 and reports of rapid, unexpected respiratory decompensations were deterrents to use.³ As hospitals prepared for a surge of patients, reports of SARS-CoV-2 transmission to healthcare personnel also emerged. Together, these issues led many institutions to recommend lower than usual thresholds for intubation. This well-intentioned guidance was based on limited historical data, a rapidly evolving literature that frequently appeared on preprint servers before peer review, or as anecdotes on social media.

As COVID-19 caseloads increased, clinicians were immediately faced with patients who rapidly reached the planned intubation threshold, but also looked very comfortable with minimal to no use of accessory muscles of respiration. In addition, the pace of respiratory decompensation among those who ultimately required intubation was slower than expected. Moreover, intensive care unit (ICU) capacity was stretched thin, raising concern for an imminent need for ventilator rationing. Lastly, the risk of SARS-CoV-2 transmission to healthcare workers appeared well-controlled with the use of personal protective equipment.⁴

In light of this accumulating experience, sites worldwide evolved quickly from their initial management strategies for COVID-19 respiratory failure. However, the deliberate process described by Soares et al in this issue of the Journal of Hospital Medicine is notable.⁵ Their transition towards the beginning of the pandemic from a conservative early intubation approach to a new strategy that encouraged use of NIV, HFNO, and self-proning is described. They were motivated by reports of good outcomes using these interventions, high mortality in intubated patients, and reassurance that aerosolization of respiratory secretions during NIV and HFNO was comparable to regular nasal cannula or face mask oxygen.³ The new protocol was defined and rapidly deployed over 4 days using multipronged communication from project and institutional leaders via in person and electronic means (email, Whatsapp, GoogleDrive). To facilitate implementation, COVID-19 patients requiring respiratory support were placed in dedicated units with bedside flowsheets for guidance. An immediate impact was demonstrated over the next 2 weeks by a significant decrease in use of mechanical ventilation in COVID-19 patients from 25.2% to 10.7%. In-hospital mortality, the primary outcome, did not change, ICU admissions decreased, as did hospital length of stay (10 vs 8.4 days, though not statistically significant), all providing supportive evidence for relative safety of the new protocol.

Soares et al exemplify a nimble system that recognized planned strategies to be problematic, and then achieved rapid implementation of a new protocol across a four-hospital system. Changes in medical practice are typically much slower, with some studies suggesting this process may take a decade or more. Implementation science focuses on translating research evidence into clinical practice using strategies tailored to particular contexts. The current study harnessed important implementation principles to quickly translate evidence into practice using effective engagement and education of key stakeholders across specialties (eg, emergency medicine, hospitalists, critical care, and respiratory therapy), the identification of pathways that mitigated barriers, frequent re-evaluation of a rapidly evolving literature, and an open-mindedness to the value of change.⁶ As the pandemic continues, traditional research and implementation science are critical not only to define optimal treatments and management strategies, but also to learn how best to implement successful interventions in an accelerated manner.⁷

The authors reported no conflicts of interest.

Dr Hochberg is supported by a National Institutes of Health training grant (T32HL007534).

Early in the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, the World Health Organization issued guidance for coronavirus disease 2019 (COVID-19) management.¹ Based on a high intubation rate among 12 subjects with Middle Eastern respiratory syndrome, noninvasive ventilation (NIV) was discouraged.² While high-flow nasal oxygen (HFNO) was recognized as a reasonable strategy to avoid endotracheal intubation,¹ uncertainty regarding the potential of both therapies to aerosolize SARS-CoV-2 and reports of rapid, unexpected respiratory decompensations were deterrents to use.³ As hospitals prepared for a surge of patients, reports of SARS-CoV-2 transmission to healthcare personnel also emerged. Together, these issues led many institutions to recommend lower than usual thresholds for intubation. This well-intentioned guidance was based on limited historical data, a rapidly evolving literature that frequently appeared on preprint servers before peer review, or as anecdotes on social media.

As COVID-19 caseloads increased, clinicians were immediately faced with patients who rapidly reached the planned intubation threshold, but also looked very comfortable with minimal to no use of accessory muscles of respiration. In addition, the pace of respiratory decompensation among those who ultimately required intubation was slower than expected. Moreover, intensive care unit (ICU) capacity was stretched thin, raising concern for an imminent need for ventilator rationing. Lastly, the risk of SARS-CoV-2 transmission to healthcare workers appeared well-controlled with the use of personal protective equipment.⁴

In light of this accumulating experience, sites worldwide evolved quickly from their initial management strategies for COVID-19 respiratory failure. However, the deliberate process described by Soares et al in this issue of the Journal of Hospital Medicine is notable.⁵ Their transition towards the beginning of the pandemic from a conservative early intubation approach to a new strategy that encouraged use of NIV, HFNO, and self-proning is described. They were motivated by reports of good outcomes using these interventions, high mortality in intubated patients, and reassurance that aerosolization of respiratory secretions during NIV and HFNO was comparable to regular nasal cannula or face mask oxygen.³ The new protocol was defined and rapidly deployed over 4 days using multipronged communication from project and institutional leaders via in person and electronic means (email, Whatsapp, GoogleDrive). To facilitate implementation, COVID-19 patients requiring respiratory support were placed in dedicated units with bedside flowsheets for guidance. An immediate impact was demonstrated over the next 2 weeks by a significant decrease in use of mechanical ventilation in COVID-19 patients from 25.2% to 10.7%. In-hospital mortality, the primary outcome, did not change, ICU admissions decreased, as did hospital length of stay (10 vs 8.4 days, though not statistically significant), all providing supportive evidence for relative safety of the new protocol.

Soares et al exemplify a nimble system that recognized planned strategies to be problematic, and then achieved rapid implementation of a new protocol across a four-hospital system. Changes in medical practice are typically much slower, with some studies suggesting this process may take a decade or more. Implementation science focuses on translating research evidence into clinical practice using strategies tailored to particular contexts. The current study harnessed important implementation principles to quickly translate evidence into practice using effective engagement and education of key stakeholders across specialties (eg, emergency medicine, hospitalists, critical care, and respiratory therapy), the identification of pathways that mitigated barriers, frequent re-evaluation of a rapidly evolving literature, and an open-mindedness to the value of change.⁶ As the pandemic continues, traditional research and implementation science are critical not only to define optimal treatments and management strategies, but also to learn how best to implement successful interventions in an accelerated manner.⁷

The authors reported no conflicts of interest.

Dr Hochberg is supported by a National Institutes of Health training grant (T32HL007534).

Early in the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, the World Health Organization issued guidance for coronavirus disease 2019 (COVID-19) management.¹ Based on a high intubation rate among 12 subjects with Middle Eastern respiratory syndrome, noninvasive ventilation (NIV) was discouraged.² While high-flow nasal oxygen (HFNO) was recognized as a reasonable strategy to avoid endotracheal intubation,¹ uncertainty regarding the potential of both therapies to aerosolize SARS-CoV-2 and reports of rapid, unexpected respiratory decompensations were deterrents to use.³ As hospitals prepared for a surge of patients, reports of SARS-CoV-2 transmission to healthcare personnel also emerged. Together, these issues led many institutions to recommend lower than usual thresholds for intubation. This well-intentioned guidance was based on limited historical data, a rapidly evolving literature that frequently appeared on preprint servers before peer review, or as anecdotes on social media.

As COVID-19 caseloads increased, clinicians were immediately faced with patients who rapidly reached the planned intubation threshold, but also looked very comfortable with minimal to no use of accessory muscles of respiration. In addition, the pace of respiratory decompensation among those who ultimately required intubation was slower than expected. Moreover, intensive care unit (ICU) capacity was stretched thin, raising concern for an imminent need for ventilator rationing. Lastly, the risk of SARS-CoV-2 transmission to healthcare workers appeared well-controlled with the use of personal protective equipment.⁴

In light of this accumulating experience, sites worldwide evolved quickly from their initial management strategies for COVID-19 respiratory failure. However, the deliberate process described by Soares et al in this issue of the Journal of Hospital Medicine is notable.⁵ Their transition towards the beginning of the pandemic from a conservative early intubation approach to a new strategy that encouraged use of NIV, HFNO, and self-proning is described. They were motivated by reports of good outcomes using these interventions, high mortality in intubated patients, and reassurance that aerosolization of respiratory secretions during NIV and HFNO was comparable to regular nasal cannula or face mask oxygen.³ The new protocol was defined and rapidly deployed over 4 days using multipronged communication from project and institutional leaders via in person and electronic means (email, Whatsapp, GoogleDrive). To facilitate implementation, COVID-19 patients requiring respiratory support were placed in dedicated units with bedside flowsheets for guidance. An immediate impact was demonstrated over the next 2 weeks by a significant decrease in use of mechanical ventilation in COVID-19 patients from 25.2% to 10.7%. In-hospital mortality, the primary outcome, did not change, ICU admissions decreased, as did hospital length of stay (10 vs 8.4 days, though not statistically significant), all providing supportive evidence for relative safety of the new protocol.

Soares et al exemplify a nimble system that recognized planned strategies to be problematic, and then achieved rapid implementation of a new protocol across a four-hospital system. Changes in medical practice are typically much slower, with some studies suggesting this process may take a decade or more. Implementation science focuses on translating research evidence into clinical practice using strategies tailored to particular contexts. The current study harnessed important implementation principles to quickly translate evidence into practice using effective engagement and education of key stakeholders across specialties (eg, emergency medicine, hospitalists, critical care, and respiratory therapy), the identification of pathways that mitigated barriers, frequent re-evaluation of a rapidly evolving literature, and an open-mindedness to the value of change.⁶ As the pandemic continues, traditional research and implementation science are critical not only to define optimal treatments and management strategies, but also to learn how best to implement successful interventions in an accelerated manner.⁷

The authors reported no conflicts of interest.

Dr Hochberg is supported by a National Institutes of Health training grant (T32HL007534).

Since 2012, when the Centers for Medicare & Medicaid Services (CMS) began linking financial penalties to hospitals with excessive readmissions for adult patients, researchers have questioned the extent to which pediatric readmissions can be used as a reliable quality measure. Compared with readmissions among adult patients, readmissions among pediatric patients are relatively uncommon. Furthermore, few (approximately 2%) qualify as potentially preventable, and pediatric readmission rates remain largely unchanged despite targeted attempts to prevent reutilization.^1,2 Nonetheless, state Medicaid agencies have continued to reduce reimbursement for hospitals based on available readmissions metrics, most commonly the Potentially Preventable Readmissions (PPR) algorithm.¹

In this issue of the Journal of Hospital Medicine, Auger et al³ performed a retrospective study to explore four existing metrics of pediatric hospital readmissions for their ability to identify preventable and unplanned readmissions. Investigators examined 30-day readmissions (n = 1,125) from 2014-2016 across multiple subspecialties, and classified readmissions by their preventability and unplanned status with use of a validated chart abstraction tool. Using the results of chart abstraction as the gold standard, investigators calculated the sensitivity and specificity, as well as estimated the positive and negative predictive values, of each readmissions metric. Auger and colleagues found that none of the four readmissions metrics could reliably assess preventability, and that only one metric reliably predicted unplanned hospital readmissions. Specifically, the commonly used PPR algorithm was estimated to have a positive predictive value of 13.0%-35.5% across a prevalence range of 10%-30%. This means that in a hospital where 10% of readmissions are truly preventable, the PPR will be wrong approximately 87% of the time. Tying payments to this metric is difficult to justify.

The authors highlighted the policy implications of the PPR falling short in its ability to identify preventable and unplanned pediatric readmissions. A good quality measure should be consistently reliable, and neither the PPR nor other measures studied meets this benchmark. Yet the findings lead to a broader conclusion: if most pediatric readmissions are not preventable, if there is no reliable way of measuring preventability, and if we have not demonstrated the ability to change patient trajectories away from reutilization, then perhaps the sun has set on using readmissions as a comprehensive quality measure for hospital-based care.

So how, then, should the hospital-to-home transition be evaluated? The paradigm of pediatric value of care is shifting to incorporate family-centered perspectives into consideration of quality measures.² There has to be a balance between healthcare costs and outcomes that affect families; measures should take into account issues such as patient and caregiver anxiety and time away from work.² Moreover, because social determinants of health and medical complexity strongly influence readmission rates,^4,5 focus should be placed on redirecting resources toward patients and families with significant medical, social, and financial needs as they transition home from the hospital. While measures of healthcare equity are currently lacking, the overall quality and equity of pediatric care transitions could be enhanced by looking beyond the narrow lens of readmission rates to incorporate actual needs assessments of families.

In summary, Auger and colleagues identified deficits in existing readmission metrics—but creating a solution that is meaningful to all stakeholders will be more complex than simply identifying a better metric. Family-centered quality metrics show promise in creating value in pediatric care within an equitable health system, but long-term evaluation of these metrics is necessary.

The authors have nothing to disclose.