Hospital Medicine

Through increased involvement with families and caregivers, community hospitals can deliver better healthcare to patients.^1,2 Furthermore, when patients bypass local hospitals and directly present to tertiary care, mortality for time-sensitive illnesses, such as sepsis, increases.³ Unfortunately, although critical access hospitals (CAHs) had an equivalent risk-adjusted mortality in 2002, they have failed to improve their performance at the same rate as that of larger hospitals and lag in quality metrics.^4,5

One potential contributor to the lagging performance may be the low uptake of the hospitalist model at these facilities.⁶ Although dedicated hospitalists have improved patient outcomes and decreased spending in large hospitals,^7-9 implementing the hospitalist medicine model on a smaller scale remains difficult. Approximately 1,300 CAHs provide necessary emergency room and inpatient services in the rural United States.¹⁰ Assuming 12-hour shifts and every-other-week assignments, providing continuous, on-location hospitalist coverage would require more than 10% of the total hospitalist workforce to cover less than 3% of all hospital admissions.^11-13

Telemedicine allows content experts, including hospitalists, to supervise patient care remotely. This provides a potential solution to the logistical challenges of supplying continuous hospitalist coverage to a remote facility with a low daily census. We hypothesized that providing continuous “virtual hospitalist” coverage through telemedicine could increase the ability of a CAH to care for patients locally, decreasing the number of transfers to tertiary care centers and improving patient and provider satisfaction. We aimed to create a 25% relative reduction in CAH Emergency Department (ED) patient encounters resulting in transfer to outside hospitals within 6 months.

This quality improvement project was exempt from Institutional Review Board review.

Setting

The University of Iowa Hospitals and Clinics (UIHC) is a 750-bed teaching hospital based in a suburban community in Eastern Iowa and the only tertiary care hospital in the state of Iowa. The UIHC Hospitalist group contains 44 staff physicians and covers more than 12 service lines (both faculty-only and resident-covered) at this facility.

Van Buren County Hospital (VBCH) is a 24-bed CAH offering emergency, internal medicine, and obstetrical services and located 80 miles southwest of UIHC. X-ray and CT scan services are available continuously, but ultrasound and magnetic resonance imaging services are available only 2-3 times per week. While tertiary care patients were transferred to UIHC, patients requiring specialty care but with less complex illnesses (eg, stable myocardial infarction) were referred to closer facilities.

Prior to implementation, coverage of the acute inpatient ward and the emergency room at VBCH was simultaneously provided by a single physician or advanced practice providers (APPs). When APPs provided coverage, a physician was required to be notified of any new admissions and was immediately available for medical emergencies. The VBCH providers worked alone in 48- to 72-hour continuous shifts as the sole coverage for both ED and inpatient units. It was frequently necessary to bring in outside providers through locum tenens agencies to fill gaps in the provider schedule. Both VBCH and UIHC used a shared electronic medical record (EMR), which was a key consideration in choosing VBCH as our pilot site. Providers at both institutions had access to identical patient information through the EMR, including radiology images, laboratory results, and provider notes.

Intervention Development and Implementation

A site visit by clinical and administrative project leads to VBCH identified three deficits that we could address through telemedicine: (1) The extended duration of VBCH shifts was detrimental to provider experience and retention; (2) Lack of local expertise in hospital medicine led to limited comfort in caring for patients with stable but medically complex conditions (eg, drug-resistant urinary tract infection); and (3) Patient transitions between VBCH and UIHC during acute care transfer were frustrating and led to negative experiences with providers and patients.

We developed a model to address these deficits using the minimum number of specialties and employees to facilitate rapid implementation. Although local care ED and inpatient care was provided by 3 APPS and a single physician provider, we mandated the coverage of all acute inpatients by the virtual hospitalists. This coverage included daily videoconference patient rounds, continuous pager coverage for new acute issues, and listing the virtual hospitalists as the attending of record for patient admissions. We scheduled contact times in the morning and afternoon to accelerate familiarity and comfort with the technology. We used a secure, Health Insurance Portability and Accountability Act of 1996 (HIPAA)-compliant platform for videoconferencing, accessible through personal computers or portable smart devices (Vidyo, VidyoInc, Hackensack, New Jersey). At VBCH, two tablet computers were provided to serve as portable platforms to use either in provider conference rooms or to be taken into patient rooms. Twice a day, at 8:45 am and 4:30 pm, virtual hospitalists, local providers, and nursing staff would videoconference and review the status and care plan for all admitted patients. In addition, virtual hospitalists performed a videoconference interview using the tablet computers with all patients on the morning following admission and at other times on an as-needed basis. We asked the virtual hospitalists to cover a minimum of 72 consecutive hours to maintain provider continuity. Local APPs documented the history, examination, and medical decision-making for billing purposes, which were cosigned by the virtual hospitalists. The virtual hospitalists also created separate notes documenting their discussions with local staff, interview and limited direct physical examination findings (eg, appearance of rashes), and medical decision making. Due to limitations of the EMR, local APPs wrote patient orders. All virtual hospitalists were credentialed by proxy at VBCH. We consulted with the UIHC legal team to ensure that virtual hospitalists would be protected under their existing malpractice insurance.

Outcome Measures

Outcome measures were divided into three categories: (1) clinical and utilization outcomes; (2) virtual hospitalist outcomes; and (3) satisfaction outcomes. The primary clinical outcome was the percentage of ED encounters resulting in transfer to a different acute care hospital. We also monitored alternative ED dispositions, including local inpatient admission. Additional clinical and utilization outcomes after ED admission included the mean daily inpatient census at VBCH and the case mix index (CMI). We selected the mean length of stay, the percentage of inpatients transferred to other hospitals, and the inpatient mortality as balance measures due to concerns of increasing the acuity of the inpatient wards beyond the comfort and expertise of local staff. Virtual hospitalist outcomes included the mean daily time commitment and the mean time commitment per patient. Virtual hospitalists self-reported their time commitments as part of their daily documentation. We chose these measures in anticipation of expanding this program to other institutions in the future. Satisfaction outcomes included a weekly survey to all VBCH physicians and nursing staff (Appendix 1), weekly group discussions with virtual hospitalists and CAH staff, and 3 interviews with patients and family members after discharge (Appendix 2).

Statistical Analysis

Baseline data collected over a period of 24 weeks were used to measure pre-implementation performance and trends at VBCH. The virtual hospitalist service was started on November 15, 2016, and the two weeks before and two weeks after this date were excluded from analysis as a transition period. To account for weekend variation, we reported data in consecutive 28-day blocks. We used Chi-square tests to compare proportional outcomes and Student’s t-tests for continuous variables. Statistical Process Control charts were used to evaluate for temporal trends in quantitative data.

Funding

Development of this project was funded through the University of Iowa Hospitalist group and the Signal Center for Health Innovations at UI Health Ventures. Virtual hospitalist clinical time was paid for by the CAH on a fractional basis of a traditional hospitalist based on projected patient volumes through analysis of baseline data. Patients were not directly billed for virtual hospitalist service but were charged for the services provided by CAH providers.

Clinical and Utilization Outcomes

During the 24-week baseline period, VBCH had 947 ED encounters and 176 combined acute inpatient and observation admissions. For the 24 weeks following the transition, there were 930 ED visits and 186 admissions. We observed a 36% (157/947 to 98/930, P < .001) decrease in ED encounters ending in patient transfer to another hospital (Figure). In parallel, VBCH ED visits leading to local admission increased by 62% of baseline (39/947 to 62/930, P = .014). There was no significant change in the fraction of ED encounters resulting in an observation stay (104/947 to 99/930, P = .814). Daily ED visits did not change after virtual hospitalist coverage began (5.64 to 5.54 visits/day, P = .734), but the percentage of ED visits ending in discharge to a nonmedical setting increased from 79.0% to 82.7% (748/947 to 769/930, P = .042).

The implementation did not have a significant impact on ward census or patient complexity (Table 1). Both CMI and mean length of stay did not change after starting the service. The study was underpowered to detect differences in rare events, including inpatient mortality and transfer after admission. Despite the decrease in transfers, inpatient census was unchanged. This coincides with a 17% decrease (196/947 to 160/930, P = .054) in the proportion of ED patients referred for admission either locally or at an outside hospital.

Virtual Hospitalist Outcomes

Satisfaction Outcomes

The staff at VBCH identified several benefits to the virtual hospitalist service. Survey responses (N = 18) were positive, with staff expressing specific gratitude for the additional education and training provided by the virtual hospitalists. On a Likert scale ranging from 1 (very poor) to 5 (excellent), the respondents gave high mean scores to the overall service experience (4.8) and the effectiveness of care delivered (4.9) but were more critical of the ability to keep patients locally (4.5) and the experience with transferring patients (3.9). We also collected free-text feedback from both patients and staff at VBCH (Table 2).

The virtual hospitalist service allowed a higher percentage of acute inpatients to receive care in their local hospital and was positively perceived by providers and patients. The per-patient time commitment by virtual hospitalists was similar to traditional hospitalist coverage¹⁴ and could scale to multiple simultaneous institutions.

Despite the increase in the proportion of patients admitted locally, neither the mean inpatient census nor the complexity of patients (as measured by CMI) increased. The increase in patients admitted locally was offset by a parallel increase in the number of ED patients discharged home. Although virtual hospitalists were available to consult on ED patients, this consultation was not mandatory unless the CAH provider felt that admission was indicated. It remains unclear whether the changes in ED disposition were due to direct intervention by virtual hospitalists, increasing local expertise with inpatient medicine, or unrelated local factors.

Although outside transfers directly from the ED dropped, there was a potential increase in acute inpatients transferred after admission that failed to reach statistical significance. We anticipated increased transfers after admission as a potential consequence of accepting more complex patients for CAH admission. Reasons for transfer included emergent transfers for medically unstable patients and scheduled transfer for subspecialist evaluation or testing. Despite the possible increase in delayed transfers, there was no significant change in CAH inpatient mortality, and the total fraction of combined ED and inpatients transferred decreased after the intervention.

Despite the benefits of keeping patients within their communities, 20%-60% of rural patients bypass their local facilities when seeking emergent care.¹⁵ Despite publicity on local media,¹⁶ we did not observe an increase in daily ED visits after implementation. Although some investigators have found that increasing the services offered decreases in rural bypass,¹⁷ others have found no or mixed effects.^18,19 Further investigations into the local factors contributing to rural bypass may yield important insights, and future implementations should not rely on rapid increases in patient volume to establish economic viability.

Although telemedicine has been applied to a variety of previous settings, to our knowledge, this marks the first collaboration between an academic medical center and a CAH to provide continuous hospitalist coverage. A previous model for pediatric inpatients showed a similar decrease in patients transferred to tertiary centers.²⁰ Virtual hospitalists differ from other adult telemedicine projects, which focused on subspecialty care or overnight coverage.²¹ The advantages of our model include the ability to proactively address deficits, even when local providers are unaware of changes to the standards of care. We believe that mandatory scheduled interactions decreased the barriers to communication and increased provider reassurance in telemedicine management of their patients. The scheduled interactions also provided additional training and development for CAH personnel, were well received by local staff, and may contribute to local provider job satisfaction, retention, and recruitment.

Past efforts to integrate academic hospitalists into CAHs improved quality metrics and provider satisfaction but were economically infeasible due to low patient volumes.²² In contrast, virtual providers can distribute their efforts across multiple areas, including covering additional CAHs, providing local patient care at their home facility, or completing academic projects. By combining two or more CAHs into a single provider, sufficient patient volume can be generated to dedicated personnel.

There were several limitations to this initial investigation:

• As a pilot between two specific institutions, modifications will be required to replicate in other CAHs or academic centers.
• Generating sufficient revenue to cover a full hospitalist salary will require adding additional responsibilities, either covering multiple CAHs simultaneously or combining virtual coverage with in-person responsibilities.
• The accuracy of the self-report remains unmeasured, and the impact of combining two or more CAHs may not be strictly additive. Attempts to supplement the self-reported time spent with additional information from the EMR and cell phone logs were complicated by the use of multiple platforms in parallel, interruptions in provider workflow, and provider multitasking.
• Due to the need for reliable local physical examinations and regulations on telehealth reimbursement, local APPs were necessary for this implementation. Although most of the CAHs have an on-site provider to provide ED coverage, CAHs with sufficient volume to necessitate separating ED and inpatient ward coverage may have difficulty supporting both APP and virtual hospitalist coverage, even on a fractional basis.
• This study was underpowered to detect rare events with significant consequences, including inpatient mortality and inpatient transfer. Although CMI suggests similar complexity in CAH patients, we have insufficient data to draw further comparisons on patient characteristics before and after the intervention.
• The analysis may be vulnerable to secular trends in the CAH patient population, as only 24 weeks of data were used as a baseline for comparison (although no significant seasonal variation was detected during that time). Extending the baseline data to include an additional 30 weeks ED encounters did not significantly alter our conclusions.
• Virtual hospitalists were dependent on physical examinations performed independently by local APPs.
• Although virtual providers were obligated to be available for videoconferencing within 60 minutes, more urgent medical decisions were sometimes made based on phone conferences between VBCH and the virtual hospitalist without video or direct patient assessment.
• We selected a CAH utilizing an identical instance of our EMR. Although this increased the ability of virtual hospitalists to split their time between virtual and local patient encounters, this limits our ability to spread this intervention beyond institutions already partnering with the UIHC.

We succeeded in reducing outside transfers at a CAH by implementing a sustainable virtual hospitalist service. This model allows patients to receive more of their care within their local communities and provides an improved inpatient experience. Next steps include expanding this service to other CAHs within our region, both to understand if this model is applicable beyond our initial site and to monitor for complications induced by scaling. If successful, virtual hospitalist coverage can provide a sustainable solution to providing the latest innovations in hospital medicine even to the most rural communities.

The authors thank Ray Brownsworth, CEO of Van Buren County Hospital, as well as all the providers and staff who worked with them to implement and improve their services. The authors also thank Pat Brophy, founder of The Signal Center for Health Innovation, for providing leadership, support, and resources for innovation.

Disclosures 

None of the authors have identified a conflict of interest in relation to this manuscript.

Funding

This project was funded through the University of Iowa Health Care and the Signal Center for Health Innovations at UI Health Ventures.

Compliance With Ethical Standards

This quality improvement project was exempt from Institutional Review Board review

Through increased involvement with families and caregivers, community hospitals can deliver better healthcare to patients.^1,2 Furthermore, when patients bypass local hospitals and directly present to tertiary care, mortality for time-sensitive illnesses, such as sepsis, increases.³ Unfortunately, although critical access hospitals (CAHs) had an equivalent risk-adjusted mortality in 2002, they have failed to improve their performance at the same rate as that of larger hospitals and lag in quality metrics.^4,5

One potential contributor to the lagging performance may be the low uptake of the hospitalist model at these facilities.⁶ Although dedicated hospitalists have improved patient outcomes and decreased spending in large hospitals,^7-9 implementing the hospitalist medicine model on a smaller scale remains difficult. Approximately 1,300 CAHs provide necessary emergency room and inpatient services in the rural United States.¹⁰ Assuming 12-hour shifts and every-other-week assignments, providing continuous, on-location hospitalist coverage would require more than 10% of the total hospitalist workforce to cover less than 3% of all hospital admissions.^11-13

Telemedicine allows content experts, including hospitalists, to supervise patient care remotely. This provides a potential solution to the logistical challenges of supplying continuous hospitalist coverage to a remote facility with a low daily census. We hypothesized that providing continuous “virtual hospitalist” coverage through telemedicine could increase the ability of a CAH to care for patients locally, decreasing the number of transfers to tertiary care centers and improving patient and provider satisfaction. We aimed to create a 25% relative reduction in CAH Emergency Department (ED) patient encounters resulting in transfer to outside hospitals within 6 months.

This quality improvement project was exempt from Institutional Review Board review.

Setting

The University of Iowa Hospitals and Clinics (UIHC) is a 750-bed teaching hospital based in a suburban community in Eastern Iowa and the only tertiary care hospital in the state of Iowa. The UIHC Hospitalist group contains 44 staff physicians and covers more than 12 service lines (both faculty-only and resident-covered) at this facility.

Van Buren County Hospital (VBCH) is a 24-bed CAH offering emergency, internal medicine, and obstetrical services and located 80 miles southwest of UIHC. X-ray and CT scan services are available continuously, but ultrasound and magnetic resonance imaging services are available only 2-3 times per week. While tertiary care patients were transferred to UIHC, patients requiring specialty care but with less complex illnesses (eg, stable myocardial infarction) were referred to closer facilities.

Prior to implementation, coverage of the acute inpatient ward and the emergency room at VBCH was simultaneously provided by a single physician or advanced practice providers (APPs). When APPs provided coverage, a physician was required to be notified of any new admissions and was immediately available for medical emergencies. The VBCH providers worked alone in 48- to 72-hour continuous shifts as the sole coverage for both ED and inpatient units. It was frequently necessary to bring in outside providers through locum tenens agencies to fill gaps in the provider schedule. Both VBCH and UIHC used a shared electronic medical record (EMR), which was a key consideration in choosing VBCH as our pilot site. Providers at both institutions had access to identical patient information through the EMR, including radiology images, laboratory results, and provider notes.

Intervention Development and Implementation

A site visit by clinical and administrative project leads to VBCH identified three deficits that we could address through telemedicine: (1) The extended duration of VBCH shifts was detrimental to provider experience and retention; (2) Lack of local expertise in hospital medicine led to limited comfort in caring for patients with stable but medically complex conditions (eg, drug-resistant urinary tract infection); and (3) Patient transitions between VBCH and UIHC during acute care transfer were frustrating and led to negative experiences with providers and patients.

We developed a model to address these deficits using the minimum number of specialties and employees to facilitate rapid implementation. Although local care ED and inpatient care was provided by 3 APPS and a single physician provider, we mandated the coverage of all acute inpatients by the virtual hospitalists. This coverage included daily videoconference patient rounds, continuous pager coverage for new acute issues, and listing the virtual hospitalists as the attending of record for patient admissions. We scheduled contact times in the morning and afternoon to accelerate familiarity and comfort with the technology. We used a secure, Health Insurance Portability and Accountability Act of 1996 (HIPAA)-compliant platform for videoconferencing, accessible through personal computers or portable smart devices (Vidyo, VidyoInc, Hackensack, New Jersey). At VBCH, two tablet computers were provided to serve as portable platforms to use either in provider conference rooms or to be taken into patient rooms. Twice a day, at 8:45 am and 4:30 pm, virtual hospitalists, local providers, and nursing staff would videoconference and review the status and care plan for all admitted patients. In addition, virtual hospitalists performed a videoconference interview using the tablet computers with all patients on the morning following admission and at other times on an as-needed basis. We asked the virtual hospitalists to cover a minimum of 72 consecutive hours to maintain provider continuity. Local APPs documented the history, examination, and medical decision-making for billing purposes, which were cosigned by the virtual hospitalists. The virtual hospitalists also created separate notes documenting their discussions with local staff, interview and limited direct physical examination findings (eg, appearance of rashes), and medical decision making. Due to limitations of the EMR, local APPs wrote patient orders. All virtual hospitalists were credentialed by proxy at VBCH. We consulted with the UIHC legal team to ensure that virtual hospitalists would be protected under their existing malpractice insurance.

Outcome Measures

Outcome measures were divided into three categories: (1) clinical and utilization outcomes; (2) virtual hospitalist outcomes; and (3) satisfaction outcomes. The primary clinical outcome was the percentage of ED encounters resulting in transfer to a different acute care hospital. We also monitored alternative ED dispositions, including local inpatient admission. Additional clinical and utilization outcomes after ED admission included the mean daily inpatient census at VBCH and the case mix index (CMI). We selected the mean length of stay, the percentage of inpatients transferred to other hospitals, and the inpatient mortality as balance measures due to concerns of increasing the acuity of the inpatient wards beyond the comfort and expertise of local staff. Virtual hospitalist outcomes included the mean daily time commitment and the mean time commitment per patient. Virtual hospitalists self-reported their time commitments as part of their daily documentation. We chose these measures in anticipation of expanding this program to other institutions in the future. Satisfaction outcomes included a weekly survey to all VBCH physicians and nursing staff (Appendix 1), weekly group discussions with virtual hospitalists and CAH staff, and 3 interviews with patients and family members after discharge (Appendix 2).

Statistical Analysis

Baseline data collected over a period of 24 weeks were used to measure pre-implementation performance and trends at VBCH. The virtual hospitalist service was started on November 15, 2016, and the two weeks before and two weeks after this date were excluded from analysis as a transition period. To account for weekend variation, we reported data in consecutive 28-day blocks. We used Chi-square tests to compare proportional outcomes and Student’s t-tests for continuous variables. Statistical Process Control charts were used to evaluate for temporal trends in quantitative data.

Funding

Development of this project was funded through the University of Iowa Hospitalist group and the Signal Center for Health Innovations at UI Health Ventures. Virtual hospitalist clinical time was paid for by the CAH on a fractional basis of a traditional hospitalist based on projected patient volumes through analysis of baseline data. Patients were not directly billed for virtual hospitalist service but were charged for the services provided by CAH providers.

Clinical and Utilization Outcomes

During the 24-week baseline period, VBCH had 947 ED encounters and 176 combined acute inpatient and observation admissions. For the 24 weeks following the transition, there were 930 ED visits and 186 admissions. We observed a 36% (157/947 to 98/930, P < .001) decrease in ED encounters ending in patient transfer to another hospital (Figure). In parallel, VBCH ED visits leading to local admission increased by 62% of baseline (39/947 to 62/930, P = .014). There was no significant change in the fraction of ED encounters resulting in an observation stay (104/947 to 99/930, P = .814). Daily ED visits did not change after virtual hospitalist coverage began (5.64 to 5.54 visits/day, P = .734), but the percentage of ED visits ending in discharge to a nonmedical setting increased from 79.0% to 82.7% (748/947 to 769/930, P = .042).

The implementation did not have a significant impact on ward census or patient complexity (Table 1). Both CMI and mean length of stay did not change after starting the service. The study was underpowered to detect differences in rare events, including inpatient mortality and transfer after admission. Despite the decrease in transfers, inpatient census was unchanged. This coincides with a 17% decrease (196/947 to 160/930, P = .054) in the proportion of ED patients referred for admission either locally or at an outside hospital.

Virtual Hospitalist Outcomes

Satisfaction Outcomes

The staff at VBCH identified several benefits to the virtual hospitalist service. Survey responses (N = 18) were positive, with staff expressing specific gratitude for the additional education and training provided by the virtual hospitalists. On a Likert scale ranging from 1 (very poor) to 5 (excellent), the respondents gave high mean scores to the overall service experience (4.8) and the effectiveness of care delivered (4.9) but were more critical of the ability to keep patients locally (4.5) and the experience with transferring patients (3.9). We also collected free-text feedback from both patients and staff at VBCH (Table 2).

The virtual hospitalist service allowed a higher percentage of acute inpatients to receive care in their local hospital and was positively perceived by providers and patients. The per-patient time commitment by virtual hospitalists was similar to traditional hospitalist coverage¹⁴ and could scale to multiple simultaneous institutions.

Despite the increase in the proportion of patients admitted locally, neither the mean inpatient census nor the complexity of patients (as measured by CMI) increased. The increase in patients admitted locally was offset by a parallel increase in the number of ED patients discharged home. Although virtual hospitalists were available to consult on ED patients, this consultation was not mandatory unless the CAH provider felt that admission was indicated. It remains unclear whether the changes in ED disposition were due to direct intervention by virtual hospitalists, increasing local expertise with inpatient medicine, or unrelated local factors.

Although outside transfers directly from the ED dropped, there was a potential increase in acute inpatients transferred after admission that failed to reach statistical significance. We anticipated increased transfers after admission as a potential consequence of accepting more complex patients for CAH admission. Reasons for transfer included emergent transfers for medically unstable patients and scheduled transfer for subspecialist evaluation or testing. Despite the possible increase in delayed transfers, there was no significant change in CAH inpatient mortality, and the total fraction of combined ED and inpatients transferred decreased after the intervention.

Despite the benefits of keeping patients within their communities, 20%-60% of rural patients bypass their local facilities when seeking emergent care.¹⁵ Despite publicity on local media,¹⁶ we did not observe an increase in daily ED visits after implementation. Although some investigators have found that increasing the services offered decreases in rural bypass,¹⁷ others have found no or mixed effects.^18,19 Further investigations into the local factors contributing to rural bypass may yield important insights, and future implementations should not rely on rapid increases in patient volume to establish economic viability.

Although telemedicine has been applied to a variety of previous settings, to our knowledge, this marks the first collaboration between an academic medical center and a CAH to provide continuous hospitalist coverage. A previous model for pediatric inpatients showed a similar decrease in patients transferred to tertiary centers.²⁰ Virtual hospitalists differ from other adult telemedicine projects, which focused on subspecialty care or overnight coverage.²¹ The advantages of our model include the ability to proactively address deficits, even when local providers are unaware of changes to the standards of care. We believe that mandatory scheduled interactions decreased the barriers to communication and increased provider reassurance in telemedicine management of their patients. The scheduled interactions also provided additional training and development for CAH personnel, were well received by local staff, and may contribute to local provider job satisfaction, retention, and recruitment.

Past efforts to integrate academic hospitalists into CAHs improved quality metrics and provider satisfaction but were economically infeasible due to low patient volumes.²² In contrast, virtual providers can distribute their efforts across multiple areas, including covering additional CAHs, providing local patient care at their home facility, or completing academic projects. By combining two or more CAHs into a single provider, sufficient patient volume can be generated to dedicated personnel.

There were several limitations to this initial investigation:

• As a pilot between two specific institutions, modifications will be required to replicate in other CAHs or academic centers.
• Generating sufficient revenue to cover a full hospitalist salary will require adding additional responsibilities, either covering multiple CAHs simultaneously or combining virtual coverage with in-person responsibilities.
• The accuracy of the self-report remains unmeasured, and the impact of combining two or more CAHs may not be strictly additive. Attempts to supplement the self-reported time spent with additional information from the EMR and cell phone logs were complicated by the use of multiple platforms in parallel, interruptions in provider workflow, and provider multitasking.
• Due to the need for reliable local physical examinations and regulations on telehealth reimbursement, local APPs were necessary for this implementation. Although most of the CAHs have an on-site provider to provide ED coverage, CAHs with sufficient volume to necessitate separating ED and inpatient ward coverage may have difficulty supporting both APP and virtual hospitalist coverage, even on a fractional basis.
• This study was underpowered to detect rare events with significant consequences, including inpatient mortality and inpatient transfer. Although CMI suggests similar complexity in CAH patients, we have insufficient data to draw further comparisons on patient characteristics before and after the intervention.
• The analysis may be vulnerable to secular trends in the CAH patient population, as only 24 weeks of data were used as a baseline for comparison (although no significant seasonal variation was detected during that time). Extending the baseline data to include an additional 30 weeks ED encounters did not significantly alter our conclusions.
• Virtual hospitalists were dependent on physical examinations performed independently by local APPs.
• Although virtual providers were obligated to be available for videoconferencing within 60 minutes, more urgent medical decisions were sometimes made based on phone conferences between VBCH and the virtual hospitalist without video or direct patient assessment.
• We selected a CAH utilizing an identical instance of our EMR. Although this increased the ability of virtual hospitalists to split their time between virtual and local patient encounters, this limits our ability to spread this intervention beyond institutions already partnering with the UIHC.

We succeeded in reducing outside transfers at a CAH by implementing a sustainable virtual hospitalist service. This model allows patients to receive more of their care within their local communities and provides an improved inpatient experience. Next steps include expanding this service to other CAHs within our region, both to understand if this model is applicable beyond our initial site and to monitor for complications induced by scaling. If successful, virtual hospitalist coverage can provide a sustainable solution to providing the latest innovations in hospital medicine even to the most rural communities.

The authors thank Ray Brownsworth, CEO of Van Buren County Hospital, as well as all the providers and staff who worked with them to implement and improve their services. The authors also thank Pat Brophy, founder of The Signal Center for Health Innovation, for providing leadership, support, and resources for innovation.

Disclosures 

None of the authors have identified a conflict of interest in relation to this manuscript.

Funding

This project was funded through the University of Iowa Health Care and the Signal Center for Health Innovations at UI Health Ventures.

Compliance With Ethical Standards

This quality improvement project was exempt from Institutional Review Board review

Through increased involvement with families and caregivers, community hospitals can deliver better healthcare to patients.^1,2 Furthermore, when patients bypass local hospitals and directly present to tertiary care, mortality for time-sensitive illnesses, such as sepsis, increases.³ Unfortunately, although critical access hospitals (CAHs) had an equivalent risk-adjusted mortality in 2002, they have failed to improve their performance at the same rate as that of larger hospitals and lag in quality metrics.^4,5

One potential contributor to the lagging performance may be the low uptake of the hospitalist model at these facilities.⁶ Although dedicated hospitalists have improved patient outcomes and decreased spending in large hospitals,^7-9 implementing the hospitalist medicine model on a smaller scale remains difficult. Approximately 1,300 CAHs provide necessary emergency room and inpatient services in the rural United States.¹⁰ Assuming 12-hour shifts and every-other-week assignments, providing continuous, on-location hospitalist coverage would require more than 10% of the total hospitalist workforce to cover less than 3% of all hospital admissions.^11-13

Telemedicine allows content experts, including hospitalists, to supervise patient care remotely. This provides a potential solution to the logistical challenges of supplying continuous hospitalist coverage to a remote facility with a low daily census. We hypothesized that providing continuous “virtual hospitalist” coverage through telemedicine could increase the ability of a CAH to care for patients locally, decreasing the number of transfers to tertiary care centers and improving patient and provider satisfaction. We aimed to create a 25% relative reduction in CAH Emergency Department (ED) patient encounters resulting in transfer to outside hospitals within 6 months.

This quality improvement project was exempt from Institutional Review Board review.

Setting

The University of Iowa Hospitals and Clinics (UIHC) is a 750-bed teaching hospital based in a suburban community in Eastern Iowa and the only tertiary care hospital in the state of Iowa. The UIHC Hospitalist group contains 44 staff physicians and covers more than 12 service lines (both faculty-only and resident-covered) at this facility.

Van Buren County Hospital (VBCH) is a 24-bed CAH offering emergency, internal medicine, and obstetrical services and located 80 miles southwest of UIHC. X-ray and CT scan services are available continuously, but ultrasound and magnetic resonance imaging services are available only 2-3 times per week. While tertiary care patients were transferred to UIHC, patients requiring specialty care but with less complex illnesses (eg, stable myocardial infarction) were referred to closer facilities.

Prior to implementation, coverage of the acute inpatient ward and the emergency room at VBCH was simultaneously provided by a single physician or advanced practice providers (APPs). When APPs provided coverage, a physician was required to be notified of any new admissions and was immediately available for medical emergencies. The VBCH providers worked alone in 48- to 72-hour continuous shifts as the sole coverage for both ED and inpatient units. It was frequently necessary to bring in outside providers through locum tenens agencies to fill gaps in the provider schedule. Both VBCH and UIHC used a shared electronic medical record (EMR), which was a key consideration in choosing VBCH as our pilot site. Providers at both institutions had access to identical patient information through the EMR, including radiology images, laboratory results, and provider notes.

Intervention Development and Implementation

A site visit by clinical and administrative project leads to VBCH identified three deficits that we could address through telemedicine: (1) The extended duration of VBCH shifts was detrimental to provider experience and retention; (2) Lack of local expertise in hospital medicine led to limited comfort in caring for patients with stable but medically complex conditions (eg, drug-resistant urinary tract infection); and (3) Patient transitions between VBCH and UIHC during acute care transfer were frustrating and led to negative experiences with providers and patients.

We developed a model to address these deficits using the minimum number of specialties and employees to facilitate rapid implementation. Although local care ED and inpatient care was provided by 3 APPS and a single physician provider, we mandated the coverage of all acute inpatients by the virtual hospitalists. This coverage included daily videoconference patient rounds, continuous pager coverage for new acute issues, and listing the virtual hospitalists as the attending of record for patient admissions. We scheduled contact times in the morning and afternoon to accelerate familiarity and comfort with the technology. We used a secure, Health Insurance Portability and Accountability Act of 1996 (HIPAA)-compliant platform for videoconferencing, accessible through personal computers or portable smart devices (Vidyo, VidyoInc, Hackensack, New Jersey). At VBCH, two tablet computers were provided to serve as portable platforms to use either in provider conference rooms or to be taken into patient rooms. Twice a day, at 8:45 am and 4:30 pm, virtual hospitalists, local providers, and nursing staff would videoconference and review the status and care plan for all admitted patients. In addition, virtual hospitalists performed a videoconference interview using the tablet computers with all patients on the morning following admission and at other times on an as-needed basis. We asked the virtual hospitalists to cover a minimum of 72 consecutive hours to maintain provider continuity. Local APPs documented the history, examination, and medical decision-making for billing purposes, which were cosigned by the virtual hospitalists. The virtual hospitalists also created separate notes documenting their discussions with local staff, interview and limited direct physical examination findings (eg, appearance of rashes), and medical decision making. Due to limitations of the EMR, local APPs wrote patient orders. All virtual hospitalists were credentialed by proxy at VBCH. We consulted with the UIHC legal team to ensure that virtual hospitalists would be protected under their existing malpractice insurance.

Outcome Measures

Outcome measures were divided into three categories: (1) clinical and utilization outcomes; (2) virtual hospitalist outcomes; and (3) satisfaction outcomes. The primary clinical outcome was the percentage of ED encounters resulting in transfer to a different acute care hospital. We also monitored alternative ED dispositions, including local inpatient admission. Additional clinical and utilization outcomes after ED admission included the mean daily inpatient census at VBCH and the case mix index (CMI). We selected the mean length of stay, the percentage of inpatients transferred to other hospitals, and the inpatient mortality as balance measures due to concerns of increasing the acuity of the inpatient wards beyond the comfort and expertise of local staff. Virtual hospitalist outcomes included the mean daily time commitment and the mean time commitment per patient. Virtual hospitalists self-reported their time commitments as part of their daily documentation. We chose these measures in anticipation of expanding this program to other institutions in the future. Satisfaction outcomes included a weekly survey to all VBCH physicians and nursing staff (Appendix 1), weekly group discussions with virtual hospitalists and CAH staff, and 3 interviews with patients and family members after discharge (Appendix 2).

Statistical Analysis

Baseline data collected over a period of 24 weeks were used to measure pre-implementation performance and trends at VBCH. The virtual hospitalist service was started on November 15, 2016, and the two weeks before and two weeks after this date were excluded from analysis as a transition period. To account for weekend variation, we reported data in consecutive 28-day blocks. We used Chi-square tests to compare proportional outcomes and Student’s t-tests for continuous variables. Statistical Process Control charts were used to evaluate for temporal trends in quantitative data.

Funding

Development of this project was funded through the University of Iowa Hospitalist group and the Signal Center for Health Innovations at UI Health Ventures. Virtual hospitalist clinical time was paid for by the CAH on a fractional basis of a traditional hospitalist based on projected patient volumes through analysis of baseline data. Patients were not directly billed for virtual hospitalist service but were charged for the services provided by CAH providers.

Clinical and Utilization Outcomes

During the 24-week baseline period, VBCH had 947 ED encounters and 176 combined acute inpatient and observation admissions. For the 24 weeks following the transition, there were 930 ED visits and 186 admissions. We observed a 36% (157/947 to 98/930, P < .001) decrease in ED encounters ending in patient transfer to another hospital (Figure). In parallel, VBCH ED visits leading to local admission increased by 62% of baseline (39/947 to 62/930, P = .014). There was no significant change in the fraction of ED encounters resulting in an observation stay (104/947 to 99/930, P = .814). Daily ED visits did not change after virtual hospitalist coverage began (5.64 to 5.54 visits/day, P = .734), but the percentage of ED visits ending in discharge to a nonmedical setting increased from 79.0% to 82.7% (748/947 to 769/930, P = .042).

The implementation did not have a significant impact on ward census or patient complexity (Table 1). Both CMI and mean length of stay did not change after starting the service. The study was underpowered to detect differences in rare events, including inpatient mortality and transfer after admission. Despite the decrease in transfers, inpatient census was unchanged. This coincides with a 17% decrease (196/947 to 160/930, P = .054) in the proportion of ED patients referred for admission either locally or at an outside hospital.

Virtual Hospitalist Outcomes

Satisfaction Outcomes

The staff at VBCH identified several benefits to the virtual hospitalist service. Survey responses (N = 18) were positive, with staff expressing specific gratitude for the additional education and training provided by the virtual hospitalists. On a Likert scale ranging from 1 (very poor) to 5 (excellent), the respondents gave high mean scores to the overall service experience (4.8) and the effectiveness of care delivered (4.9) but were more critical of the ability to keep patients locally (4.5) and the experience with transferring patients (3.9). We also collected free-text feedback from both patients and staff at VBCH (Table 2).

The virtual hospitalist service allowed a higher percentage of acute inpatients to receive care in their local hospital and was positively perceived by providers and patients. The per-patient time commitment by virtual hospitalists was similar to traditional hospitalist coverage¹⁴ and could scale to multiple simultaneous institutions.

Despite the increase in the proportion of patients admitted locally, neither the mean inpatient census nor the complexity of patients (as measured by CMI) increased. The increase in patients admitted locally was offset by a parallel increase in the number of ED patients discharged home. Although virtual hospitalists were available to consult on ED patients, this consultation was not mandatory unless the CAH provider felt that admission was indicated. It remains unclear whether the changes in ED disposition were due to direct intervention by virtual hospitalists, increasing local expertise with inpatient medicine, or unrelated local factors.

Although outside transfers directly from the ED dropped, there was a potential increase in acute inpatients transferred after admission that failed to reach statistical significance. We anticipated increased transfers after admission as a potential consequence of accepting more complex patients for CAH admission. Reasons for transfer included emergent transfers for medically unstable patients and scheduled transfer for subspecialist evaluation or testing. Despite the possible increase in delayed transfers, there was no significant change in CAH inpatient mortality, and the total fraction of combined ED and inpatients transferred decreased after the intervention.

Despite the benefits of keeping patients within their communities, 20%-60% of rural patients bypass their local facilities when seeking emergent care.¹⁵ Despite publicity on local media,¹⁶ we did not observe an increase in daily ED visits after implementation. Although some investigators have found that increasing the services offered decreases in rural bypass,¹⁷ others have found no or mixed effects.^18,19 Further investigations into the local factors contributing to rural bypass may yield important insights, and future implementations should not rely on rapid increases in patient volume to establish economic viability.

Although telemedicine has been applied to a variety of previous settings, to our knowledge, this marks the first collaboration between an academic medical center and a CAH to provide continuous hospitalist coverage. A previous model for pediatric inpatients showed a similar decrease in patients transferred to tertiary centers.²⁰ Virtual hospitalists differ from other adult telemedicine projects, which focused on subspecialty care or overnight coverage.²¹ The advantages of our model include the ability to proactively address deficits, even when local providers are unaware of changes to the standards of care. We believe that mandatory scheduled interactions decreased the barriers to communication and increased provider reassurance in telemedicine management of their patients. The scheduled interactions also provided additional training and development for CAH personnel, were well received by local staff, and may contribute to local provider job satisfaction, retention, and recruitment.

Past efforts to integrate academic hospitalists into CAHs improved quality metrics and provider satisfaction but were economically infeasible due to low patient volumes.²² In contrast, virtual providers can distribute their efforts across multiple areas, including covering additional CAHs, providing local patient care at their home facility, or completing academic projects. By combining two or more CAHs into a single provider, sufficient patient volume can be generated to dedicated personnel.

There were several limitations to this initial investigation:

• As a pilot between two specific institutions, modifications will be required to replicate in other CAHs or academic centers.
• Generating sufficient revenue to cover a full hospitalist salary will require adding additional responsibilities, either covering multiple CAHs simultaneously or combining virtual coverage with in-person responsibilities.
• The accuracy of the self-report remains unmeasured, and the impact of combining two or more CAHs may not be strictly additive. Attempts to supplement the self-reported time spent with additional information from the EMR and cell phone logs were complicated by the use of multiple platforms in parallel, interruptions in provider workflow, and provider multitasking.
• Due to the need for reliable local physical examinations and regulations on telehealth reimbursement, local APPs were necessary for this implementation. Although most of the CAHs have an on-site provider to provide ED coverage, CAHs with sufficient volume to necessitate separating ED and inpatient ward coverage may have difficulty supporting both APP and virtual hospitalist coverage, even on a fractional basis.
• This study was underpowered to detect rare events with significant consequences, including inpatient mortality and inpatient transfer. Although CMI suggests similar complexity in CAH patients, we have insufficient data to draw further comparisons on patient characteristics before and after the intervention.
• The analysis may be vulnerable to secular trends in the CAH patient population, as only 24 weeks of data were used as a baseline for comparison (although no significant seasonal variation was detected during that time). Extending the baseline data to include an additional 30 weeks ED encounters did not significantly alter our conclusions.
• Virtual hospitalists were dependent on physical examinations performed independently by local APPs.
• Although virtual providers were obligated to be available for videoconferencing within 60 minutes, more urgent medical decisions were sometimes made based on phone conferences between VBCH and the virtual hospitalist without video or direct patient assessment.
• We selected a CAH utilizing an identical instance of our EMR. Although this increased the ability of virtual hospitalists to split their time between virtual and local patient encounters, this limits our ability to spread this intervention beyond institutions already partnering with the UIHC.

We succeeded in reducing outside transfers at a CAH by implementing a sustainable virtual hospitalist service. This model allows patients to receive more of their care within their local communities and provides an improved inpatient experience. Next steps include expanding this service to other CAHs within our region, both to understand if this model is applicable beyond our initial site and to monitor for complications induced by scaling. If successful, virtual hospitalist coverage can provide a sustainable solution to providing the latest innovations in hospital medicine even to the most rural communities.

The authors thank Ray Brownsworth, CEO of Van Buren County Hospital, as well as all the providers and staff who worked with them to implement and improve their services. The authors also thank Pat Brophy, founder of The Signal Center for Health Innovation, for providing leadership, support, and resources for innovation.

Disclosures 

None of the authors have identified a conflict of interest in relation to this manuscript.

Funding

This project was funded through the University of Iowa Health Care and the Signal Center for Health Innovations at UI Health Ventures.

Compliance With Ethical Standards

This quality improvement project was exempt from Institutional Review Board review

Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.^1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.⁵ Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.⁵

However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.⁵ Meanwhile, CAD is a clear risk factor for perioperative cardiac events.^6-8

Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.⁵ To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.

Surgical Cohort

The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.⁹ We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.

Prediction of Dyslipidemia

The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.¹⁰ Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.

Prediction of Obstructive CAD

To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.¹¹ The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.

Prediction of Perioperative Event Risk

To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.¹² All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.⁵

Prediction of Test Accuracy

We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.¹³ (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.¹⁴) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.¹⁵ Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).

The median predicted probability of obstructive CAD was 5.1% (IQR: 1.8%-13.9%). Among patients with a predicted risk of a perioperative event of 1% or greater, the median probability of obstructive CAD was 18.1% (IQR: 9.6%-32.3%). The correlation between the predicted probabilities of CAD and a perioperative event was low (0.32), but highly statistically significant (P < .001).

Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.

DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.

In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.

Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.

However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.

A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.⁵ Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.

Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.

Acknowledgements

The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.^1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.⁵ Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.⁵

However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.⁵ Meanwhile, CAD is a clear risk factor for perioperative cardiac events.^6-8

Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.⁵ To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.

Surgical Cohort

The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.⁹ We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.

Prediction of Dyslipidemia

The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.¹⁰ Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.

Prediction of Obstructive CAD

To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.¹¹ The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.

Prediction of Perioperative Event Risk

To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.¹² All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.⁵

Prediction of Test Accuracy

We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.¹³ (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.¹⁴) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.¹⁵ Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).

The median predicted probability of obstructive CAD was 5.1% (IQR: 1.8%-13.9%). Among patients with a predicted risk of a perioperative event of 1% or greater, the median probability of obstructive CAD was 18.1% (IQR: 9.6%-32.3%). The correlation between the predicted probabilities of CAD and a perioperative event was low (0.32), but highly statistically significant (P < .001).

Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.

DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.

In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.

Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.

However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.

A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.⁵ Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.

Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.

Acknowledgements

The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.^1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.⁵ Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.⁵

However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.⁵ Meanwhile, CAD is a clear risk factor for perioperative cardiac events.^6-8

Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.⁵ To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.

Surgical Cohort

The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.⁹ We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.

Prediction of Dyslipidemia

The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.¹⁰ Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.

Prediction of Obstructive CAD

To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.¹¹ The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.

Prediction of Perioperative Event Risk

To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.¹² All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.⁵

Prediction of Test Accuracy

We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.¹³ (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.¹⁴) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.¹⁵ Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).

The median predicted probability of obstructive CAD was 5.1% (IQR: 1.8%-13.9%). Among patients with a predicted risk of a perioperative event of 1% or greater, the median probability of obstructive CAD was 18.1% (IQR: 9.6%-32.3%). The correlation between the predicted probabilities of CAD and a perioperative event was low (0.32), but highly statistically significant (P < .001).

Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.

DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.

In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.

Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.

However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.

A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.⁵ Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.

Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.

Acknowledgements

The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

While the use of pediatric readmission rates as a quality metric remains controversial, pediatric hospital-to-home transitions need improvement.¹ As many as a third of pediatric readmissions are preventable,² but the multifactorial and complex nature of factors that contribute to pediatric readmissions presents a challenge in tackling readmission. Several factors are associated with increased risk of readmission; these factors include both clinical and sociodemographic characteristics;³ however, we still have much to learn. Further, the only large trial of an intervention to prevent pediatric readmissions across all comers (nontargeted) was unsuccessful in decreasing reutilization.⁴ By contrast, various studies have succeeded in reducing readmission and/or emergency department revisit rates associated with inpatient interventions in select populations.⁵ Currently, however, no standardized or validated pediatric risk prediction tool can reliably identify the high-risk patients who may benefit from interventions. In the Journal of Hospital Medicine, Brittan and colleagues add to the literature base exploring the factors associated with an increased 30-day readmission risk by trialing an electronic health record (EHR)-based tool composed of three components: presence of home health, polypharmacy in the form of ≥6 medications, and presence of a caregiver who prefers a language other than English.⁶

This brief report contributes significantly to the literature. First, the presence of a tool embedded within the pediatric EHR and readily accessible at the point of clinical care is novel. Study authors purposefully chose components easily extractable from the EHR which update automatically. This infrastructure generates an automated score that is easily accessible to clinicians in real-time. Second, the transparency of the tool is notable given its display via the EHR’s “Discharge Readiness Report,” where a clinician can view not only the total composite score (1 point for each component) but also the specific components for which a point was allocated. Although a composite score in and of itself is potentially helpful, understanding specific factors that contribute to a patient’s increased risk of readmission allows for better targeting of interventions. For example, in Brittan’s simple, three-component model, the presence of polypharmacy might trigger a pharmacist to meet with the family prior to discharge to discuss indications for and how to properly administer medications. Finally, a multidisciplinary team composed of clinicians, nurse-family educators, case managers, social workers, and informatics experts developed and implemented this tool. Although the roll-out and longitudinal use of this tool is not described, the engagement of these multiple provider-types is likely to increase successful roll-out and utilization of the tool.

Unfortunately, the utility of this tool in predicting readmission is limited as evidenced by its low c-statistic. This limitation may be due to several reasons. The tool was not originally built as a tool to predict readmissions but rather an instrument to identify complex discharge care as part of a quality improvement initiative to improve discharge processes. Given the questions about readmission risk prediction, the authors explored the potential for the tool to predict readmission risk. The authors acknowledge that the tool excluded many known readmission risk factors based upon inconsistent documentation within the EHR and the desire to emphasize only modifiable factors. Thus, variables, including prior hospitalization which is a well-documented risk factor for readmissions (but not modifiable) and social determinants of health (which are not consistently documented), were excluded. Additionally, the included variable of “language preference” may have been a considerably broad characteristic. Limited English proficiency has been increasingly recognized as a construct placing patients at higher risk for adverse outcomes. However, caregivers with high English proficiency also exhibit varying degrees of health literacy. The inclusion of health literacy may be additive to a readmission risk prediction tool. Finally, the outcome is not well-described with regard to identification of “unplanned” events. Thus, their outcome measure may have included planned admissions for which the readmission risk prediction tool would be irrelevant.

In summary, Brittan and colleagues engaged a multidisciplinary group of providers to address discharge planning processes and leveraged the EHR to support their efforts in the form of a brief screening tool. Although this tool was not predictive of hospital readmissions, it highlights the opportunity to better utilize the EHR to gather meaningful, real-time data and subsequently use this information to positively impact our clinical care and allocation of resources. The tool should serve as a stepping stone to building a more extensive tool with inclusion of other known and potential readmission risk factors, thus resulting in a clinically relevant readmission risk prediction tool.

Disclosures

The authors have nothing to disclose.

While the use of pediatric readmission rates as a quality metric remains controversial, pediatric hospital-to-home transitions need improvement.¹ As many as a third of pediatric readmissions are preventable,² but the multifactorial and complex nature of factors that contribute to pediatric readmissions presents a challenge in tackling readmission. Several factors are associated with increased risk of readmission; these factors include both clinical and sociodemographic characteristics;³ however, we still have much to learn. Further, the only large trial of an intervention to prevent pediatric readmissions across all comers (nontargeted) was unsuccessful in decreasing reutilization.⁴ By contrast, various studies have succeeded in reducing readmission and/or emergency department revisit rates associated with inpatient interventions in select populations.⁵ Currently, however, no standardized or validated pediatric risk prediction tool can reliably identify the high-risk patients who may benefit from interventions. In the Journal of Hospital Medicine, Brittan and colleagues add to the literature base exploring the factors associated with an increased 30-day readmission risk by trialing an electronic health record (EHR)-based tool composed of three components: presence of home health, polypharmacy in the form of ≥6 medications, and presence of a caregiver who prefers a language other than English.⁶

This brief report contributes significantly to the literature. First, the presence of a tool embedded within the pediatric EHR and readily accessible at the point of clinical care is novel. Study authors purposefully chose components easily extractable from the EHR which update automatically. This infrastructure generates an automated score that is easily accessible to clinicians in real-time. Second, the transparency of the tool is notable given its display via the EHR’s “Discharge Readiness Report,” where a clinician can view not only the total composite score (1 point for each component) but also the specific components for which a point was allocated. Although a composite score in and of itself is potentially helpful, understanding specific factors that contribute to a patient’s increased risk of readmission allows for better targeting of interventions. For example, in Brittan’s simple, three-component model, the presence of polypharmacy might trigger a pharmacist to meet with the family prior to discharge to discuss indications for and how to properly administer medications. Finally, a multidisciplinary team composed of clinicians, nurse-family educators, case managers, social workers, and informatics experts developed and implemented this tool. Although the roll-out and longitudinal use of this tool is not described, the engagement of these multiple provider-types is likely to increase successful roll-out and utilization of the tool.

Unfortunately, the utility of this tool in predicting readmission is limited as evidenced by its low c-statistic. This limitation may be due to several reasons. The tool was not originally built as a tool to predict readmissions but rather an instrument to identify complex discharge care as part of a quality improvement initiative to improve discharge processes. Given the questions about readmission risk prediction, the authors explored the potential for the tool to predict readmission risk. The authors acknowledge that the tool excluded many known readmission risk factors based upon inconsistent documentation within the EHR and the desire to emphasize only modifiable factors. Thus, variables, including prior hospitalization which is a well-documented risk factor for readmissions (but not modifiable) and social determinants of health (which are not consistently documented), were excluded. Additionally, the included variable of “language preference” may have been a considerably broad characteristic. Limited English proficiency has been increasingly recognized as a construct placing patients at higher risk for adverse outcomes. However, caregivers with high English proficiency also exhibit varying degrees of health literacy. The inclusion of health literacy may be additive to a readmission risk prediction tool. Finally, the outcome is not well-described with regard to identification of “unplanned” events. Thus, their outcome measure may have included planned admissions for which the readmission risk prediction tool would be irrelevant.

In summary, Brittan and colleagues engaged a multidisciplinary group of providers to address discharge planning processes and leveraged the EHR to support their efforts in the form of a brief screening tool. Although this tool was not predictive of hospital readmissions, it highlights the opportunity to better utilize the EHR to gather meaningful, real-time data and subsequently use this information to positively impact our clinical care and allocation of resources. The tool should serve as a stepping stone to building a more extensive tool with inclusion of other known and potential readmission risk factors, thus resulting in a clinically relevant readmission risk prediction tool.

Disclosures

The authors have nothing to disclose.

While the use of pediatric readmission rates as a quality metric remains controversial, pediatric hospital-to-home transitions need improvement.¹ As many as a third of pediatric readmissions are preventable,² but the multifactorial and complex nature of factors that contribute to pediatric readmissions presents a challenge in tackling readmission. Several factors are associated with increased risk of readmission; these factors include both clinical and sociodemographic characteristics;³ however, we still have much to learn. Further, the only large trial of an intervention to prevent pediatric readmissions across all comers (nontargeted) was unsuccessful in decreasing reutilization.⁴ By contrast, various studies have succeeded in reducing readmission and/or emergency department revisit rates associated with inpatient interventions in select populations.⁵ Currently, however, no standardized or validated pediatric risk prediction tool can reliably identify the high-risk patients who may benefit from interventions. In the Journal of Hospital Medicine, Brittan and colleagues add to the literature base exploring the factors associated with an increased 30-day readmission risk by trialing an electronic health record (EHR)-based tool composed of three components: presence of home health, polypharmacy in the form of ≥6 medications, and presence of a caregiver who prefers a language other than English.⁶

This brief report contributes significantly to the literature. First, the presence of a tool embedded within the pediatric EHR and readily accessible at the point of clinical care is novel. Study authors purposefully chose components easily extractable from the EHR which update automatically. This infrastructure generates an automated score that is easily accessible to clinicians in real-time. Second, the transparency of the tool is notable given its display via the EHR’s “Discharge Readiness Report,” where a clinician can view not only the total composite score (1 point for each component) but also the specific components for which a point was allocated. Although a composite score in and of itself is potentially helpful, understanding specific factors that contribute to a patient’s increased risk of readmission allows for better targeting of interventions. For example, in Brittan’s simple, three-component model, the presence of polypharmacy might trigger a pharmacist to meet with the family prior to discharge to discuss indications for and how to properly administer medications. Finally, a multidisciplinary team composed of clinicians, nurse-family educators, case managers, social workers, and informatics experts developed and implemented this tool. Although the roll-out and longitudinal use of this tool is not described, the engagement of these multiple provider-types is likely to increase successful roll-out and utilization of the tool.

Unfortunately, the utility of this tool in predicting readmission is limited as evidenced by its low c-statistic. This limitation may be due to several reasons. The tool was not originally built as a tool to predict readmissions but rather an instrument to identify complex discharge care as part of a quality improvement initiative to improve discharge processes. Given the questions about readmission risk prediction, the authors explored the potential for the tool to predict readmission risk. The authors acknowledge that the tool excluded many known readmission risk factors based upon inconsistent documentation within the EHR and the desire to emphasize only modifiable factors. Thus, variables, including prior hospitalization which is a well-documented risk factor for readmissions (but not modifiable) and social determinants of health (which are not consistently documented), were excluded. Additionally, the included variable of “language preference” may have been a considerably broad characteristic. Limited English proficiency has been increasingly recognized as a construct placing patients at higher risk for adverse outcomes. However, caregivers with high English proficiency also exhibit varying degrees of health literacy. The inclusion of health literacy may be additive to a readmission risk prediction tool. Finally, the outcome is not well-described with regard to identification of “unplanned” events. Thus, their outcome measure may have included planned admissions for which the readmission risk prediction tool would be irrelevant.

In summary, Brittan and colleagues engaged a multidisciplinary group of providers to address discharge planning processes and leveraged the EHR to support their efforts in the form of a brief screening tool. Although this tool was not predictive of hospital readmissions, it highlights the opportunity to better utilize the EHR to gather meaningful, real-time data and subsequently use this information to positively impact our clinical care and allocation of resources. The tool should serve as a stepping stone to building a more extensive tool with inclusion of other known and potential readmission risk factors, thus resulting in a clinically relevant readmission risk prediction tool.

Disclosures

The authors have nothing to disclose.

As hospitalized children become more medically complex, hospital-to-home care transitions will become increasingly challenging. During a quality improvement (QI) initiative, we developed an electronic tool to improve the quality of our hospital discharge process.

We modeled the concept of a paper-based Early Screen for Discharge Planning – Child Version, which identifies children with multiple medical conditions, home nursing-care needs, tube feedings, presence of intravenous lines or drains, or post hospital care that requires coordination.¹We opted for an electronic tool to automate screening and increase visibilty of patients’ care transitional needs via the electronic health record (EHR).

The tool was designed by our QI team to address weaknesses in our discharge process (eg, discharge instructions that are not translated appropriately) and causes of preventable readmission at our institution (eg, discharge teaching, home care, and medication-related problems).^2,3 The tool’s selected components comprised those that might complicate or delay discharge care and included indicators of home health, polypharmacy, and caregiver language preference. Additional features were considered but withheld from the initial tool for several reasons noted in the methods.

We describe the development and implementation of this electronic tool. Given the paucity of pediatric risk models, we conducted an analysis of the tool’s potential to predict readmissions. We anticipated good predictive performance because the tool includes measures previously associated with readmission (eg, technology dependence, polypharmacy, and language barrier).^4-7 If successful in distinguishing readmission, this embedded discharge planning tool could also serve as a pediatric readmission risk score.

Setting

This work was conducted at the Children’s Hospital Colorado as part of a national QI collaborative. The hospital’s EHR is Epic (Verona, Wisconsin). The project was approved as QI by the Children’s Hospital Organizational Research Risk and Quality Improvement Review Panel, precluding review from the Colorado Multiple Institutional Review Board.

Tool Design, Implementation, and Use

A team of clinicians, nurse–family educators, case managers, social workers, and informatics experts helped design the instrument between 2014 and 2015. In addition to the selected features (number of discharge medications, presence of home health, and language preference), we considered adding the number of consulting specialists but had previously improved our process for scheduling follow-up appointments. Diagnoses were not systematically or discretely documented to be reliably extracted in real time. We excluded known readmission predictor variables (such as length of stay [LOS] and prior hospitalizations) from the initial model to maintain emphasis on the modifiable discharge processes. Additional considerations, such as health literacy and social determinants, were not systematically measured to be operationally usable.

To generate the score, the clinical documentation of home-health orders categorizes children with home care. Each home-care equipment or service category is documented in separate flowsheet rows, allowing for identification of distinct categories (Table). Total parenteral nutrition, intravenous medications, and durable medical equipment and supplies are counted as home care. The number of discharge medications is approximated by inpatient medication orders and finalized as the number of discharge medication orders. The medications include new, historic, and as-needed medications (if included among discharge medication orders). Home oxygen is not counted as medication. The preferred language of the family caregiver is recorded during patient registration or by the admitting inpatient nurse and is gleaned from either of these flowsheet sources.

Evaluation of Predictive Utility for Readmission

We performed a retrospective cohort study on patients aged 0-21 years who were discharged between January 1, 2014 and December 30, 2015. This study was performed to determine the optimal cut points for the continuous variables (discharge medications and home-care orders) and to evaluate the predictive value of the composite score.

Unplanned readmission within 30 days was used as the primary outcome. The index hospitalization was randomly selected for patients with >1 admissions to avoid biasing the results with multiple hospitalizations from individual patients.

Patient characteristics were summarized using percentages for categorical variables and the median and interquartile range (IQR) for continuous variables. We examined bivariate associations for each of the tool’s predictor elements with readmission using Chi-square and Wilcoxon tests (level of statistical significance was set at 0.05). Receiver operating characteristics (ROC) analyses established optimal dichotomization points for medications and home-care orders using the maximized Youden’s index (sensitivity + specificity – 1).⁹ Dichotomization of these variables was selected for ease of implementation and interpretation. Similarly, the composite score was treated as a categorical predictor and because sensitivity analysis using it as a continuous predictor did not improve the tool's discriminatory properties.

The area under the ROC curve (AUC) was estimated to evaluate the performance of the composite score (as a categorical variable) using a predictive logistic regression model. To establish internal validity, we performed 10-fold cross validation analysis.¹⁰ Measures of predictive performance included the c statistic and the Brier score. The c statistic represents the discriminative ability of a model. A model with AUC > 0.9 signifies high discrimination, 0.7-0.9 indicates moderate discrimination, and 0.5-0.7 suggests low discrimination. The Brier score is a measure of the overall accuracy of the predictions and ranges from 0 to 1, with 0 representing perfect accuracy.¹⁰ Analyses were performed using SAS v9.4.

Cohort Characteristics

Analysis was restricted to patients with at least 30 days of available follow-up time after the index admission (N = 29,542 patients; Figure). Patient characteristics from the index admission are shown in the Table. The median age was 5 years, and median LOS was 2 days. A total of 19% of patients were discharged with ≥6 discharge medications, 15% received ≥1 home-health orders, and 10% were documented to have a non-English speaking family caregiver. Almost 28% had a composite score of 1 and 8% a score ≥2. The unplanned 30-day readmission rate was 4%. In bivariate analysis, children with readmission had longer LOS, more discharge medications, and more home care than children without readmission. Caregiver language preference was not associated with readmission.

ROC analysis indicated that dichotomizing number of medications at ≥6 vs. <6 and home health at 0 vs. ≥1 categories maximized the sensitivity and specificity for predicting 30-day unplanned readmissions. In predictive logistic regression analysis, the odds of readmission was significantly higher in children with a composite score of 1 vs. 0 (odds ratio [OR], 1.7; 95% CI 1.5-2) and a score of ≥2 vs. 0 (OR, 4.2; 95% CI, 3.6-4.9). The c statistic for this model was 0.62, and the Brier score was 0.037. Internal validation of the predictive logistic regression model yielded identical results.

Disclosures

The authors have no relevant financial relationships to disclose.

Funding

This study was supported by an institutional Clinical and Operational Effectiveness and Patient Safety Small Grants Program

As hospitalized children become more medically complex, hospital-to-home care transitions will become increasingly challenging. During a quality improvement (QI) initiative, we developed an electronic tool to improve the quality of our hospital discharge process.

We modeled the concept of a paper-based Early Screen for Discharge Planning – Child Version, which identifies children with multiple medical conditions, home nursing-care needs, tube feedings, presence of intravenous lines or drains, or post hospital care that requires coordination.¹We opted for an electronic tool to automate screening and increase visibilty of patients’ care transitional needs via the electronic health record (EHR).

The tool was designed by our QI team to address weaknesses in our discharge process (eg, discharge instructions that are not translated appropriately) and causes of preventable readmission at our institution (eg, discharge teaching, home care, and medication-related problems).^2,3 The tool’s selected components comprised those that might complicate or delay discharge care and included indicators of home health, polypharmacy, and caregiver language preference. Additional features were considered but withheld from the initial tool for several reasons noted in the methods.

We describe the development and implementation of this electronic tool. Given the paucity of pediatric risk models, we conducted an analysis of the tool’s potential to predict readmissions. We anticipated good predictive performance because the tool includes measures previously associated with readmission (eg, technology dependence, polypharmacy, and language barrier).^4-7 If successful in distinguishing readmission, this embedded discharge planning tool could also serve as a pediatric readmission risk score.

Setting

This work was conducted at the Children’s Hospital Colorado as part of a national QI collaborative. The hospital’s EHR is Epic (Verona, Wisconsin). The project was approved as QI by the Children’s Hospital Organizational Research Risk and Quality Improvement Review Panel, precluding review from the Colorado Multiple Institutional Review Board.

Tool Design, Implementation, and Use

A team of clinicians, nurse–family educators, case managers, social workers, and informatics experts helped design the instrument between 2014 and 2015. In addition to the selected features (number of discharge medications, presence of home health, and language preference), we considered adding the number of consulting specialists but had previously improved our process for scheduling follow-up appointments. Diagnoses were not systematically or discretely documented to be reliably extracted in real time. We excluded known readmission predictor variables (such as length of stay [LOS] and prior hospitalizations) from the initial model to maintain emphasis on the modifiable discharge processes. Additional considerations, such as health literacy and social determinants, were not systematically measured to be operationally usable.

To generate the score, the clinical documentation of home-health orders categorizes children with home care. Each home-care equipment or service category is documented in separate flowsheet rows, allowing for identification of distinct categories (Table). Total parenteral nutrition, intravenous medications, and durable medical equipment and supplies are counted as home care. The number of discharge medications is approximated by inpatient medication orders and finalized as the number of discharge medication orders. The medications include new, historic, and as-needed medications (if included among discharge medication orders). Home oxygen is not counted as medication. The preferred language of the family caregiver is recorded during patient registration or by the admitting inpatient nurse and is gleaned from either of these flowsheet sources.

Evaluation of Predictive Utility for Readmission

We performed a retrospective cohort study on patients aged 0-21 years who were discharged between January 1, 2014 and December 30, 2015. This study was performed to determine the optimal cut points for the continuous variables (discharge medications and home-care orders) and to evaluate the predictive value of the composite score.

Unplanned readmission within 30 days was used as the primary outcome. The index hospitalization was randomly selected for patients with >1 admissions to avoid biasing the results with multiple hospitalizations from individual patients.

Patient characteristics were summarized using percentages for categorical variables and the median and interquartile range (IQR) for continuous variables. We examined bivariate associations for each of the tool’s predictor elements with readmission using Chi-square and Wilcoxon tests (level of statistical significance was set at 0.05). Receiver operating characteristics (ROC) analyses established optimal dichotomization points for medications and home-care orders using the maximized Youden’s index (sensitivity + specificity – 1).⁹ Dichotomization of these variables was selected for ease of implementation and interpretation. Similarly, the composite score was treated as a categorical predictor and because sensitivity analysis using it as a continuous predictor did not improve the tool's discriminatory properties.

The area under the ROC curve (AUC) was estimated to evaluate the performance of the composite score (as a categorical variable) using a predictive logistic regression model. To establish internal validity, we performed 10-fold cross validation analysis.¹⁰ Measures of predictive performance included the c statistic and the Brier score. The c statistic represents the discriminative ability of a model. A model with AUC > 0.9 signifies high discrimination, 0.7-0.9 indicates moderate discrimination, and 0.5-0.7 suggests low discrimination. The Brier score is a measure of the overall accuracy of the predictions and ranges from 0 to 1, with 0 representing perfect accuracy.¹⁰ Analyses were performed using SAS v9.4.

Cohort Characteristics

Analysis was restricted to patients with at least 30 days of available follow-up time after the index admission (N = 29,542 patients; Figure). Patient characteristics from the index admission are shown in the Table. The median age was 5 years, and median LOS was 2 days. A total of 19% of patients were discharged with ≥6 discharge medications, 15% received ≥1 home-health orders, and 10% were documented to have a non-English speaking family caregiver. Almost 28% had a composite score of 1 and 8% a score ≥2. The unplanned 30-day readmission rate was 4%. In bivariate analysis, children with readmission had longer LOS, more discharge medications, and more home care than children without readmission. Caregiver language preference was not associated with readmission.

ROC analysis indicated that dichotomizing number of medications at ≥6 vs. <6 and home health at 0 vs. ≥1 categories maximized the sensitivity and specificity for predicting 30-day unplanned readmissions. In predictive logistic regression analysis, the odds of readmission was significantly higher in children with a composite score of 1 vs. 0 (odds ratio [OR], 1.7; 95% CI 1.5-2) and a score of ≥2 vs. 0 (OR, 4.2; 95% CI, 3.6-4.9). The c statistic for this model was 0.62, and the Brier score was 0.037. Internal validation of the predictive logistic regression model yielded identical results.

Disclosures

The authors have no relevant financial relationships to disclose.

Funding

This study was supported by an institutional Clinical and Operational Effectiveness and Patient Safety Small Grants Program

As hospitalized children become more medically complex, hospital-to-home care transitions will become increasingly challenging. During a quality improvement (QI) initiative, we developed an electronic tool to improve the quality of our hospital discharge process.

We modeled the concept of a paper-based Early Screen for Discharge Planning – Child Version, which identifies children with multiple medical conditions, home nursing-care needs, tube feedings, presence of intravenous lines or drains, or post hospital care that requires coordination.¹We opted for an electronic tool to automate screening and increase visibilty of patients’ care transitional needs via the electronic health record (EHR).

The tool was designed by our QI team to address weaknesses in our discharge process (eg, discharge instructions that are not translated appropriately) and causes of preventable readmission at our institution (eg, discharge teaching, home care, and medication-related problems).^2,3 The tool’s selected components comprised those that might complicate or delay discharge care and included indicators of home health, polypharmacy, and caregiver language preference. Additional features were considered but withheld from the initial tool for several reasons noted in the methods.

We describe the development and implementation of this electronic tool. Given the paucity of pediatric risk models, we conducted an analysis of the tool’s potential to predict readmissions. We anticipated good predictive performance because the tool includes measures previously associated with readmission (eg, technology dependence, polypharmacy, and language barrier).^4-7 If successful in distinguishing readmission, this embedded discharge planning tool could also serve as a pediatric readmission risk score.

Setting

This work was conducted at the Children’s Hospital Colorado as part of a national QI collaborative. The hospital’s EHR is Epic (Verona, Wisconsin). The project was approved as QI by the Children’s Hospital Organizational Research Risk and Quality Improvement Review Panel, precluding review from the Colorado Multiple Institutional Review Board.

Tool Design, Implementation, and Use

A team of clinicians, nurse–family educators, case managers, social workers, and informatics experts helped design the instrument between 2014 and 2015. In addition to the selected features (number of discharge medications, presence of home health, and language preference), we considered adding the number of consulting specialists but had previously improved our process for scheduling follow-up appointments. Diagnoses were not systematically or discretely documented to be reliably extracted in real time. We excluded known readmission predictor variables (such as length of stay [LOS] and prior hospitalizations) from the initial model to maintain emphasis on the modifiable discharge processes. Additional considerations, such as health literacy and social determinants, were not systematically measured to be operationally usable.

To generate the score, the clinical documentation of home-health orders categorizes children with home care. Each home-care equipment or service category is documented in separate flowsheet rows, allowing for identification of distinct categories (Table). Total parenteral nutrition, intravenous medications, and durable medical equipment and supplies are counted as home care. The number of discharge medications is approximated by inpatient medication orders and finalized as the number of discharge medication orders. The medications include new, historic, and as-needed medications (if included among discharge medication orders). Home oxygen is not counted as medication. The preferred language of the family caregiver is recorded during patient registration or by the admitting inpatient nurse and is gleaned from either of these flowsheet sources.

Evaluation of Predictive Utility for Readmission

We performed a retrospective cohort study on patients aged 0-21 years who were discharged between January 1, 2014 and December 30, 2015. This study was performed to determine the optimal cut points for the continuous variables (discharge medications and home-care orders) and to evaluate the predictive value of the composite score.

Unplanned readmission within 30 days was used as the primary outcome. The index hospitalization was randomly selected for patients with >1 admissions to avoid biasing the results with multiple hospitalizations from individual patients.

Patient characteristics were summarized using percentages for categorical variables and the median and interquartile range (IQR) for continuous variables. We examined bivariate associations for each of the tool’s predictor elements with readmission using Chi-square and Wilcoxon tests (level of statistical significance was set at 0.05). Receiver operating characteristics (ROC) analyses established optimal dichotomization points for medications and home-care orders using the maximized Youden’s index (sensitivity + specificity – 1).⁹ Dichotomization of these variables was selected for ease of implementation and interpretation. Similarly, the composite score was treated as a categorical predictor and because sensitivity analysis using it as a continuous predictor did not improve the tool's discriminatory properties.

The area under the ROC curve (AUC) was estimated to evaluate the performance of the composite score (as a categorical variable) using a predictive logistic regression model. To establish internal validity, we performed 10-fold cross validation analysis.¹⁰ Measures of predictive performance included the c statistic and the Brier score. The c statistic represents the discriminative ability of a model. A model with AUC > 0.9 signifies high discrimination, 0.7-0.9 indicates moderate discrimination, and 0.5-0.7 suggests low discrimination. The Brier score is a measure of the overall accuracy of the predictions and ranges from 0 to 1, with 0 representing perfect accuracy.¹⁰ Analyses were performed using SAS v9.4.

Cohort Characteristics

Analysis was restricted to patients with at least 30 days of available follow-up time after the index admission (N = 29,542 patients; Figure). Patient characteristics from the index admission are shown in the Table. The median age was 5 years, and median LOS was 2 days. A total of 19% of patients were discharged with ≥6 discharge medications, 15% received ≥1 home-health orders, and 10% were documented to have a non-English speaking family caregiver. Almost 28% had a composite score of 1 and 8% a score ≥2. The unplanned 30-day readmission rate was 4%. In bivariate analysis, children with readmission had longer LOS, more discharge medications, and more home care than children without readmission. Caregiver language preference was not associated with readmission.

ROC analysis indicated that dichotomizing number of medications at ≥6 vs. <6 and home health at 0 vs. ≥1 categories maximized the sensitivity and specificity for predicting 30-day unplanned readmissions. In predictive logistic regression analysis, the odds of readmission was significantly higher in children with a composite score of 1 vs. 0 (odds ratio [OR], 1.7; 95% CI 1.5-2) and a score of ≥2 vs. 0 (OR, 4.2; 95% CI, 3.6-4.9). The c statistic for this model was 0.62, and the Brier score was 0.037. Internal validation of the predictive logistic regression model yielded identical results.

Disclosures

The authors have no relevant financial relationships to disclose.

Funding

This study was supported by an institutional Clinical and Operational Effectiveness and Patient Safety Small Grants Program

A 73-year-old man presented to the emergency department in late winter with fevers, myalgias, fatigue, low back pain, and poor oral intake. Four days earlier, he had fallen and hit his head. His partner also noticed a few episodes of confusion in the days leading up to presentation.

The patient’s symptoms are nonspecific. Fevers prompt the consideration of systemic infection, though fevers can also be seen in a broad range of noninfectious processes, including malignancy, vasculitis, autoimmune conditions, endocrinopathies, and drug reaction. The clinical picture warrants prompt and comprehensive evaluation, beginning with further detailed history (current illnesses, exposures, travel, vaccinations, medications, cancer screenings, weight change) and a careful physical examination, which will help guide laboratory testing and imaging.

His past medical history was notable for coronary artery disease for which he underwent coronary artery bypass grafting five years prior, hypertension, hyperlipidemia, diet-controlled type 2 diabetes mellitus, gastroesophageal reflux disease, osteoarthritis leading to chronic knee and hand pain, and a history of mildly low testosterone levels. His medications included hydrocodone and acetaminophen, metoprolol tartrate, omeprazole, topical testosterone gel (prescribed for daily use, used intermittently), and aspirin. He was retired and lived in rural Michigan with his female partner. He previously worked as a truck driver and used to train racehorses. He had quit smoking five years earlier. He denied alcohol or injection drug use.

The patient has significant underlying medical conditions. Considering infectious causes of his symptoms, it is notable that he has no reported immunodeficiency. It would be relevant to know if he has been tested for HIV. His rural residence and work with horses raise the possibility of zoonotic infections, including plague (Yersinia pestis), brucellosis (Brucella species), Q fever (Coxiella burnetti), Rhodococcus equi, or group C or G Streptococci. Information about tuberculosis risk factors, other geographic exposures, recent dental work, and ill contacts might be helpful to elucidate the causes of this nonspecific febrile illness with a possible CNS component. With regard to malignancy, it would be helpful to ask about recent weight loss, lymphadenopathy, and prior cancer screenings. Considering other etiologies, he does not report a history of autoimmune or endocrine conditions. However, it is important to consider a vasculitis, such as giant cell arteritis or polyarteritis nodosa, autoimmune conditions, and endocrinopathies such as thyrotoxicosis. The differential diagnosis for his clinical syndrome remains broad.

Vital signs were temperature 37.3°C, heart rate 88 beats per minute, respiratory rate 18 breaths per minute, blood pressure 105/64 mmHg, and oxygen saturation 93% on room air. Oral examination revealed poor dentition. The heart had a normal rate and regular rhythm with no murmurs, rubs, or gallops, and lungs were clear to auscultation bilaterally. The abdomen was unremarkable. Examination of the back was notable for mild tenderness to palpation over the sacrum. He was oriented to person, place, and time, with intact cranial nerves and a nonfocal neurologic examination. The remainder of his examination was normal. The white blood cell (WBC) count was 11.1 × 10³/μL, with 84% neutrophils and 9% bands, hemoglobin 13.6 g/dL, platelet count 54 × 10³/μL, sodium 122 mmol/L, potassium 3.3 mmol/L, chloride 89 mmol/L, bicarbonate 21 mmol/L, creatinine 1.64 mg/dL, albumin 2.7 g/dL, alkaline phosphatase 136 U/L, AST 60 U/L, ALT 37 U/L, and total bilirubin 2.1 mg/dL.

He had presented to the emergency department five days earlier with fever, flank pain, nausea, vomiting, and weakness. At that time, he had a temperature of 38.2°C, but vital signs otherwise had been normal. Laboratory studies had revealed WBC count 14.0 × 10³/μL, hemoglobin 13.7 g/dL, platelet count 175 × 10³/μL, sodium 129 mmol/L, chloride 97 mmol/dL, bicarbonate 23 mmol/L, creatinine 1.1 mg/dL, and total bilirubin 1.6 mg/dL. Urinalysis had been negative. He had received one liter of intravenous normal saline and ketorolac for pain and had been discharged with the diagnosis of a viral illness.

A picture of a progressive, subacute illness with multisystem involvement appears to be emerging, and there are several abnormalities consistent with infection, including fever, leukocytosis with bandemia, thrombocytopenia, renal dysfunction, and elevated bilirubin. His borderline hypotension may be due to uninterrupted use of his antihypertensive medication in the setting of poor oral intake or may indicate incipient sepsis. Focal sacral tenderness raises the possibility of vertebral osteomyelitis or epidural abscess, either from a contiguous focus of infection from the surrounding structures, or as a site of seeding from bacteremia. His prior confusion episodes might have been secondary to a systemic process; however, CNS imaging should be done, given the history of confusion and recent fall. Further diagnostic studies are warranted, including: blood cultures; peripheral blood smear; imaging of the spine, chest, abdomen, and pelvis; electrocardiogram; and possibly echocardiogram. Although noninfectious etiologies should not be discounted, the constellation of findings is more compatible with infection.

Two sets of blood cultures and a viral respiratory swab were obtained. Computed tomography (CT) of the head without contrast was negative for acute bleeding or other intracranial pathology. Lumbosacral radiography revealed degenerative changes with intact alignment of the sacrum. The patient was admitted with plans to pursue lumbar puncture if altered mental status recurred. The viral swab was negative. Within 24 hours, one set of blood cultures (both bottles) grew lactose-negative, oxidase-negative, gram-negative rods.

Gram-negative rods (GNRs) rarely are contaminants in blood cultures and should be considered significant until proven otherwise. Prompt empiric therapy and investigation to identify the primary source of bacteremia must be initiated. Although the most common GNRs isolated from blood cultures are enteric coliform organisms such as E. coli, Klebsiella, and Enterobacter, these typically are lactose-positive. Additional possibilities should be considered, including Salmonella species or other organisms comprising the “HACEK” group. This latter group is commonly associated with endocarditis, but the majority are oxidase-positive and have more fastidious growth requirements. Although there are other gram-negative organisms to consider, they have other distinguishing characteristics that have not been indicated in the microbiology results. Broad-spectrum antibiotic therapy is appropriate while awaiting the final identification of the GNR. A thorough search for a primary source and secondary sites of hematogenous seeding should be conducted. His only localizing symptom was tenderness over the sacrum, and this should be further assessed by sensitive imaging such as magnetic resonance imaging (MRI). The identity of the GNR would guide further diagnostic evaluation. For example, a respiratory organism such as Haemophilus influenzae would prompt a CT scan of the chest. Isolation of an enteric or a coliform GNR such as E. coli would prompt abdominal and pelvic imaging to assess for occult abscess. An “HACEK” group organism would prompt echocardiography to evaluate for endocarditis.

He was started on piperacillin–tazobactam. GNR bacteremia without a clear source prompted a CT of the chest, abdomen, and pelvis with and without contrast. The images were unremarkable, with the exception of a signal abnormality in the left psoas muscle concerning for abscess (Figure 1). MRI of the same region revealed L2-4 osteomyelitis and discitis with bilateral psoas abscesses but without epidural abscess (Figure 2).

Because of increasing reports of antibiotic resistance in GNRs, even in community-acquired infections, it is appropriate to initially treat with a broad-spectrum antibiotic such as a fourth-generation cephalosporin or carbapenem while awaiting identification and susceptibility results to guide definitive therapy. In addition to antimicrobial therapy, treatment of psoas abscess usually requires drainage. Vertebral osteomyelitis from a hematogenous source can often be treated with antibiotics alone, as long as there are no associated complications such as epidural abscess and spine instability. Imaging should be reviewed for pathology of the surrounding structures, and surgical consultation should be obtained.

Neurosurgery, Interventional Radiology, and Infectious Disease services were consulted. Antibiotic coverage was expanded to vancomycin, cefepime, and metronidazole due to the possibility of polymicrobial infection. No surgical intervention was recommended since the abscesses were too small to drain.

The next day, the GNR was identified as Serratia marcescens.

S. marcescens is a widely distributed organism in the environment, but not a common component of endogenous human flora. Serratia is generally considered as an opportunistic nosocomial pathogen. Community-acquired infection with this organism is unusual and implies exogenous acquisition. A careful re-evaluation of exposures, including injection drug use or other parenteral exposures is important to identify the likely source of infection, as these have been previously linked to outbreaks of environmental organisms. Based on the presumed pathogenesis of infection and the initial microbiology suggesting monomicrobial Serratia infection, antibiotics should be narrowed based on the susceptibility results. There is concern that antibiotics might not adequately penetrate the abscesses and result in a lack of clinical improvement and/or lead to the emergence of antibiotic resistance during therapy. This is an important concern with Serratia, which typically harbors an AmpC beta-lactamase that can mediate resistance to broad-spectrum cephalosporins. If medical therapy alone without drainage is planned, short-interval re-imaging is warranted.

Blood cultures from days two and three of hospitalization also grew S. marcescens. No other organisms grew. Based on culture sensitivity data, antibiotics were narrowed to ceftriaxone.

This surprising culture result prompted the medical team to obtain screening laboratory tests for immunocompromising conditions and to revisit the patient’s history. His type 2 diabetes mellitus was well controlled with a hemoglobin A1c of 6.5%. HIV testing was negative. Further questioning of the patient revealed that he had fallen from a truck onto rocks four months prior, injuring his back and hip, but without puncture of the skin or loss of consciousness; he denied recent falls or other injuries but reported significant chronic knee pain. He had not been hospitalized recently. He had never taken corticosteroids or immunomodulatory medications. He continued to deny injection drug use. He did, however, clarify that his work with racehorses, which was originally understood to be a prior hobby, was ongoing, including recent work of cleaning the stables.

The following morning, he experienced confusion, rigors, and hypoxia, which prompted transfer to the intensive care unit (ICU).

Acute worsening during treatment is worrisome, and could be a potential complication of his infection or treatment – or even a separate process altogether. Knee pain in the setting of bacteremia raises the possibility of septic or crystal-induced arthritis and warrants imaging. Confusion and hypoxia might represent secondary sites of seeding from bacteremia (CNS infection and pneumonia, respectively) or manifestations of endocarditis, the latter being unusual for Serratia. An echocardiogram should be obtained. Other neurologic causes, including seizure, should also be considered. Further evaluation by chest imaging and repeat neurologic examination and imaging should be performed. Emergence of resistance during therapy is a theoretical concern with Serratia as an AmpC beta-lactamase-containing organism. While awaiting additional microbiology data, an empiric change to an AmpC beta-lactamase stable antibiotic such as a carbapenem should be made, especially since he has clinically deteriorated on therapy with a β-lactamase susceptible antibiotic, raising concerns of the emergence of resistance on initial therapy.

Antibiotics were changed to meropenem, vancomycin, and metronidazole given the clinical worsening and concerns that this represented infection unresponsive to prior antibiotics. The acute episode resolved spontaneously after one hour. His neurologic examination remained nonfocal. Chest radiography, urinalysis, urine culture, and right upper quadrant ultrasound were unremarkable. Transesophageal echocardiogram revealed no heart valve vegetations. MRI and bone scan of the lower extremities did not show any evidence of septic arthritis or other infection. He remained stable and was transferred out of the ICU the following day. Antibiotic coverage was switched to cefepime. On discussion with his significant other, this event was found to be similar to the intermittent confusion that occurred in the days prior to admission.

The acute onset and other features of these intermittent periods of deterioration are compatible with infection; intermittent seeding of the blood with microbes or their products (eg, lipopolysaccharides) from an abscess or vascular infection could explain these episodes. Some of the previous hypotheses to explain the episodes, such as a secondary infectious process, have not been supported by diagnostic testing or the clinical course. He needs close clinical monitoring and interval assessment of the known sites of infection.

Ten days after osteomyelitis and discitis were diagnosed, the patient developed worsening low back pain, prompting repeat spine MRI. This was significant for bilateral psoas abscess enlargement and extension of osteomyelitis and discitis (Figure 3). He was re-evaluated by Neurosurgery and Interventional Radiology and underwent psoas abscess drainage; abscess cultures grew S. marcescens.

He slowly improved over several weeks and was discharged to a subacute rehabilitation facility. He completed a 3.5-week course of intravenous antibiotics before leaving against medical advice. He completed eight weeks of oral trimethoprim-sulfamethoxazole and remains without long-term sequelae from the infection.

S. marcescens is a gram-negative rod in the Enterobacteriaceae family known for its red pigment. Primarily, S. marcescens causes nosocomial infections, most commonly of the respiratory and urinary tracts. However, a wide range of manifestations has been documented, including meningitis, ocular infections (conjunctivitis, keratitis, endophthalmitis), endocarditis, skin infections (cellulitis, necrotizing fasciitis), and osteomyelitis.^{1, 2} S. marcescens is often reported as the cause of outbreaks in ICUs;^3-6 infection is thought to occur via contamination of water pipes, hospital equipment, and disinfectants.^{3, 7} Its natural environment includes soil, water, and GI tracts of animals,⁴ and there are published reports of S. marcescens infection in horses.^{8, 9} This patient was most likely exposed to S. marcescens through his work with horses and their environment.

S. marcescens has wide-ranging target organs, and successful treatment can be difficult. S. marcescens can infect the renal, respiratory, gastrointestinal, ocular, cardiovascular, and musculoskeletal systems. S. marcescens, like other “SPACE” organisms (Serratia, Pseudomonas, Acinetobacter, Citrobacter, Enterobacter), expresses inducible AmpC beta-lactamase.¹⁰ At baseline, AmpC beta-lactamase expression is repressed.¹¹ Mutants with stably de-repressed (constitutively expressed) AmpC can be selected during therapy and lead to clinical failure, as has been best described during therapy for Enterobacter infections.¹² Infectious Disease consultation may be helpful when caring for patients with S. marcescens bacteremia given these complexities.

This was an unusual case of S. marcescens infection. It most commonly infects immunocompromised hosts. Reported risk factors include solid organ or hematopoietic stem cell transplant, malignancy, HIV/AIDS, and receipt of immunosuppressive agents. The patient did not have these risk factors, but did have well-controlled type 2 diabetes mellitus. Although diabetes is associated with an increased risk of infection and more severe infections,^{13, 14} there is no evidence in the literature that well-controlled type 2 diabetes mellitus compromises the immune system. A few case reports document cutaneous S. marcescens infection in immunocompetent adults.^15,16 A case report of S. marcescens septic arthritis and adjacent osteomyelitis has also been published, but the patient had poorly controlled diabetes.¹⁷ This case provides a report of systemic S. marcescens infection in an individual without clear risk factors.

S. marcescens osteomyelitis is rare, and there have been only a few prior case reports.^2,18 The presentation of osteomyelitis, regardless of the causative organism, is subtle, often insidious, and can easily be missed. Hospitalists should have a high index of suspicion for the diagnosis as it requires prompt evaluation and treatment for complications, including epidural abscess. Risk factors include diabetes mellitus, rheumatoid arthritis, injection drug use, and other immunocompromising illnesses.¹⁹ Degenerative changes in the spine such as osteoarthritis may be risk factors as well,²⁰ though not well studied or quantified. A hypothesized mechanism involves local inflammation and joint damage, leaving the area susceptible to bacterial seeding. Osteoarthritis and degenerative disc disease, along with exposure to racehorses, likely put this patient at risk for bacterial seeding in the vertebrae, ultimately leading to a “dark horse” diagnosis.

• Serratia marcescens is a gram-negative rod bacterium that most commonly infects immunocompromised individuals in hospital settings. This report demonstrates that S. marcescens can cause serious infection in immunocompetent, nonhospitalized adults.
• S. marcescens bacteremia or infection of organs outside of the urinary or respiratory systems is uncommon, and therapy can be complicated by emergence of resistance.
• The clinical presentation of vertebral osteomyelitis and discitis and psoas abscess can be subtle and may present without typical signs and symptoms of infection.

ACKNOWLEDGEMENTS

The authors thank the patient and his partner for their willingness to have his story published, Laura Petersen, MHSA, for providing assistance with references and manuscript editing, and Shadi Azar, MBBS, for assistance in selecting the cross-sectional images.

Disclosures

The authors have no conflicts of interest to disclose.

A 73-year-old man presented to the emergency department in late winter with fevers, myalgias, fatigue, low back pain, and poor oral intake. Four days earlier, he had fallen and hit his head. His partner also noticed a few episodes of confusion in the days leading up to presentation.

The patient’s symptoms are nonspecific. Fevers prompt the consideration of systemic infection, though fevers can also be seen in a broad range of noninfectious processes, including malignancy, vasculitis, autoimmune conditions, endocrinopathies, and drug reaction. The clinical picture warrants prompt and comprehensive evaluation, beginning with further detailed history (current illnesses, exposures, travel, vaccinations, medications, cancer screenings, weight change) and a careful physical examination, which will help guide laboratory testing and imaging.

His past medical history was notable for coronary artery disease for which he underwent coronary artery bypass grafting five years prior, hypertension, hyperlipidemia, diet-controlled type 2 diabetes mellitus, gastroesophageal reflux disease, osteoarthritis leading to chronic knee and hand pain, and a history of mildly low testosterone levels. His medications included hydrocodone and acetaminophen, metoprolol tartrate, omeprazole, topical testosterone gel (prescribed for daily use, used intermittently), and aspirin. He was retired and lived in rural Michigan with his female partner. He previously worked as a truck driver and used to train racehorses. He had quit smoking five years earlier. He denied alcohol or injection drug use.

The patient has significant underlying medical conditions. Considering infectious causes of his symptoms, it is notable that he has no reported immunodeficiency. It would be relevant to know if he has been tested for HIV. His rural residence and work with horses raise the possibility of zoonotic infections, including plague (Yersinia pestis), brucellosis (Brucella species), Q fever (Coxiella burnetti), Rhodococcus equi, or group C or G Streptococci. Information about tuberculosis risk factors, other geographic exposures, recent dental work, and ill contacts might be helpful to elucidate the causes of this nonspecific febrile illness with a possible CNS component. With regard to malignancy, it would be helpful to ask about recent weight loss, lymphadenopathy, and prior cancer screenings. Considering other etiologies, he does not report a history of autoimmune or endocrine conditions. However, it is important to consider a vasculitis, such as giant cell arteritis or polyarteritis nodosa, autoimmune conditions, and endocrinopathies such as thyrotoxicosis. The differential diagnosis for his clinical syndrome remains broad.

Vital signs were temperature 37.3°C, heart rate 88 beats per minute, respiratory rate 18 breaths per minute, blood pressure 105/64 mmHg, and oxygen saturation 93% on room air. Oral examination revealed poor dentition. The heart had a normal rate and regular rhythm with no murmurs, rubs, or gallops, and lungs were clear to auscultation bilaterally. The abdomen was unremarkable. Examination of the back was notable for mild tenderness to palpation over the sacrum. He was oriented to person, place, and time, with intact cranial nerves and a nonfocal neurologic examination. The remainder of his examination was normal. The white blood cell (WBC) count was 11.1 × 10³/μL, with 84% neutrophils and 9% bands, hemoglobin 13.6 g/dL, platelet count 54 × 10³/μL, sodium 122 mmol/L, potassium 3.3 mmol/L, chloride 89 mmol/L, bicarbonate 21 mmol/L, creatinine 1.64 mg/dL, albumin 2.7 g/dL, alkaline phosphatase 136 U/L, AST 60 U/L, ALT 37 U/L, and total bilirubin 2.1 mg/dL.

He had presented to the emergency department five days earlier with fever, flank pain, nausea, vomiting, and weakness. At that time, he had a temperature of 38.2°C, but vital signs otherwise had been normal. Laboratory studies had revealed WBC count 14.0 × 10³/μL, hemoglobin 13.7 g/dL, platelet count 175 × 10³/μL, sodium 129 mmol/L, chloride 97 mmol/dL, bicarbonate 23 mmol/L, creatinine 1.1 mg/dL, and total bilirubin 1.6 mg/dL. Urinalysis had been negative. He had received one liter of intravenous normal saline and ketorolac for pain and had been discharged with the diagnosis of a viral illness.

A picture of a progressive, subacute illness with multisystem involvement appears to be emerging, and there are several abnormalities consistent with infection, including fever, leukocytosis with bandemia, thrombocytopenia, renal dysfunction, and elevated bilirubin. His borderline hypotension may be due to uninterrupted use of his antihypertensive medication in the setting of poor oral intake or may indicate incipient sepsis. Focal sacral tenderness raises the possibility of vertebral osteomyelitis or epidural abscess, either from a contiguous focus of infection from the surrounding structures, or as a site of seeding from bacteremia. His prior confusion episodes might have been secondary to a systemic process; however, CNS imaging should be done, given the history of confusion and recent fall. Further diagnostic studies are warranted, including: blood cultures; peripheral blood smear; imaging of the spine, chest, abdomen, and pelvis; electrocardiogram; and possibly echocardiogram. Although noninfectious etiologies should not be discounted, the constellation of findings is more compatible with infection.

Two sets of blood cultures and a viral respiratory swab were obtained. Computed tomography (CT) of the head without contrast was negative for acute bleeding or other intracranial pathology. Lumbosacral radiography revealed degenerative changes with intact alignment of the sacrum. The patient was admitted with plans to pursue lumbar puncture if altered mental status recurred. The viral swab was negative. Within 24 hours, one set of blood cultures (both bottles) grew lactose-negative, oxidase-negative, gram-negative rods.

Gram-negative rods (GNRs) rarely are contaminants in blood cultures and should be considered significant until proven otherwise. Prompt empiric therapy and investigation to identify the primary source of bacteremia must be initiated. Although the most common GNRs isolated from blood cultures are enteric coliform organisms such as E. coli, Klebsiella, and Enterobacter, these typically are lactose-positive. Additional possibilities should be considered, including Salmonella species or other organisms comprising the “HACEK” group. This latter group is commonly associated with endocarditis, but the majority are oxidase-positive and have more fastidious growth requirements. Although there are other gram-negative organisms to consider, they have other distinguishing characteristics that have not been indicated in the microbiology results. Broad-spectrum antibiotic therapy is appropriate while awaiting the final identification of the GNR. A thorough search for a primary source and secondary sites of hematogenous seeding should be conducted. His only localizing symptom was tenderness over the sacrum, and this should be further assessed by sensitive imaging such as magnetic resonance imaging (MRI). The identity of the GNR would guide further diagnostic evaluation. For example, a respiratory organism such as Haemophilus influenzae would prompt a CT scan of the chest. Isolation of an enteric or a coliform GNR such as E. coli would prompt abdominal and pelvic imaging to assess for occult abscess. An “HACEK” group organism would prompt echocardiography to evaluate for endocarditis.

He was started on piperacillin–tazobactam. GNR bacteremia without a clear source prompted a CT of the chest, abdomen, and pelvis with and without contrast. The images were unremarkable, with the exception of a signal abnormality in the left psoas muscle concerning for abscess (Figure 1). MRI of the same region revealed L2-4 osteomyelitis and discitis with bilateral psoas abscesses but without epidural abscess (Figure 2).

Because of increasing reports of antibiotic resistance in GNRs, even in community-acquired infections, it is appropriate to initially treat with a broad-spectrum antibiotic such as a fourth-generation cephalosporin or carbapenem while awaiting identification and susceptibility results to guide definitive therapy. In addition to antimicrobial therapy, treatment of psoas abscess usually requires drainage. Vertebral osteomyelitis from a hematogenous source can often be treated with antibiotics alone, as long as there are no associated complications such as epidural abscess and spine instability. Imaging should be reviewed for pathology of the surrounding structures, and surgical consultation should be obtained.

Neurosurgery, Interventional Radiology, and Infectious Disease services were consulted. Antibiotic coverage was expanded to vancomycin, cefepime, and metronidazole due to the possibility of polymicrobial infection. No surgical intervention was recommended since the abscesses were too small to drain.

The next day, the GNR was identified as Serratia marcescens.

S. marcescens is a widely distributed organism in the environment, but not a common component of endogenous human flora. Serratia is generally considered as an opportunistic nosocomial pathogen. Community-acquired infection with this organism is unusual and implies exogenous acquisition. A careful re-evaluation of exposures, including injection drug use or other parenteral exposures is important to identify the likely source of infection, as these have been previously linked to outbreaks of environmental organisms. Based on the presumed pathogenesis of infection and the initial microbiology suggesting monomicrobial Serratia infection, antibiotics should be narrowed based on the susceptibility results. There is concern that antibiotics might not adequately penetrate the abscesses and result in a lack of clinical improvement and/or lead to the emergence of antibiotic resistance during therapy. This is an important concern with Serratia, which typically harbors an AmpC beta-lactamase that can mediate resistance to broad-spectrum cephalosporins. If medical therapy alone without drainage is planned, short-interval re-imaging is warranted.

Blood cultures from days two and three of hospitalization also grew S. marcescens. No other organisms grew. Based on culture sensitivity data, antibiotics were narrowed to ceftriaxone.

This surprising culture result prompted the medical team to obtain screening laboratory tests for immunocompromising conditions and to revisit the patient’s history. His type 2 diabetes mellitus was well controlled with a hemoglobin A1c of 6.5%. HIV testing was negative. Further questioning of the patient revealed that he had fallen from a truck onto rocks four months prior, injuring his back and hip, but without puncture of the skin or loss of consciousness; he denied recent falls or other injuries but reported significant chronic knee pain. He had not been hospitalized recently. He had never taken corticosteroids or immunomodulatory medications. He continued to deny injection drug use. He did, however, clarify that his work with racehorses, which was originally understood to be a prior hobby, was ongoing, including recent work of cleaning the stables.

The following morning, he experienced confusion, rigors, and hypoxia, which prompted transfer to the intensive care unit (ICU).

Acute worsening during treatment is worrisome, and could be a potential complication of his infection or treatment – or even a separate process altogether. Knee pain in the setting of bacteremia raises the possibility of septic or crystal-induced arthritis and warrants imaging. Confusion and hypoxia might represent secondary sites of seeding from bacteremia (CNS infection and pneumonia, respectively) or manifestations of endocarditis, the latter being unusual for Serratia. An echocardiogram should be obtained. Other neurologic causes, including seizure, should also be considered. Further evaluation by chest imaging and repeat neurologic examination and imaging should be performed. Emergence of resistance during therapy is a theoretical concern with Serratia as an AmpC beta-lactamase-containing organism. While awaiting additional microbiology data, an empiric change to an AmpC beta-lactamase stable antibiotic such as a carbapenem should be made, especially since he has clinically deteriorated on therapy with a β-lactamase susceptible antibiotic, raising concerns of the emergence of resistance on initial therapy.

Antibiotics were changed to meropenem, vancomycin, and metronidazole given the clinical worsening and concerns that this represented infection unresponsive to prior antibiotics. The acute episode resolved spontaneously after one hour. His neurologic examination remained nonfocal. Chest radiography, urinalysis, urine culture, and right upper quadrant ultrasound were unremarkable. Transesophageal echocardiogram revealed no heart valve vegetations. MRI and bone scan of the lower extremities did not show any evidence of septic arthritis or other infection. He remained stable and was transferred out of the ICU the following day. Antibiotic coverage was switched to cefepime. On discussion with his significant other, this event was found to be similar to the intermittent confusion that occurred in the days prior to admission.

The acute onset and other features of these intermittent periods of deterioration are compatible with infection; intermittent seeding of the blood with microbes or their products (eg, lipopolysaccharides) from an abscess or vascular infection could explain these episodes. Some of the previous hypotheses to explain the episodes, such as a secondary infectious process, have not been supported by diagnostic testing or the clinical course. He needs close clinical monitoring and interval assessment of the known sites of infection.

Ten days after osteomyelitis and discitis were diagnosed, the patient developed worsening low back pain, prompting repeat spine MRI. This was significant for bilateral psoas abscess enlargement and extension of osteomyelitis and discitis (Figure 3). He was re-evaluated by Neurosurgery and Interventional Radiology and underwent psoas abscess drainage; abscess cultures grew S. marcescens.

He slowly improved over several weeks and was discharged to a subacute rehabilitation facility. He completed a 3.5-week course of intravenous antibiotics before leaving against medical advice. He completed eight weeks of oral trimethoprim-sulfamethoxazole and remains without long-term sequelae from the infection.

S. marcescens is a gram-negative rod in the Enterobacteriaceae family known for its red pigment. Primarily, S. marcescens causes nosocomial infections, most commonly of the respiratory and urinary tracts. However, a wide range of manifestations has been documented, including meningitis, ocular infections (conjunctivitis, keratitis, endophthalmitis), endocarditis, skin infections (cellulitis, necrotizing fasciitis), and osteomyelitis.^{1, 2} S. marcescens is often reported as the cause of outbreaks in ICUs;^3-6 infection is thought to occur via contamination of water pipes, hospital equipment, and disinfectants.^{3, 7} Its natural environment includes soil, water, and GI tracts of animals,⁴ and there are published reports of S. marcescens infection in horses.^{8, 9} This patient was most likely exposed to S. marcescens through his work with horses and their environment.

S. marcescens has wide-ranging target organs, and successful treatment can be difficult. S. marcescens can infect the renal, respiratory, gastrointestinal, ocular, cardiovascular, and musculoskeletal systems. S. marcescens, like other “SPACE” organisms (Serratia, Pseudomonas, Acinetobacter, Citrobacter, Enterobacter), expresses inducible AmpC beta-lactamase.¹⁰ At baseline, AmpC beta-lactamase expression is repressed.¹¹ Mutants with stably de-repressed (constitutively expressed) AmpC can be selected during therapy and lead to clinical failure, as has been best described during therapy for Enterobacter infections.¹² Infectious Disease consultation may be helpful when caring for patients with S. marcescens bacteremia given these complexities.

This was an unusual case of S. marcescens infection. It most commonly infects immunocompromised hosts. Reported risk factors include solid organ or hematopoietic stem cell transplant, malignancy, HIV/AIDS, and receipt of immunosuppressive agents. The patient did not have these risk factors, but did have well-controlled type 2 diabetes mellitus. Although diabetes is associated with an increased risk of infection and more severe infections,^{13, 14} there is no evidence in the literature that well-controlled type 2 diabetes mellitus compromises the immune system. A few case reports document cutaneous S. marcescens infection in immunocompetent adults.^15,16 A case report of S. marcescens septic arthritis and adjacent osteomyelitis has also been published, but the patient had poorly controlled diabetes.¹⁷ This case provides a report of systemic S. marcescens infection in an individual without clear risk factors.

S. marcescens osteomyelitis is rare, and there have been only a few prior case reports.^2,18 The presentation of osteomyelitis, regardless of the causative organism, is subtle, often insidious, and can easily be missed. Hospitalists should have a high index of suspicion for the diagnosis as it requires prompt evaluation and treatment for complications, including epidural abscess. Risk factors include diabetes mellitus, rheumatoid arthritis, injection drug use, and other immunocompromising illnesses.¹⁹ Degenerative changes in the spine such as osteoarthritis may be risk factors as well,²⁰ though not well studied or quantified. A hypothesized mechanism involves local inflammation and joint damage, leaving the area susceptible to bacterial seeding. Osteoarthritis and degenerative disc disease, along with exposure to racehorses, likely put this patient at risk for bacterial seeding in the vertebrae, ultimately leading to a “dark horse” diagnosis.

• Serratia marcescens is a gram-negative rod bacterium that most commonly infects immunocompromised individuals in hospital settings. This report demonstrates that S. marcescens can cause serious infection in immunocompetent, nonhospitalized adults.
• S. marcescens bacteremia or infection of organs outside of the urinary or respiratory systems is uncommon, and therapy can be complicated by emergence of resistance.
• The clinical presentation of vertebral osteomyelitis and discitis and psoas abscess can be subtle and may present without typical signs and symptoms of infection.

ACKNOWLEDGEMENTS

The authors thank the patient and his partner for their willingness to have his story published, Laura Petersen, MHSA, for providing assistance with references and manuscript editing, and Shadi Azar, MBBS, for assistance in selecting the cross-sectional images.

Disclosures

The authors have no conflicts of interest to disclose.

A 73-year-old man presented to the emergency department in late winter with fevers, myalgias, fatigue, low back pain, and poor oral intake. Four days earlier, he had fallen and hit his head. His partner also noticed a few episodes of confusion in the days leading up to presentation.

The patient’s symptoms are nonspecific. Fevers prompt the consideration of systemic infection, though fevers can also be seen in a broad range of noninfectious processes, including malignancy, vasculitis, autoimmune conditions, endocrinopathies, and drug reaction. The clinical picture warrants prompt and comprehensive evaluation, beginning with further detailed history (current illnesses, exposures, travel, vaccinations, medications, cancer screenings, weight change) and a careful physical examination, which will help guide laboratory testing and imaging.

His past medical history was notable for coronary artery disease for which he underwent coronary artery bypass grafting five years prior, hypertension, hyperlipidemia, diet-controlled type 2 diabetes mellitus, gastroesophageal reflux disease, osteoarthritis leading to chronic knee and hand pain, and a history of mildly low testosterone levels. His medications included hydrocodone and acetaminophen, metoprolol tartrate, omeprazole, topical testosterone gel (prescribed for daily use, used intermittently), and aspirin. He was retired and lived in rural Michigan with his female partner. He previously worked as a truck driver and used to train racehorses. He had quit smoking five years earlier. He denied alcohol or injection drug use.

The patient has significant underlying medical conditions. Considering infectious causes of his symptoms, it is notable that he has no reported immunodeficiency. It would be relevant to know if he has been tested for HIV. His rural residence and work with horses raise the possibility of zoonotic infections, including plague (Yersinia pestis), brucellosis (Brucella species), Q fever (Coxiella burnetti), Rhodococcus equi, or group C or G Streptococci. Information about tuberculosis risk factors, other geographic exposures, recent dental work, and ill contacts might be helpful to elucidate the causes of this nonspecific febrile illness with a possible CNS component. With regard to malignancy, it would be helpful to ask about recent weight loss, lymphadenopathy, and prior cancer screenings. Considering other etiologies, he does not report a history of autoimmune or endocrine conditions. However, it is important to consider a vasculitis, such as giant cell arteritis or polyarteritis nodosa, autoimmune conditions, and endocrinopathies such as thyrotoxicosis. The differential diagnosis for his clinical syndrome remains broad.

Vital signs were temperature 37.3°C, heart rate 88 beats per minute, respiratory rate 18 breaths per minute, blood pressure 105/64 mmHg, and oxygen saturation 93% on room air. Oral examination revealed poor dentition. The heart had a normal rate and regular rhythm with no murmurs, rubs, or gallops, and lungs were clear to auscultation bilaterally. The abdomen was unremarkable. Examination of the back was notable for mild tenderness to palpation over the sacrum. He was oriented to person, place, and time, with intact cranial nerves and a nonfocal neurologic examination. The remainder of his examination was normal. The white blood cell (WBC) count was 11.1 × 10³/μL, with 84% neutrophils and 9% bands, hemoglobin 13.6 g/dL, platelet count 54 × 10³/μL, sodium 122 mmol/L, potassium 3.3 mmol/L, chloride 89 mmol/L, bicarbonate 21 mmol/L, creatinine 1.64 mg/dL, albumin 2.7 g/dL, alkaline phosphatase 136 U/L, AST 60 U/L, ALT 37 U/L, and total bilirubin 2.1 mg/dL.

He had presented to the emergency department five days earlier with fever, flank pain, nausea, vomiting, and weakness. At that time, he had a temperature of 38.2°C, but vital signs otherwise had been normal. Laboratory studies had revealed WBC count 14.0 × 10³/μL, hemoglobin 13.7 g/dL, platelet count 175 × 10³/μL, sodium 129 mmol/L, chloride 97 mmol/dL, bicarbonate 23 mmol/L, creatinine 1.1 mg/dL, and total bilirubin 1.6 mg/dL. Urinalysis had been negative. He had received one liter of intravenous normal saline and ketorolac for pain and had been discharged with the diagnosis of a viral illness.

A picture of a progressive, subacute illness with multisystem involvement appears to be emerging, and there are several abnormalities consistent with infection, including fever, leukocytosis with bandemia, thrombocytopenia, renal dysfunction, and elevated bilirubin. His borderline hypotension may be due to uninterrupted use of his antihypertensive medication in the setting of poor oral intake or may indicate incipient sepsis. Focal sacral tenderness raises the possibility of vertebral osteomyelitis or epidural abscess, either from a contiguous focus of infection from the surrounding structures, or as a site of seeding from bacteremia. His prior confusion episodes might have been secondary to a systemic process; however, CNS imaging should be done, given the history of confusion and recent fall. Further diagnostic studies are warranted, including: blood cultures; peripheral blood smear; imaging of the spine, chest, abdomen, and pelvis; electrocardiogram; and possibly echocardiogram. Although noninfectious etiologies should not be discounted, the constellation of findings is more compatible with infection.

Two sets of blood cultures and a viral respiratory swab were obtained. Computed tomography (CT) of the head without contrast was negative for acute bleeding or other intracranial pathology. Lumbosacral radiography revealed degenerative changes with intact alignment of the sacrum. The patient was admitted with plans to pursue lumbar puncture if altered mental status recurred. The viral swab was negative. Within 24 hours, one set of blood cultures (both bottles) grew lactose-negative, oxidase-negative, gram-negative rods.

Gram-negative rods (GNRs) rarely are contaminants in blood cultures and should be considered significant until proven otherwise. Prompt empiric therapy and investigation to identify the primary source of bacteremia must be initiated. Although the most common GNRs isolated from blood cultures are enteric coliform organisms such as E. coli, Klebsiella, and Enterobacter, these typically are lactose-positive. Additional possibilities should be considered, including Salmonella species or other organisms comprising the “HACEK” group. This latter group is commonly associated with endocarditis, but the majority are oxidase-positive and have more fastidious growth requirements. Although there are other gram-negative organisms to consider, they have other distinguishing characteristics that have not been indicated in the microbiology results. Broad-spectrum antibiotic therapy is appropriate while awaiting the final identification of the GNR. A thorough search for a primary source and secondary sites of hematogenous seeding should be conducted. His only localizing symptom was tenderness over the sacrum, and this should be further assessed by sensitive imaging such as magnetic resonance imaging (MRI). The identity of the GNR would guide further diagnostic evaluation. For example, a respiratory organism such as Haemophilus influenzae would prompt a CT scan of the chest. Isolation of an enteric or a coliform GNR such as E. coli would prompt abdominal and pelvic imaging to assess for occult abscess. An “HACEK” group organism would prompt echocardiography to evaluate for endocarditis.

He was started on piperacillin–tazobactam. GNR bacteremia without a clear source prompted a CT of the chest, abdomen, and pelvis with and without contrast. The images were unremarkable, with the exception of a signal abnormality in the left psoas muscle concerning for abscess (Figure 1). MRI of the same region revealed L2-4 osteomyelitis and discitis with bilateral psoas abscesses but without epidural abscess (Figure 2).

Because of increasing reports of antibiotic resistance in GNRs, even in community-acquired infections, it is appropriate to initially treat with a broad-spectrum antibiotic such as a fourth-generation cephalosporin or carbapenem while awaiting identification and susceptibility results to guide definitive therapy. In addition to antimicrobial therapy, treatment of psoas abscess usually requires drainage. Vertebral osteomyelitis from a hematogenous source can often be treated with antibiotics alone, as long as there are no associated complications such as epidural abscess and spine instability. Imaging should be reviewed for pathology of the surrounding structures, and surgical consultation should be obtained.

Neurosurgery, Interventional Radiology, and Infectious Disease services were consulted. Antibiotic coverage was expanded to vancomycin, cefepime, and metronidazole due to the possibility of polymicrobial infection. No surgical intervention was recommended since the abscesses were too small to drain.

The next day, the GNR was identified as Serratia marcescens.

S. marcescens is a widely distributed organism in the environment, but not a common component of endogenous human flora. Serratia is generally considered as an opportunistic nosocomial pathogen. Community-acquired infection with this organism is unusual and implies exogenous acquisition. A careful re-evaluation of exposures, including injection drug use or other parenteral exposures is important to identify the likely source of infection, as these have been previously linked to outbreaks of environmental organisms. Based on the presumed pathogenesis of infection and the initial microbiology suggesting monomicrobial Serratia infection, antibiotics should be narrowed based on the susceptibility results. There is concern that antibiotics might not adequately penetrate the abscesses and result in a lack of clinical improvement and/or lead to the emergence of antibiotic resistance during therapy. This is an important concern with Serratia, which typically harbors an AmpC beta-lactamase that can mediate resistance to broad-spectrum cephalosporins. If medical therapy alone without drainage is planned, short-interval re-imaging is warranted.

Blood cultures from days two and three of hospitalization also grew S. marcescens. No other organisms grew. Based on culture sensitivity data, antibiotics were narrowed to ceftriaxone.

This surprising culture result prompted the medical team to obtain screening laboratory tests for immunocompromising conditions and to revisit the patient’s history. His type 2 diabetes mellitus was well controlled with a hemoglobin A1c of 6.5%. HIV testing was negative. Further questioning of the patient revealed that he had fallen from a truck onto rocks four months prior, injuring his back and hip, but without puncture of the skin or loss of consciousness; he denied recent falls or other injuries but reported significant chronic knee pain. He had not been hospitalized recently. He had never taken corticosteroids or immunomodulatory medications. He continued to deny injection drug use. He did, however, clarify that his work with racehorses, which was originally understood to be a prior hobby, was ongoing, including recent work of cleaning the stables.

The following morning, he experienced confusion, rigors, and hypoxia, which prompted transfer to the intensive care unit (ICU).

Acute worsening during treatment is worrisome, and could be a potential complication of his infection or treatment – or even a separate process altogether. Knee pain in the setting of bacteremia raises the possibility of septic or crystal-induced arthritis and warrants imaging. Confusion and hypoxia might represent secondary sites of seeding from bacteremia (CNS infection and pneumonia, respectively) or manifestations of endocarditis, the latter being unusual for Serratia. An echocardiogram should be obtained. Other neurologic causes, including seizure, should also be considered. Further evaluation by chest imaging and repeat neurologic examination and imaging should be performed. Emergence of resistance during therapy is a theoretical concern with Serratia as an AmpC beta-lactamase-containing organism. While awaiting additional microbiology data, an empiric change to an AmpC beta-lactamase stable antibiotic such as a carbapenem should be made, especially since he has clinically deteriorated on therapy with a β-lactamase susceptible antibiotic, raising concerns of the emergence of resistance on initial therapy.

Antibiotics were changed to meropenem, vancomycin, and metronidazole given the clinical worsening and concerns that this represented infection unresponsive to prior antibiotics. The acute episode resolved spontaneously after one hour. His neurologic examination remained nonfocal. Chest radiography, urinalysis, urine culture, and right upper quadrant ultrasound were unremarkable. Transesophageal echocardiogram revealed no heart valve vegetations. MRI and bone scan of the lower extremities did not show any evidence of septic arthritis or other infection. He remained stable and was transferred out of the ICU the following day. Antibiotic coverage was switched to cefepime. On discussion with his significant other, this event was found to be similar to the intermittent confusion that occurred in the days prior to admission.

The acute onset and other features of these intermittent periods of deterioration are compatible with infection; intermittent seeding of the blood with microbes or their products (eg, lipopolysaccharides) from an abscess or vascular infection could explain these episodes. Some of the previous hypotheses to explain the episodes, such as a secondary infectious process, have not been supported by diagnostic testing or the clinical course. He needs close clinical monitoring and interval assessment of the known sites of infection.

Ten days after osteomyelitis and discitis were diagnosed, the patient developed worsening low back pain, prompting repeat spine MRI. This was significant for bilateral psoas abscess enlargement and extension of osteomyelitis and discitis (Figure 3). He was re-evaluated by Neurosurgery and Interventional Radiology and underwent psoas abscess drainage; abscess cultures grew S. marcescens.

He slowly improved over several weeks and was discharged to a subacute rehabilitation facility. He completed a 3.5-week course of intravenous antibiotics before leaving against medical advice. He completed eight weeks of oral trimethoprim-sulfamethoxazole and remains without long-term sequelae from the infection.

S. marcescens is a gram-negative rod in the Enterobacteriaceae family known for its red pigment. Primarily, S. marcescens causes nosocomial infections, most commonly of the respiratory and urinary tracts. However, a wide range of manifestations has been documented, including meningitis, ocular infections (conjunctivitis, keratitis, endophthalmitis), endocarditis, skin infections (cellulitis, necrotizing fasciitis), and osteomyelitis.^{1, 2} S. marcescens is often reported as the cause of outbreaks in ICUs;^3-6 infection is thought to occur via contamination of water pipes, hospital equipment, and disinfectants.^{3, 7} Its natural environment includes soil, water, and GI tracts of animals,⁴ and there are published reports of S. marcescens infection in horses.^{8, 9} This patient was most likely exposed to S. marcescens through his work with horses and their environment.

S. marcescens has wide-ranging target organs, and successful treatment can be difficult. S. marcescens can infect the renal, respiratory, gastrointestinal, ocular, cardiovascular, and musculoskeletal systems. S. marcescens, like other “SPACE” organisms (Serratia, Pseudomonas, Acinetobacter, Citrobacter, Enterobacter), expresses inducible AmpC beta-lactamase.¹⁰ At baseline, AmpC beta-lactamase expression is repressed.¹¹ Mutants with stably de-repressed (constitutively expressed) AmpC can be selected during therapy and lead to clinical failure, as has been best described during therapy for Enterobacter infections.¹² Infectious Disease consultation may be helpful when caring for patients with S. marcescens bacteremia given these complexities.

This was an unusual case of S. marcescens infection. It most commonly infects immunocompromised hosts. Reported risk factors include solid organ or hematopoietic stem cell transplant, malignancy, HIV/AIDS, and receipt of immunosuppressive agents. The patient did not have these risk factors, but did have well-controlled type 2 diabetes mellitus. Although diabetes is associated with an increased risk of infection and more severe infections,^{13, 14} there is no evidence in the literature that well-controlled type 2 diabetes mellitus compromises the immune system. A few case reports document cutaneous S. marcescens infection in immunocompetent adults.^15,16 A case report of S. marcescens septic arthritis and adjacent osteomyelitis has also been published, but the patient had poorly controlled diabetes.¹⁷ This case provides a report of systemic S. marcescens infection in an individual without clear risk factors.

S. marcescens osteomyelitis is rare, and there have been only a few prior case reports.^2,18 The presentation of osteomyelitis, regardless of the causative organism, is subtle, often insidious, and can easily be missed. Hospitalists should have a high index of suspicion for the diagnosis as it requires prompt evaluation and treatment for complications, including epidural abscess. Risk factors include diabetes mellitus, rheumatoid arthritis, injection drug use, and other immunocompromising illnesses.¹⁹ Degenerative changes in the spine such as osteoarthritis may be risk factors as well,²⁰ though not well studied or quantified. A hypothesized mechanism involves local inflammation and joint damage, leaving the area susceptible to bacterial seeding. Osteoarthritis and degenerative disc disease, along with exposure to racehorses, likely put this patient at risk for bacterial seeding in the vertebrae, ultimately leading to a “dark horse” diagnosis.

• Serratia marcescens is a gram-negative rod bacterium that most commonly infects immunocompromised individuals in hospital settings. This report demonstrates that S. marcescens can cause serious infection in immunocompetent, nonhospitalized adults.
• S. marcescens bacteremia or infection of organs outside of the urinary or respiratory systems is uncommon, and therapy can be complicated by emergence of resistance.
• The clinical presentation of vertebral osteomyelitis and discitis and psoas abscess can be subtle and may present without typical signs and symptoms of infection.

ACKNOWLEDGEMENTS

The authors thank the patient and his partner for their willingness to have his story published, Laura Petersen, MHSA, for providing assistance with references and manuscript editing, and Shadi Azar, MBBS, for assistance in selecting the cross-sectional images.

Disclosures

The authors have no conflicts of interest to disclose.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care, but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Bedside thoracentesis can cause serious complications, such as pneumothorax, re-expansion pulmonary edema, or hemorrhage. These rare complications have led many hospitalists to routinely order chest radiographs (CXRs) following thoracentesis. However, post-thoracentesis CXRs are usually not indicated and can lead to unnecessary radiation exposure and expense. Rather than obtaining routine CXRs, hospitalists should use postprocedural signs and symptoms to identify the occasional patients who require imaging. A risk-stratified approach is a safe and cost-effective way to avoid unnecessary radiographs.

A 52-year-old man with decompensated liver disease and hepatic hydrothorax is hospitalized for increasing dyspnea caused by a recurrent pleural effusion. Diuretics do not improve his dyspnea, and his hospitalist recommends a therapeutic thoracentesis for symptom relief. The patient does not have any significant procedural risk factors: He does not have preexisting pulmonary or pleural disease, his platelet count is 105,000 × 10³/µl, and his international normalized ratio is 1.3. Bedside sonography demonstrates a large, free-flowing, right-sided pleural effusion. The hospitalist performs an uncomplicated ultrasound-guided removal of 1.5 L of straw-colored fluid with a catheter-over-needle kit. The patient does not have any pain or increased shortness of breath during or after the procedure. The hospitalist reflexively orders a routine chest radiograph to assess for pneumothorax.

Pleural effusions are newly diagnosed in more than 1.5 million Americans annually,¹ and hospitalists frequently care for patients requiring thoracentesis. Internal medicine residents traditionally learn to perform this procedure during residency, and thoracentesis remains a common task for both residents and hospitalists.² Patients typically tolerate thoracentesis well, but they can develop serious complications such as pneumothorax, re-expansion pulmonary edema, or hemothorax. Before the advent of bedside ultrasound, these complications occurred relatively commonly; a 2010 systematic review, for example, found that the rate of pneumothorax from thoracentesis performed without ultrasound was 9.3%.³ Other studies have identified even higher rates of complications, including two case series in which investigators found a 14% rate of major complications⁴ and a pneumothorax rate of nearly 30%.⁵ Postprocedure radiographs became common practice because of the high rate of complications, and this practice has persisted for many practitioners despite the substantial safety improvements introduced by bedside ultrasonography.⁶

Hospitalists might think routine CXRs are helpful after ultrasound-guided thoracentesis for additional reasons. First, modern guidelines reflecting the low risk of complications after ultrasound-guided procedures have not been released by United States pulmonary medicine societies, and some clinicians may continue to follow practices acquired during the era of unguided thoracentesis. Second, performing postprocedure imaging has become ingrained as a standard part of some institutional procedure checklists⁶ and some prominent textbooks continue to recommend the practice.⁷ For some hospitalists, this testing reflex may be reinforced by other common procedures, such as placing a nasogastric tube or a central venous catheter, for which a postprocedure CXR is standard practice. Thus, ordering postprocedure imaging can become internalized as the safe, checklist-based final step of a procedure. Third, hospitalists may order a postprocedure CXR for reasons other than detecting procedural complications. The pleural effusion might be thought to obscure a parenchymal or endobronchial lesion for which a postprocedure CXR may reveal an important finding. Finally, a CXR also may also satisfy the clinician’s curiosity regarding the completeness of drainage.

A routine post-thoracentesis CXR is not necessary for three reasons. First, the use of ultrasound marking or guidance has substantially improved site selection and reduced the rate of complications for experienced operators. For example, a 2010 systematic review found an overall rate of pneumothorax of 4% for ultrasound-guided procedures performed between 1986 and 2006,³ whereas more recently published data suggest the current rate of pneumothorax is closer to 1% when ultrasound marking or guidance is used.^8,9 One study of 462 consecutive patients with malignant pleural effusions, for example, showed that the rate of pneumothorax with ultrasound-guided needle-over-catheter thoracentesis was 0.97% (3/310 patients), compared with a rate of 8.89% (12/135 patients) when the procedure was performed without ultrasound.⁹ Another prospective, randomized study of 160 patients with various causes of pleural effusion showed that the rate of pneumothorax with ultrasound-marked thoracentesis was 1.25% (1/80 patients), compared with 12.5% (10/80 patients) for procedures performed without ultrasound.⁸ Hospitalists who competently use ultrasound guidance should act on modern estimates of complications and may also choose to incorporate postprocedure ultrasound into their practice. Indeed, the Society of Hospital Medicine recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized on postprocedure ultrasound.¹⁰

Second, procedural factors and postprocedural symptoms (new chest pain, dyspnea, or persistent cough) reliably identify patients with high risk of clinically meaningful complications. On one hand, only 1% to 2% of asymptomatic patients have a postprocedure pneumothorax, and clinical monitoring does not lead to chest tube placement in almost all of these cases.¹¹ On the other hand, 67% to 72% of symptomatic patients are found to have complications.¹² Doyle et al¹³ showed that the use of symptoms and procedure-specific factors (such as the aspiration of air, difficult procedure, multiple needle passes, or high operator suspicion of pneumothorax) could obviate the need for routine CXRs in approximately 60% of their procedures without any serious consequences.

Third, postprocedural CXRs very rarely reveal new or unexpected findings. For example, in one series,¹² only 3.8% of postdrainage radiographs uncovered new findings, none of which clarified the underlying diagnosis or changed management. To assess the utility of an initial thoracentesis and decide about repeat procedures, begin by asking the patient about symptoms and perform a physical exam.

CXRs might be helpful in certain scenarios, even when a complication is not suspected. For example, a postprocedure CXR to detect nonexpandable lung or evaluate the rate of recurrence may guide definitive management of patients with recurrent or malignant pleural effusion. Determining completeness of drainage may also assist with planning for palliative measures such as pleurodesis or indwelling pleural catheter placement. A postprocedure CXR is also helpful in patients with a technically difficult procedure or in those with symptoms during or immediately after the procedure. This recommendation is consistent with the 2010 British Thoracic Society guidelines, which recommend CXRs for procedures where air was withdrawn, the procedure was difficult, multiple needle passes were required, or the patient became symptomatic.¹⁴ The Society of Hospital Medicine’s recent Position Statement concurs with these guidelines and recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized by postprocedure ultrasound.¹⁰

Hospitalists should not rountinely obtain post-thoracentesis CXRs in asymptomatic patients. Clinical monitoring with subsequent symptom-guided evaluation lowers costs, avoids unnecessary radiation exposure, and has been shown to be successful in a large case series of more than 9,300 patients.¹⁵ Some coughing should be expected with all large-volume thoracenteses as a normal response to re-expansion of atelectatic lung. The coughing should not persist past the immediate postprocedure period. If symptoms arise or if a complication is expected, the test of choice is either CXR or, if the hospitalist is a competent sonographer, bedside sonography. Bedside sonography is a low-cost, noninvasive method and has been well studied in the diagnosis of post-thoracentesis pneumothorax.¹⁶ CXRs may still be needed to confirm findings by sonography, to investigate postprocedural symptoms in those with pleural adhesions or other lung/pleural diseases (because ultrasonography is less reliable in these patients), or if reexpansion pulmonary edema or other complications are suspected. A robust quality improvement strategy to reduce unnecessary post-thoracentesis CXRs could result in cost savings and spare patients from radiation exposure, because a recent study of almost 1,000 thoracenteses performed at an academic medical center demonstrated that internal medicine residents, pulmonologists, and interventional radiologists order a CXR following 95% of thoracenteses.¹⁷ For a hypothetical hospital that orders 100 unnecessary post-thoracentesis CXRs annually, hospitalists could avoid approximately $7,000 in wasted expense per year.¹⁸

• Do not routinely order post-thoracentesis CXRs.
• Order a post-thoracentesis CXR if (1) the patient had new chest pain, dyspnea, or persistent cough during or after the procedure; (2) procedural features suggest increased risk of a complication (multiple needle passes, aspiration of air, difficulty obtaining fluid); or (3) a definitive palliative procedure will be arranged based on lung expansion.
• If qualified, use bedside sonography as a first step in the diagnosis of pneumothorax, reserving CXRs for those patients in whom accurate sonography is not possible, an alternative diagnosis is suspected, or when sonography findings are equivocal.

Following the uncomplicated thoracentesis, the hospitalist reconsidered the initial decision to order a CXR and rapidly assessed the patient’s risk of complications. Because the procedure required only one needle pass, air was not aspirated, and the patient did not experience prolonged coughing or pain, the CXR order was canceled. The patient recovered uneventfully and was spared the cost and radiation associated with the proposed CXR.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].cknowledgments

Acknowledgements

The authors would like to thank Patricia Kritek and Somnath Mookherjee for their comments on an early version of this manuscript.

Disclosures

The authors have nothing to disclose.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care, but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Bedside thoracentesis can cause serious complications, such as pneumothorax, re-expansion pulmonary edema, or hemorrhage. These rare complications have led many hospitalists to routinely order chest radiographs (CXRs) following thoracentesis. However, post-thoracentesis CXRs are usually not indicated and can lead to unnecessary radiation exposure and expense. Rather than obtaining routine CXRs, hospitalists should use postprocedural signs and symptoms to identify the occasional patients who require imaging. A risk-stratified approach is a safe and cost-effective way to avoid unnecessary radiographs.

A 52-year-old man with decompensated liver disease and hepatic hydrothorax is hospitalized for increasing dyspnea caused by a recurrent pleural effusion. Diuretics do not improve his dyspnea, and his hospitalist recommends a therapeutic thoracentesis for symptom relief. The patient does not have any significant procedural risk factors: He does not have preexisting pulmonary or pleural disease, his platelet count is 105,000 × 10³/µl, and his international normalized ratio is 1.3. Bedside sonography demonstrates a large, free-flowing, right-sided pleural effusion. The hospitalist performs an uncomplicated ultrasound-guided removal of 1.5 L of straw-colored fluid with a catheter-over-needle kit. The patient does not have any pain or increased shortness of breath during or after the procedure. The hospitalist reflexively orders a routine chest radiograph to assess for pneumothorax.

Pleural effusions are newly diagnosed in more than 1.5 million Americans annually,¹ and hospitalists frequently care for patients requiring thoracentesis. Internal medicine residents traditionally learn to perform this procedure during residency, and thoracentesis remains a common task for both residents and hospitalists.² Patients typically tolerate thoracentesis well, but they can develop serious complications such as pneumothorax, re-expansion pulmonary edema, or hemothorax. Before the advent of bedside ultrasound, these complications occurred relatively commonly; a 2010 systematic review, for example, found that the rate of pneumothorax from thoracentesis performed without ultrasound was 9.3%.³ Other studies have identified even higher rates of complications, including two case series in which investigators found a 14% rate of major complications⁴ and a pneumothorax rate of nearly 30%.⁵ Postprocedure radiographs became common practice because of the high rate of complications, and this practice has persisted for many practitioners despite the substantial safety improvements introduced by bedside ultrasonography.⁶

Hospitalists might think routine CXRs are helpful after ultrasound-guided thoracentesis for additional reasons. First, modern guidelines reflecting the low risk of complications after ultrasound-guided procedures have not been released by United States pulmonary medicine societies, and some clinicians may continue to follow practices acquired during the era of unguided thoracentesis. Second, performing postprocedure imaging has become ingrained as a standard part of some institutional procedure checklists⁶ and some prominent textbooks continue to recommend the practice.⁷ For some hospitalists, this testing reflex may be reinforced by other common procedures, such as placing a nasogastric tube or a central venous catheter, for which a postprocedure CXR is standard practice. Thus, ordering postprocedure imaging can become internalized as the safe, checklist-based final step of a procedure. Third, hospitalists may order a postprocedure CXR for reasons other than detecting procedural complications. The pleural effusion might be thought to obscure a parenchymal or endobronchial lesion for which a postprocedure CXR may reveal an important finding. Finally, a CXR also may also satisfy the clinician’s curiosity regarding the completeness of drainage.

A routine post-thoracentesis CXR is not necessary for three reasons. First, the use of ultrasound marking or guidance has substantially improved site selection and reduced the rate of complications for experienced operators. For example, a 2010 systematic review found an overall rate of pneumothorax of 4% for ultrasound-guided procedures performed between 1986 and 2006,³ whereas more recently published data suggest the current rate of pneumothorax is closer to 1% when ultrasound marking or guidance is used.^8,9 One study of 462 consecutive patients with malignant pleural effusions, for example, showed that the rate of pneumothorax with ultrasound-guided needle-over-catheter thoracentesis was 0.97% (3/310 patients), compared with a rate of 8.89% (12/135 patients) when the procedure was performed without ultrasound.⁹ Another prospective, randomized study of 160 patients with various causes of pleural effusion showed that the rate of pneumothorax with ultrasound-marked thoracentesis was 1.25% (1/80 patients), compared with 12.5% (10/80 patients) for procedures performed without ultrasound.⁸ Hospitalists who competently use ultrasound guidance should act on modern estimates of complications and may also choose to incorporate postprocedure ultrasound into their practice. Indeed, the Society of Hospital Medicine recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized on postprocedure ultrasound.¹⁰

Second, procedural factors and postprocedural symptoms (new chest pain, dyspnea, or persistent cough) reliably identify patients with high risk of clinically meaningful complications. On one hand, only 1% to 2% of asymptomatic patients have a postprocedure pneumothorax, and clinical monitoring does not lead to chest tube placement in almost all of these cases.¹¹ On the other hand, 67% to 72% of symptomatic patients are found to have complications.¹² Doyle et al¹³ showed that the use of symptoms and procedure-specific factors (such as the aspiration of air, difficult procedure, multiple needle passes, or high operator suspicion of pneumothorax) could obviate the need for routine CXRs in approximately 60% of their procedures without any serious consequences.

Third, postprocedural CXRs very rarely reveal new or unexpected findings. For example, in one series,¹² only 3.8% of postdrainage radiographs uncovered new findings, none of which clarified the underlying diagnosis or changed management. To assess the utility of an initial thoracentesis and decide about repeat procedures, begin by asking the patient about symptoms and perform a physical exam.

CXRs might be helpful in certain scenarios, even when a complication is not suspected. For example, a postprocedure CXR to detect nonexpandable lung or evaluate the rate of recurrence may guide definitive management of patients with recurrent or malignant pleural effusion. Determining completeness of drainage may also assist with planning for palliative measures such as pleurodesis or indwelling pleural catheter placement. A postprocedure CXR is also helpful in patients with a technically difficult procedure or in those with symptoms during or immediately after the procedure. This recommendation is consistent with the 2010 British Thoracic Society guidelines, which recommend CXRs for procedures where air was withdrawn, the procedure was difficult, multiple needle passes were required, or the patient became symptomatic.¹⁴ The Society of Hospital Medicine’s recent Position Statement concurs with these guidelines and recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized by postprocedure ultrasound.¹⁰

Hospitalists should not rountinely obtain post-thoracentesis CXRs in asymptomatic patients. Clinical monitoring with subsequent symptom-guided evaluation lowers costs, avoids unnecessary radiation exposure, and has been shown to be successful in a large case series of more than 9,300 patients.¹⁵ Some coughing should be expected with all large-volume thoracenteses as a normal response to re-expansion of atelectatic lung. The coughing should not persist past the immediate postprocedure period. If symptoms arise or if a complication is expected, the test of choice is either CXR or, if the hospitalist is a competent sonographer, bedside sonography. Bedside sonography is a low-cost, noninvasive method and has been well studied in the diagnosis of post-thoracentesis pneumothorax.¹⁶ CXRs may still be needed to confirm findings by sonography, to investigate postprocedural symptoms in those with pleural adhesions or other lung/pleural diseases (because ultrasonography is less reliable in these patients), or if reexpansion pulmonary edema or other complications are suspected. A robust quality improvement strategy to reduce unnecessary post-thoracentesis CXRs could result in cost savings and spare patients from radiation exposure, because a recent study of almost 1,000 thoracenteses performed at an academic medical center demonstrated that internal medicine residents, pulmonologists, and interventional radiologists order a CXR following 95% of thoracenteses.¹⁷ For a hypothetical hospital that orders 100 unnecessary post-thoracentesis CXRs annually, hospitalists could avoid approximately $7,000 in wasted expense per year.¹⁸

• Do not routinely order post-thoracentesis CXRs.
• Order a post-thoracentesis CXR if (1) the patient had new chest pain, dyspnea, or persistent cough during or after the procedure; (2) procedural features suggest increased risk of a complication (multiple needle passes, aspiration of air, difficulty obtaining fluid); or (3) a definitive palliative procedure will be arranged based on lung expansion.
• If qualified, use bedside sonography as a first step in the diagnosis of pneumothorax, reserving CXRs for those patients in whom accurate sonography is not possible, an alternative diagnosis is suspected, or when sonography findings are equivocal.

Following the uncomplicated thoracentesis, the hospitalist reconsidered the initial decision to order a CXR and rapidly assessed the patient’s risk of complications. Because the procedure required only one needle pass, air was not aspirated, and the patient did not experience prolonged coughing or pain, the CXR order was canceled. The patient recovered uneventfully and was spared the cost and radiation associated with the proposed CXR.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].cknowledgments

Acknowledgements

The authors would like to thank Patricia Kritek and Somnath Mookherjee for their comments on an early version of this manuscript.

Disclosures

The authors have nothing to disclose.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care, but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Bedside thoracentesis can cause serious complications, such as pneumothorax, re-expansion pulmonary edema, or hemorrhage. These rare complications have led many hospitalists to routinely order chest radiographs (CXRs) following thoracentesis. However, post-thoracentesis CXRs are usually not indicated and can lead to unnecessary radiation exposure and expense. Rather than obtaining routine CXRs, hospitalists should use postprocedural signs and symptoms to identify the occasional patients who require imaging. A risk-stratified approach is a safe and cost-effective way to avoid unnecessary radiographs.

A 52-year-old man with decompensated liver disease and hepatic hydrothorax is hospitalized for increasing dyspnea caused by a recurrent pleural effusion. Diuretics do not improve his dyspnea, and his hospitalist recommends a therapeutic thoracentesis for symptom relief. The patient does not have any significant procedural risk factors: He does not have preexisting pulmonary or pleural disease, his platelet count is 105,000 × 10³/µl, and his international normalized ratio is 1.3. Bedside sonography demonstrates a large, free-flowing, right-sided pleural effusion. The hospitalist performs an uncomplicated ultrasound-guided removal of 1.5 L of straw-colored fluid with a catheter-over-needle kit. The patient does not have any pain or increased shortness of breath during or after the procedure. The hospitalist reflexively orders a routine chest radiograph to assess for pneumothorax.

Pleural effusions are newly diagnosed in more than 1.5 million Americans annually,¹ and hospitalists frequently care for patients requiring thoracentesis. Internal medicine residents traditionally learn to perform this procedure during residency, and thoracentesis remains a common task for both residents and hospitalists.² Patients typically tolerate thoracentesis well, but they can develop serious complications such as pneumothorax, re-expansion pulmonary edema, or hemothorax. Before the advent of bedside ultrasound, these complications occurred relatively commonly; a 2010 systematic review, for example, found that the rate of pneumothorax from thoracentesis performed without ultrasound was 9.3%.³ Other studies have identified even higher rates of complications, including two case series in which investigators found a 14% rate of major complications⁴ and a pneumothorax rate of nearly 30%.⁵ Postprocedure radiographs became common practice because of the high rate of complications, and this practice has persisted for many practitioners despite the substantial safety improvements introduced by bedside ultrasonography.⁶

Hospitalists might think routine CXRs are helpful after ultrasound-guided thoracentesis for additional reasons. First, modern guidelines reflecting the low risk of complications after ultrasound-guided procedures have not been released by United States pulmonary medicine societies, and some clinicians may continue to follow practices acquired during the era of unguided thoracentesis. Second, performing postprocedure imaging has become ingrained as a standard part of some institutional procedure checklists⁶ and some prominent textbooks continue to recommend the practice.⁷ For some hospitalists, this testing reflex may be reinforced by other common procedures, such as placing a nasogastric tube or a central venous catheter, for which a postprocedure CXR is standard practice. Thus, ordering postprocedure imaging can become internalized as the safe, checklist-based final step of a procedure. Third, hospitalists may order a postprocedure CXR for reasons other than detecting procedural complications. The pleural effusion might be thought to obscure a parenchymal or endobronchial lesion for which a postprocedure CXR may reveal an important finding. Finally, a CXR also may also satisfy the clinician’s curiosity regarding the completeness of drainage.

A routine post-thoracentesis CXR is not necessary for three reasons. First, the use of ultrasound marking or guidance has substantially improved site selection and reduced the rate of complications for experienced operators. For example, a 2010 systematic review found an overall rate of pneumothorax of 4% for ultrasound-guided procedures performed between 1986 and 2006,³ whereas more recently published data suggest the current rate of pneumothorax is closer to 1% when ultrasound marking or guidance is used.^8,9 One study of 462 consecutive patients with malignant pleural effusions, for example, showed that the rate of pneumothorax with ultrasound-guided needle-over-catheter thoracentesis was 0.97% (3/310 patients), compared with a rate of 8.89% (12/135 patients) when the procedure was performed without ultrasound.⁹ Another prospective, randomized study of 160 patients with various causes of pleural effusion showed that the rate of pneumothorax with ultrasound-marked thoracentesis was 1.25% (1/80 patients), compared with 12.5% (10/80 patients) for procedures performed without ultrasound.⁸ Hospitalists who competently use ultrasound guidance should act on modern estimates of complications and may also choose to incorporate postprocedure ultrasound into their practice. Indeed, the Society of Hospital Medicine recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized on postprocedure ultrasound.¹⁰

Second, procedural factors and postprocedural symptoms (new chest pain, dyspnea, or persistent cough) reliably identify patients with high risk of clinically meaningful complications. On one hand, only 1% to 2% of asymptomatic patients have a postprocedure pneumothorax, and clinical monitoring does not lead to chest tube placement in almost all of these cases.¹¹ On the other hand, 67% to 72% of symptomatic patients are found to have complications.¹² Doyle et al¹³ showed that the use of symptoms and procedure-specific factors (such as the aspiration of air, difficult procedure, multiple needle passes, or high operator suspicion of pneumothorax) could obviate the need for routine CXRs in approximately 60% of their procedures without any serious consequences.

Third, postprocedural CXRs very rarely reveal new or unexpected findings. For example, in one series,¹² only 3.8% of postdrainage radiographs uncovered new findings, none of which clarified the underlying diagnosis or changed management. To assess the utility of an initial thoracentesis and decide about repeat procedures, begin by asking the patient about symptoms and perform a physical exam.

CXRs might be helpful in certain scenarios, even when a complication is not suspected. For example, a postprocedure CXR to detect nonexpandable lung or evaluate the rate of recurrence may guide definitive management of patients with recurrent or malignant pleural effusion. Determining completeness of drainage may also assist with planning for palliative measures such as pleurodesis or indwelling pleural catheter placement. A postprocedure CXR is also helpful in patients with a technically difficult procedure or in those with symptoms during or immediately after the procedure. This recommendation is consistent with the 2010 British Thoracic Society guidelines, which recommend CXRs for procedures where air was withdrawn, the procedure was difficult, multiple needle passes were required, or the patient became symptomatic.¹⁴ The Society of Hospital Medicine’s recent Position Statement concurs with these guidelines and recommends against routine chest radiography in asymptomatic patients when sliding lung is visualized by postprocedure ultrasound.¹⁰

Hospitalists should not rountinely obtain post-thoracentesis CXRs in asymptomatic patients. Clinical monitoring with subsequent symptom-guided evaluation lowers costs, avoids unnecessary radiation exposure, and has been shown to be successful in a large case series of more than 9,300 patients.¹⁵ Some coughing should be expected with all large-volume thoracenteses as a normal response to re-expansion of atelectatic lung. The coughing should not persist past the immediate postprocedure period. If symptoms arise or if a complication is expected, the test of choice is either CXR or, if the hospitalist is a competent sonographer, bedside sonography. Bedside sonography is a low-cost, noninvasive method and has been well studied in the diagnosis of post-thoracentesis pneumothorax.¹⁶ CXRs may still be needed to confirm findings by sonography, to investigate postprocedural symptoms in those with pleural adhesions or other lung/pleural diseases (because ultrasonography is less reliable in these patients), or if reexpansion pulmonary edema or other complications are suspected. A robust quality improvement strategy to reduce unnecessary post-thoracentesis CXRs could result in cost savings and spare patients from radiation exposure, because a recent study of almost 1,000 thoracenteses performed at an academic medical center demonstrated that internal medicine residents, pulmonologists, and interventional radiologists order a CXR following 95% of thoracenteses.¹⁷ For a hypothetical hospital that orders 100 unnecessary post-thoracentesis CXRs annually, hospitalists could avoid approximately $7,000 in wasted expense per year.¹⁸

• Do not routinely order post-thoracentesis CXRs.
• Order a post-thoracentesis CXR if (1) the patient had new chest pain, dyspnea, or persistent cough during or after the procedure; (2) procedural features suggest increased risk of a complication (multiple needle passes, aspiration of air, difficulty obtaining fluid); or (3) a definitive palliative procedure will be arranged based on lung expansion.
• If qualified, use bedside sonography as a first step in the diagnosis of pneumothorax, reserving CXRs for those patients in whom accurate sonography is not possible, an alternative diagnosis is suspected, or when sonography findings are equivocal.

Following the uncomplicated thoracentesis, the hospitalist reconsidered the initial decision to order a CXR and rapidly assessed the patient’s risk of complications. Because the procedure required only one needle pass, air was not aspirated, and the patient did not experience prolonged coughing or pain, the CXR order was canceled. The patient recovered uneventfully and was spared the cost and radiation associated with the proposed CXR.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].cknowledgments

Acknowledgements

The authors would like to thank Patricia Kritek and Somnath Mookherjee for their comments on an early version of this manuscript.

Disclosures

The authors have nothing to disclose.

Cigarette smoking is the leading cause of preventable deaths in the United States.¹ Smoking contributes to several health problems that require hospitalization. Hospitalization also offers smokers an opportunity to quit because hospital policies prohibit smoking indoors while a health threat increases the motivation to quit.² Brief bedside smoking cessation counseling with follow-up contact after discharge increases postdischarge tobacco abstinence rates by 37%.² Identifying the characteristics of patients who are most likely to stop smoking after hospital discharge could identify strategies for interventions to help more smokers to succeed. It could also guide hospital clinicians’ efforts to provide effective brief messages to promote cessation by inpatients under their care during this teachable moment.

Sociodemographic factors, tobacco use, and psychological and medical factors have been associated with successful quit attempts by smokers in the general population.^3,4 Far less is known about the predictors of success in quitting smoking and maintaining abstinence after hospitalization. The characteristics associated with abstinence at postdischarge follow-up in prior studies of hospitalized smokers were male gender, greater confidence in quitting, greater readiness to quit, less nicotine dependence, and having a smoking-related illness.^5-8 However, most of the prior studies were limited to 1 geographic region,^5,6 focused only on a specific subgroup (eg, coronary patients⁹), or did not biochemically verify tobacco abstinence.⁸ In fact, to our knowledge, only one prior study has examined the predictors of quitting among a broad sample of hospital patients enrolled across multiple hospitals and biochemically verified abstinence.⁶ That study was conducted nearly two decades ago in one Midwestern state.

Thus, the present study aimed to identify factors independently associated with sustained postdischarge tobacco abstinence among hospitalized smokers who planned to quit smoking.¹⁰ Building on previous work, this study includes a large number of smokers with varied diagnoses admitted to one of three hospitals in two states, uses biochemically verified abstinence as the outcome measure, and examines multiple variables that were identified during the inpatient stay. We hypothesize that consistent with prior literature on this topic, factors independently associated with cessation in the present study will include confidence and intention to quit, degree of nicotine dependence, and a discharge diagnosis of a smoking-related disease.

We analyzed data from the Helping HAND2 Trial (HH2; NCT01714323), a randomized clinical trial conducted at the following three hospitals: Massachusetts General Hospital (MGH) in Boston, MA; University of Pittsburgh Medical Center (UPMC) in Pittsburgh, PA; and North Shore Medical Center (NSMC) in Salem, MA. Enrollment occurred from December 2012 to July 2014. The study methodology has been reported elsewhere.¹¹ This study was approved by the Institutional Review Boards of Partners HealthCare and University of Pittsburgh.

Hospital inpatients were eligible for enrollment if they were

• >18 years old, daily smokers, received smoking cessation counseling in the hospital (ie, standard of care for inpatient smokers), and planned to quit or try to quit smoking after discharge. Exclusion criteria included no access to a telephone, not speaking English, psychiatric or cognitive impairment, medical instability, or admission to obstetric or psychiatric units. All participants were offered nicotine replacement and one counseling session by a tobacco treatment specialist during hospitalization.

Participants were enrolled before discharge and randomly assigned to Sustained Care (Intervention) or Standard Care (Control) conditions.^10,11 In the Standard Care condition, participants received advice to call a free telephone quit line and a tailored recommendation for postdischarge pharmacotherapy. Participants randomized to Sustained Care received a free 30-day supply of their choice of FDA-approved tobacco cessation pharmacotherapy at hospital discharge (refillable twice) and five automated interactive voice response calls over three months postdischarge to allow them to access counseling or refill medications.

Baseline Demographic and Smoking Characteristics

A baseline survey assessed demographic variables (age, gender, race/ethnicity, education), tobacco use (cigarettes smoked per day, time to first morning cigarette,¹² other tobacco use, and prior quit attempts), intention to quit after discharge (ie,“What is your plan about smoking after you leave the hospital,” with the intent measured across four categorical response options), perceived importance of and confidence in quitting after discharge (five-point Likert scales ranging from “not at all” to “very”), and the presence of another smoker at home. Depression and anxiety symptoms were assessed using the Patient Health Questionnaire (PHQ-4¹³). Alcohol use (AUDIT-C¹⁴) and past-year use of cocaine, stimulants, opioids, and marijuana were also measured. Health insurance, length of stay, and primary discharge diagnoses were abstracted from the medical record. Smoking-related disease categories were derived from the 2014 U.S. Surgeon General’s Report.¹

Follow-up Assessment

Telephone surveys were administered by the research staff sixmonths after hospital discharge. Participants who reported past seven-day tobacco abstinence (ie, abstinence from tobacco for the past seven days reported at the 6-month call) were asked to provide a mailed saliva sample to assay for cotinine, a nicotine metabolite, to verify self-reported abstinence. Participants who reported nicotine replacement therapy use were asked to provide an in-person measurement of expired air carbon monoxide (CO) instead. Self-reported abstinence was biochemically verified if saliva cotinine was <10 ng/ml or if CO was <9 ppm.¹¹

Outcomes

The dependent variable, consistent with the parent trial, was biochemically confirmed past seven-day tobacco abstinence at six-month follow-up. Nonrespondents and those failing to provide a sample for confirmation were considered as smokers. In addition, a sensitivity analysis used complete cases only, excluding cases with missing smoking status outcomes.

Analysis

Bivariate associations of baseline predictor variables and biochemically confirmed abstinence were examined using chi-square tests for categorical variables and t tests or Wilcoxon rank sum tests for continuous variables. Using multiple logistic regression analyses, we identified variables that were independently associated with confirmed abstinence. The final models included all factors that were associated with cessation in the bivariate analysis (P < .10), factors associated with abstinence in the literature regardless of statistical significance (gender, AUDIT-C score),⁴ study site, and study condition. A two-sided p value of <.05 was considered to be statistically significant. Analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC).

Baseline characteristics of the 1,357 smokers enrolled in the trial are reported in Table 1. One-third of participants had a smoking-related discharge diagnosis. The median self-reported confidence in quitting was three on a five-point scale, and nearly half of the participants reported planning to stay abstinent after discharge. At six-month follow-up, 75% of participants completed the assessment, and seven-day tobacco abstinence was reported by 389 participants (29%) and biochemically confirmed in 218 participants (16%).

Results of the multiple logistic regression analysis predicting biochemically confirmed abstinence at six months are presented in Table 2. Factors independently associated with confirmed abstinence were a smoking-related primary discharge diagnosis (AOR = 1.98, 95% CI: 1.41-2.77), greater confidence in the ability to quit smoking (AOR = 1.31, 95% CI: 1.07-1.60), and stronger intention to quit (plan to stay abstinent after discharge vs. try to stay abstinent; AOR = 1.68, 95% CI: 1.19-2.38). Similar variables emerged as independent predictors of abstinence when the analysis was limited to complete cases, with an exception that one additional predictor, time to first cigarette after 30 minutes of waking, had statistical significance at the 0.05 level (Table 2).

We examined the associations between factors that were identifiable in the hospital and postdischarge tobacco abstinence among a general sample of hospitalized patients enrolled in a smoking cessation trial. The odds of biochemically confirmed abstinence at six months were higher among participants who reported higher levels of confidence in quitting smoking, those reporting having a definite plan to quit (vs. try to) after discharge, and those with a smoking-related primary discharge diagnosis.

Our findings are largely consistent with the prior literature on this topic, which has demonstrated that increased confidence in quitting, having a plan to quit smoking, and the presence of a smoking-related disease are associated with quit success at follow-up among hospital patients as well as in the general adult population.^3-7 Our finding that nicotine dependence predicted quit success in the complete case analysis, but not when imputing smoking status, aligns with prior studies of hospitalized smokers, which have shown an inconclusive relationship between nicotine dependence and quit success.^6,8 Despite a clear relationship of dependence to quit success among adult smokers, evidence in the hospital literature has been inconsistent. This inconsistency is likely due to the differing interventions across studies (eg, counseling vs. pharmacotherapy), the differing outcome variables (eg, self-report vs. biochemically verified), as well as the different patient populations selected to participate.

Unfortunately, smoking cessation is infrequently addressed in routine health care settings,^15,16 highlighting a gap in care. For example, one survey study¹⁶ found that while many health care professionals report asking about smoking status and advising smokers to quit, fewer clinicians assess smokers’ interest or intention to stop smoking, assist with cessation, or arrange follow-up. Our results indicate that assessing an inpatient smoker’s intentions, motivation, and confidence for cessation and attempting to improve low levels of these factors could enhance cessation success. Because motivation is a malleable construct, repeated assessment by hospital clinicians of a patient’s motivation and confidence to quit is needed.

Our results also confirm that inpatient efforts to improve smoking cessation postdischarge should target smokers’ resolve to quit and confidence in the ability to succeed. Motivational interventions and cognitive-behavioral therapy are effective strategies that can resolve ambivalence and increase confidence to quit and should be components of brief interventions delivered in inpatient settings.^17,18 Although individuals with a smoking-related illness may already possess some resolve to quit based on their illness, they may be candidates for interventions focused primarily on developing self-efficacy. Indeed, supporting self-efficacy is a major goal of effective bedside counseling and can be bolstered via problem-solving, motivational techniques, and education about pharmacotherapy during a tobacco-specific consult such as the one that these participants experienced. Armed with these resources, smokers with and without a smoking-related disease may be more likely to execute a plan to quit after discharge.

A study limitation is that our results can be generalized only to hospital inpatients who were willing to try to quit smoking after discharge, because the parent trial excluded smokers with lower levels of motivation. Similarly, these results may not be generalizable to obstetric or psychiatric inpatients, who were excluded from this trial.

In conclusion, our results underscore the importance of assessing motivation and self-efficacy in hospitalized smokers and targeting these factors in intervention efforts. Although future research should aim to identify better methods to alter these factors, in the short run, hospital clinicians could target these factors when discussing tobacco use with inpatient smokers.

Acknowledgments

The authors are grateful for the hard work of MGH, NSMC, and UPMC’s tobacco treatment services, the hospital providers, and study research staff.

Disclosures

Drs. Rigotti and Park received royalties from UpToDate and have received a research grant from Pfizer regarding smoking cessation. Dr. Rigotti has consulted (without pay) for Pfizer. Dr. Singer has served as a consultant to Pfizer but on a topic separate from smoking cessation. No other authors have conflicts of interest to disclose.
Role of Funding Source: The study was funded by NIH/NHLBI [grant #R01-HL11821]. The funding organization had no role in the study design, collection, analysis, and interpretation of the data, preparation of the manuscript, or decision to submit the manuscript for publication.

Clinical Trial Registration: NCT01714323

Cigarette smoking is the leading cause of preventable deaths in the United States.¹ Smoking contributes to several health problems that require hospitalization. Hospitalization also offers smokers an opportunity to quit because hospital policies prohibit smoking indoors while a health threat increases the motivation to quit.² Brief bedside smoking cessation counseling with follow-up contact after discharge increases postdischarge tobacco abstinence rates by 37%.² Identifying the characteristics of patients who are most likely to stop smoking after hospital discharge could identify strategies for interventions to help more smokers to succeed. It could also guide hospital clinicians’ efforts to provide effective brief messages to promote cessation by inpatients under their care during this teachable moment.

Sociodemographic factors, tobacco use, and psychological and medical factors have been associated with successful quit attempts by smokers in the general population.^3,4 Far less is known about the predictors of success in quitting smoking and maintaining abstinence after hospitalization. The characteristics associated with abstinence at postdischarge follow-up in prior studies of hospitalized smokers were male gender, greater confidence in quitting, greater readiness to quit, less nicotine dependence, and having a smoking-related illness.^5-8 However, most of the prior studies were limited to 1 geographic region,^5,6 focused only on a specific subgroup (eg, coronary patients⁹), or did not biochemically verify tobacco abstinence.⁸ In fact, to our knowledge, only one prior study has examined the predictors of quitting among a broad sample of hospital patients enrolled across multiple hospitals and biochemically verified abstinence.⁶ That study was conducted nearly two decades ago in one Midwestern state.

Thus, the present study aimed to identify factors independently associated with sustained postdischarge tobacco abstinence among hospitalized smokers who planned to quit smoking.¹⁰ Building on previous work, this study includes a large number of smokers with varied diagnoses admitted to one of three hospitals in two states, uses biochemically verified abstinence as the outcome measure, and examines multiple variables that were identified during the inpatient stay. We hypothesize that consistent with prior literature on this topic, factors independently associated with cessation in the present study will include confidence and intention to quit, degree of nicotine dependence, and a discharge diagnosis of a smoking-related disease.

We analyzed data from the Helping HAND2 Trial (HH2; NCT01714323), a randomized clinical trial conducted at the following three hospitals: Massachusetts General Hospital (MGH) in Boston, MA; University of Pittsburgh Medical Center (UPMC) in Pittsburgh, PA; and North Shore Medical Center (NSMC) in Salem, MA. Enrollment occurred from December 2012 to July 2014. The study methodology has been reported elsewhere.¹¹ This study was approved by the Institutional Review Boards of Partners HealthCare and University of Pittsburgh.

Hospital inpatients were eligible for enrollment if they were

• >18 years old, daily smokers, received smoking cessation counseling in the hospital (ie, standard of care for inpatient smokers), and planned to quit or try to quit smoking after discharge. Exclusion criteria included no access to a telephone, not speaking English, psychiatric or cognitive impairment, medical instability, or admission to obstetric or psychiatric units. All participants were offered nicotine replacement and one counseling session by a tobacco treatment specialist during hospitalization.

Participants were enrolled before discharge and randomly assigned to Sustained Care (Intervention) or Standard Care (Control) conditions.^10,11 In the Standard Care condition, participants received advice to call a free telephone quit line and a tailored recommendation for postdischarge pharmacotherapy. Participants randomized to Sustained Care received a free 30-day supply of their choice of FDA-approved tobacco cessation pharmacotherapy at hospital discharge (refillable twice) and five automated interactive voice response calls over three months postdischarge to allow them to access counseling or refill medications.

Baseline Demographic and Smoking Characteristics

A baseline survey assessed demographic variables (age, gender, race/ethnicity, education), tobacco use (cigarettes smoked per day, time to first morning cigarette,¹² other tobacco use, and prior quit attempts), intention to quit after discharge (ie,“What is your plan about smoking after you leave the hospital,” with the intent measured across four categorical response options), perceived importance of and confidence in quitting after discharge (five-point Likert scales ranging from “not at all” to “very”), and the presence of another smoker at home. Depression and anxiety symptoms were assessed using the Patient Health Questionnaire (PHQ-4¹³). Alcohol use (AUDIT-C¹⁴) and past-year use of cocaine, stimulants, opioids, and marijuana were also measured. Health insurance, length of stay, and primary discharge diagnoses were abstracted from the medical record. Smoking-related disease categories were derived from the 2014 U.S. Surgeon General’s Report.¹

Follow-up Assessment

Telephone surveys were administered by the research staff sixmonths after hospital discharge. Participants who reported past seven-day tobacco abstinence (ie, abstinence from tobacco for the past seven days reported at the 6-month call) were asked to provide a mailed saliva sample to assay for cotinine, a nicotine metabolite, to verify self-reported abstinence. Participants who reported nicotine replacement therapy use were asked to provide an in-person measurement of expired air carbon monoxide (CO) instead. Self-reported abstinence was biochemically verified if saliva cotinine was <10 ng/ml or if CO was <9 ppm.¹¹

Outcomes

The dependent variable, consistent with the parent trial, was biochemically confirmed past seven-day tobacco abstinence at six-month follow-up. Nonrespondents and those failing to provide a sample for confirmation were considered as smokers. In addition, a sensitivity analysis used complete cases only, excluding cases with missing smoking status outcomes.

Analysis

Bivariate associations of baseline predictor variables and biochemically confirmed abstinence were examined using chi-square tests for categorical variables and t tests or Wilcoxon rank sum tests for continuous variables. Using multiple logistic regression analyses, we identified variables that were independently associated with confirmed abstinence. The final models included all factors that were associated with cessation in the bivariate analysis (P < .10), factors associated with abstinence in the literature regardless of statistical significance (gender, AUDIT-C score),⁴ study site, and study condition. A two-sided p value of <.05 was considered to be statistically significant. Analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC).

Baseline characteristics of the 1,357 smokers enrolled in the trial are reported in Table 1. One-third of participants had a smoking-related discharge diagnosis. The median self-reported confidence in quitting was three on a five-point scale, and nearly half of the participants reported planning to stay abstinent after discharge. At six-month follow-up, 75% of participants completed the assessment, and seven-day tobacco abstinence was reported by 389 participants (29%) and biochemically confirmed in 218 participants (16%).

Results of the multiple logistic regression analysis predicting biochemically confirmed abstinence at six months are presented in Table 2. Factors independently associated with confirmed abstinence were a smoking-related primary discharge diagnosis (AOR = 1.98, 95% CI: 1.41-2.77), greater confidence in the ability to quit smoking (AOR = 1.31, 95% CI: 1.07-1.60), and stronger intention to quit (plan to stay abstinent after discharge vs. try to stay abstinent; AOR = 1.68, 95% CI: 1.19-2.38). Similar variables emerged as independent predictors of abstinence when the analysis was limited to complete cases, with an exception that one additional predictor, time to first cigarette after 30 minutes of waking, had statistical significance at the 0.05 level (Table 2).

We examined the associations between factors that were identifiable in the hospital and postdischarge tobacco abstinence among a general sample of hospitalized patients enrolled in a smoking cessation trial. The odds of biochemically confirmed abstinence at six months were higher among participants who reported higher levels of confidence in quitting smoking, those reporting having a definite plan to quit (vs. try to) after discharge, and those with a smoking-related primary discharge diagnosis.

Our findings are largely consistent with the prior literature on this topic, which has demonstrated that increased confidence in quitting, having a plan to quit smoking, and the presence of a smoking-related disease are associated with quit success at follow-up among hospital patients as well as in the general adult population.^3-7 Our finding that nicotine dependence predicted quit success in the complete case analysis, but not when imputing smoking status, aligns with prior studies of hospitalized smokers, which have shown an inconclusive relationship between nicotine dependence and quit success.^6,8 Despite a clear relationship of dependence to quit success among adult smokers, evidence in the hospital literature has been inconsistent. This inconsistency is likely due to the differing interventions across studies (eg, counseling vs. pharmacotherapy), the differing outcome variables (eg, self-report vs. biochemically verified), as well as the different patient populations selected to participate.

Unfortunately, smoking cessation is infrequently addressed in routine health care settings,^15,16 highlighting a gap in care. For example, one survey study¹⁶ found that while many health care professionals report asking about smoking status and advising smokers to quit, fewer clinicians assess smokers’ interest or intention to stop smoking, assist with cessation, or arrange follow-up. Our results indicate that assessing an inpatient smoker’s intentions, motivation, and confidence for cessation and attempting to improve low levels of these factors could enhance cessation success. Because motivation is a malleable construct, repeated assessment by hospital clinicians of a patient’s motivation and confidence to quit is needed.

Our results also confirm that inpatient efforts to improve smoking cessation postdischarge should target smokers’ resolve to quit and confidence in the ability to succeed. Motivational interventions and cognitive-behavioral therapy are effective strategies that can resolve ambivalence and increase confidence to quit and should be components of brief interventions delivered in inpatient settings.^17,18 Although individuals with a smoking-related illness may already possess some resolve to quit based on their illness, they may be candidates for interventions focused primarily on developing self-efficacy. Indeed, supporting self-efficacy is a major goal of effective bedside counseling and can be bolstered via problem-solving, motivational techniques, and education about pharmacotherapy during a tobacco-specific consult such as the one that these participants experienced. Armed with these resources, smokers with and without a smoking-related disease may be more likely to execute a plan to quit after discharge.

A study limitation is that our results can be generalized only to hospital inpatients who were willing to try to quit smoking after discharge, because the parent trial excluded smokers with lower levels of motivation. Similarly, these results may not be generalizable to obstetric or psychiatric inpatients, who were excluded from this trial.

In conclusion, our results underscore the importance of assessing motivation and self-efficacy in hospitalized smokers and targeting these factors in intervention efforts. Although future research should aim to identify better methods to alter these factors, in the short run, hospital clinicians could target these factors when discussing tobacco use with inpatient smokers.

Acknowledgments

The authors are grateful for the hard work of MGH, NSMC, and UPMC’s tobacco treatment services, the hospital providers, and study research staff.

Disclosures

Drs. Rigotti and Park received royalties from UpToDate and have received a research grant from Pfizer regarding smoking cessation. Dr. Rigotti has consulted (without pay) for Pfizer. Dr. Singer has served as a consultant to Pfizer but on a topic separate from smoking cessation. No other authors have conflicts of interest to disclose.
Role of Funding Source: The study was funded by NIH/NHLBI [grant #R01-HL11821]. The funding organization had no role in the study design, collection, analysis, and interpretation of the data, preparation of the manuscript, or decision to submit the manuscript for publication.

Clinical Trial Registration: NCT01714323

Cigarette smoking is the leading cause of preventable deaths in the United States.¹ Smoking contributes to several health problems that require hospitalization. Hospitalization also offers smokers an opportunity to quit because hospital policies prohibit smoking indoors while a health threat increases the motivation to quit.² Brief bedside smoking cessation counseling with follow-up contact after discharge increases postdischarge tobacco abstinence rates by 37%.² Identifying the characteristics of patients who are most likely to stop smoking after hospital discharge could identify strategies for interventions to help more smokers to succeed. It could also guide hospital clinicians’ efforts to provide effective brief messages to promote cessation by inpatients under their care during this teachable moment.

Sociodemographic factors, tobacco use, and psychological and medical factors have been associated with successful quit attempts by smokers in the general population.^3,4 Far less is known about the predictors of success in quitting smoking and maintaining abstinence after hospitalization. The characteristics associated with abstinence at postdischarge follow-up in prior studies of hospitalized smokers were male gender, greater confidence in quitting, greater readiness to quit, less nicotine dependence, and having a smoking-related illness.^5-8 However, most of the prior studies were limited to 1 geographic region,^5,6 focused only on a specific subgroup (eg, coronary patients⁹), or did not biochemically verify tobacco abstinence.⁸ In fact, to our knowledge, only one prior study has examined the predictors of quitting among a broad sample of hospital patients enrolled across multiple hospitals and biochemically verified abstinence.⁶ That study was conducted nearly two decades ago in one Midwestern state.

Thus, the present study aimed to identify factors independently associated with sustained postdischarge tobacco abstinence among hospitalized smokers who planned to quit smoking.¹⁰ Building on previous work, this study includes a large number of smokers with varied diagnoses admitted to one of three hospitals in two states, uses biochemically verified abstinence as the outcome measure, and examines multiple variables that were identified during the inpatient stay. We hypothesize that consistent with prior literature on this topic, factors independently associated with cessation in the present study will include confidence and intention to quit, degree of nicotine dependence, and a discharge diagnosis of a smoking-related disease.

We analyzed data from the Helping HAND2 Trial (HH2; NCT01714323), a randomized clinical trial conducted at the following three hospitals: Massachusetts General Hospital (MGH) in Boston, MA; University of Pittsburgh Medical Center (UPMC) in Pittsburgh, PA; and North Shore Medical Center (NSMC) in Salem, MA. Enrollment occurred from December 2012 to July 2014. The study methodology has been reported elsewhere.¹¹ This study was approved by the Institutional Review Boards of Partners HealthCare and University of Pittsburgh.

Hospital inpatients were eligible for enrollment if they were

• >18 years old, daily smokers, received smoking cessation counseling in the hospital (ie, standard of care for inpatient smokers), and planned to quit or try to quit smoking after discharge. Exclusion criteria included no access to a telephone, not speaking English, psychiatric or cognitive impairment, medical instability, or admission to obstetric or psychiatric units. All participants were offered nicotine replacement and one counseling session by a tobacco treatment specialist during hospitalization.

Participants were enrolled before discharge and randomly assigned to Sustained Care (Intervention) or Standard Care (Control) conditions.^10,11 In the Standard Care condition, participants received advice to call a free telephone quit line and a tailored recommendation for postdischarge pharmacotherapy. Participants randomized to Sustained Care received a free 30-day supply of their choice of FDA-approved tobacco cessation pharmacotherapy at hospital discharge (refillable twice) and five automated interactive voice response calls over three months postdischarge to allow them to access counseling or refill medications.

Baseline Demographic and Smoking Characteristics

A baseline survey assessed demographic variables (age, gender, race/ethnicity, education), tobacco use (cigarettes smoked per day, time to first morning cigarette,¹² other tobacco use, and prior quit attempts), intention to quit after discharge (ie,“What is your plan about smoking after you leave the hospital,” with the intent measured across four categorical response options), perceived importance of and confidence in quitting after discharge (five-point Likert scales ranging from “not at all” to “very”), and the presence of another smoker at home. Depression and anxiety symptoms were assessed using the Patient Health Questionnaire (PHQ-4¹³). Alcohol use (AUDIT-C¹⁴) and past-year use of cocaine, stimulants, opioids, and marijuana were also measured. Health insurance, length of stay, and primary discharge diagnoses were abstracted from the medical record. Smoking-related disease categories were derived from the 2014 U.S. Surgeon General’s Report.¹

Follow-up Assessment

Telephone surveys were administered by the research staff sixmonths after hospital discharge. Participants who reported past seven-day tobacco abstinence (ie, abstinence from tobacco for the past seven days reported at the 6-month call) were asked to provide a mailed saliva sample to assay for cotinine, a nicotine metabolite, to verify self-reported abstinence. Participants who reported nicotine replacement therapy use were asked to provide an in-person measurement of expired air carbon monoxide (CO) instead. Self-reported abstinence was biochemically verified if saliva cotinine was <10 ng/ml or if CO was <9 ppm.¹¹

Outcomes

The dependent variable, consistent with the parent trial, was biochemically confirmed past seven-day tobacco abstinence at six-month follow-up. Nonrespondents and those failing to provide a sample for confirmation were considered as smokers. In addition, a sensitivity analysis used complete cases only, excluding cases with missing smoking status outcomes.

Analysis

Bivariate associations of baseline predictor variables and biochemically confirmed abstinence were examined using chi-square tests for categorical variables and t tests or Wilcoxon rank sum tests for continuous variables. Using multiple logistic regression analyses, we identified variables that were independently associated with confirmed abstinence. The final models included all factors that were associated with cessation in the bivariate analysis (P < .10), factors associated with abstinence in the literature regardless of statistical significance (gender, AUDIT-C score),⁴ study site, and study condition. A two-sided p value of <.05 was considered to be statistically significant. Analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC).

Baseline characteristics of the 1,357 smokers enrolled in the trial are reported in Table 1. One-third of participants had a smoking-related discharge diagnosis. The median self-reported confidence in quitting was three on a five-point scale, and nearly half of the participants reported planning to stay abstinent after discharge. At six-month follow-up, 75% of participants completed the assessment, and seven-day tobacco abstinence was reported by 389 participants (29%) and biochemically confirmed in 218 participants (16%).

Results of the multiple logistic regression analysis predicting biochemically confirmed abstinence at six months are presented in Table 2. Factors independently associated with confirmed abstinence were a smoking-related primary discharge diagnosis (AOR = 1.98, 95% CI: 1.41-2.77), greater confidence in the ability to quit smoking (AOR = 1.31, 95% CI: 1.07-1.60), and stronger intention to quit (plan to stay abstinent after discharge vs. try to stay abstinent; AOR = 1.68, 95% CI: 1.19-2.38). Similar variables emerged as independent predictors of abstinence when the analysis was limited to complete cases, with an exception that one additional predictor, time to first cigarette after 30 minutes of waking, had statistical significance at the 0.05 level (Table 2).

We examined the associations between factors that were identifiable in the hospital and postdischarge tobacco abstinence among a general sample of hospitalized patients enrolled in a smoking cessation trial. The odds of biochemically confirmed abstinence at six months were higher among participants who reported higher levels of confidence in quitting smoking, those reporting having a definite plan to quit (vs. try to) after discharge, and those with a smoking-related primary discharge diagnosis.

Our findings are largely consistent with the prior literature on this topic, which has demonstrated that increased confidence in quitting, having a plan to quit smoking, and the presence of a smoking-related disease are associated with quit success at follow-up among hospital patients as well as in the general adult population.^3-7 Our finding that nicotine dependence predicted quit success in the complete case analysis, but not when imputing smoking status, aligns with prior studies of hospitalized smokers, which have shown an inconclusive relationship between nicotine dependence and quit success.^6,8 Despite a clear relationship of dependence to quit success among adult smokers, evidence in the hospital literature has been inconsistent. This inconsistency is likely due to the differing interventions across studies (eg, counseling vs. pharmacotherapy), the differing outcome variables (eg, self-report vs. biochemically verified), as well as the different patient populations selected to participate.

Unfortunately, smoking cessation is infrequently addressed in routine health care settings,^15,16 highlighting a gap in care. For example, one survey study¹⁶ found that while many health care professionals report asking about smoking status and advising smokers to quit, fewer clinicians assess smokers’ interest or intention to stop smoking, assist with cessation, or arrange follow-up. Our results indicate that assessing an inpatient smoker’s intentions, motivation, and confidence for cessation and attempting to improve low levels of these factors could enhance cessation success. Because motivation is a malleable construct, repeated assessment by hospital clinicians of a patient’s motivation and confidence to quit is needed.

Our results also confirm that inpatient efforts to improve smoking cessation postdischarge should target smokers’ resolve to quit and confidence in the ability to succeed. Motivational interventions and cognitive-behavioral therapy are effective strategies that can resolve ambivalence and increase confidence to quit and should be components of brief interventions delivered in inpatient settings.^17,18 Although individuals with a smoking-related illness may already possess some resolve to quit based on their illness, they may be candidates for interventions focused primarily on developing self-efficacy. Indeed, supporting self-efficacy is a major goal of effective bedside counseling and can be bolstered via problem-solving, motivational techniques, and education about pharmacotherapy during a tobacco-specific consult such as the one that these participants experienced. Armed with these resources, smokers with and without a smoking-related disease may be more likely to execute a plan to quit after discharge.

A study limitation is that our results can be generalized only to hospital inpatients who were willing to try to quit smoking after discharge, because the parent trial excluded smokers with lower levels of motivation. Similarly, these results may not be generalizable to obstetric or psychiatric inpatients, who were excluded from this trial.

In conclusion, our results underscore the importance of assessing motivation and self-efficacy in hospitalized smokers and targeting these factors in intervention efforts. Although future research should aim to identify better methods to alter these factors, in the short run, hospital clinicians could target these factors when discussing tobacco use with inpatient smokers.

Acknowledgments

The authors are grateful for the hard work of MGH, NSMC, and UPMC’s tobacco treatment services, the hospital providers, and study research staff.

Disclosures

Drs. Rigotti and Park received royalties from UpToDate and have received a research grant from Pfizer regarding smoking cessation. Dr. Rigotti has consulted (without pay) for Pfizer. Dr. Singer has served as a consultant to Pfizer but on a topic separate from smoking cessation. No other authors have conflicts of interest to disclose.
Role of Funding Source: The study was funded by NIH/NHLBI [grant #R01-HL11821]. The funding organization had no role in the study design, collection, analysis, and interpretation of the data, preparation of the manuscript, or decision to submit the manuscript for publication.

Clinical Trial Registration: NCT01714323

Respiratory illnesses are the leading causes of pediatric hospitalizations in the United States.¹ The 30-day hospital readmission rate for respiratory illnesses is being considered for implementation as a national hospital performance measure, as it may be an indicator of lower quality care (eg, poor hospital management of disease, inadequate patient/caretaker education prior to discharge). In adult populations, readmissions can be used to reliably identify variation in hospital performance and successfully drive efforts to improve the value of care.^{2, 3} In contrast, there are persistent concerns about using pediatric readmissions to identify variation in hospital performance, largely due to lower patient volumes.^4-7 To increase the value of pediatric hospital care, it is important to develop ways to meaningfully measure quality of care and further, to better understand the relationship between measures of quality and healthcare costs.

In December 2016, the National Quality Forum (NQF) endorsed a Pediatric Lower Respiratory Infection (LRI) Readmission Measure.⁸ This measure was developed by the Pediatric Quality Measurement Program, through the Agency for Healthcare Research and Quality. The goal of this program was to “increase the portfolio of evidence-based, consensus pediatric quality measures available to public and private purchasers of children’s healthcare services, providers, and consumers.”⁹

In anticipation of the national implementation of pediatric readmission measures, we examined whether the Pediatric LRI Readmission Measure could meaningfully identify high and low performers across all types of hospitals admitting children (general hospitals and children’s hospitals) using an all-payer claims database. A recent analysis by Nakamura et al. identified high and low performers using this measure¹⁰ but limited the analysis to hospitals with >50 pediatric LRI admissions per year, an approach that excludes many general hospitals. Since general hospitals provide the majority of care for children hospitalized with respiratory infections,¹¹ we aimed to evaluate the measure in a broadly inclusive analysis that included all hospital types. Because low patient volumes might limit use of the measure,^4,6 we tested several broadened variations of the measure. We also examined the relationship between hospital performance in pediatric LRI readmissions and healthcare costs.

Our analysis is intended to inform utilizers of pediatric quality metrics and policy makers about the feasibility of using these metrics to publicly report hospital performance and/or identify exceptional hospitals for understanding best practices in pediatric inpatient care.¹²

Study Design and Data Source

We conducted an observational, retrospective cohort analysis using the 2012-2014 California Office of Statewide Health Planning and Development (OSHPD) nonpublic inpatient and emergency department databases.¹³ The OSHPD databases are compiled annually through mandatory reporting by all licensed nonfederal hospitals in California. The databases contain demographic (eg, age, gender) and utilization data (eg, charges) and can track readmissions to hospitals other than the index hospital. The databases capture administrative claims from approximately 450 hospitals, composed of 16 million inpatients, emergency department patients, and ambulatory surgery patients annually. Data quality is monitored through the California OSHPD.

Study Population

Our study included children aged ≤18 years with LRI, defined using the NQF Pediatric LRI Readmissions Measure: a primary diagnosis of bronchiolitis, influenza, or pneumonia, or a secondary diagnosis of bronchiolitis, influenza, or pneumonia, with a primary diagnosis of asthma, respiratory failure, sepsis, or bacteremia.⁸ International classification of Diseases, 9^th edition (ICD-9) diagnostic codes used are in Appendix 1.

Per the NQF measure specifications,⁸ records were excluded if they were from hospitals with <80% of records complete with core elements (unique patient identifier, admission date, end-of-service date, and ICD-9 primary diagnosis code). In addition, records were excluded for the following reasons: (1) individual record missing core elements, (2) discharge disposition “death,” (3) 30-day follow-up data not available, (4) primary “newborn” or mental health diagnosis, or (5) primary ICD-9 procedure code for a planned procedure or chemotherapy.

Patient characteristics for hospital admissions with and without 30-day readmissions or 30-day emergency department (ED) revisits were summarized. For the continuous variable age, mean and standard deviation for each group were calculated. For categorical variables (sex, race, payer, and number of chronic conditions), numbers and proportions were determined. Univariate tests of comparison were carried out using the Student’s t test for age and chi-square tests for all categorical variables. Categories of payer with small values were combined for ease of description (categories combined into “other:” workers’ compensation, county indigent programs, other government, other indigent, self-pay, other payer). We identified chronic conditions using the Agency for Healthcare Research and Quality Chronic Condition Indicator (CCI) system, which classifies ICD-9-CM diagnosis codes as chronic or acute and places each code into 1 of 18 mutually exclusive categories (organ systems, disease categories, or other categories). The case-mix adjustment model incorporates a binary variable for each CCI category (0-1, 2, 3, or >4 chronic conditions) per the NQF measure specifications.⁸ This study was approved by the University of California, San Francisco Institutional Review Board.

Outcomes

Our primary outcome was the hospital-level rate of 30-day readmission after hospital discharge, consistent with the NQF measure.⁸ We identified outlier hospitals for 30-day readmission rate using the Centers for Medicare and Medicaid Services (CMS) methodology, which defines outlier hospitals as those for whom adjusted readmission rate confidence intervals do not overlap with the overall group mean rate.^{5, 14}

We also determined the hospital-level average cost per index hospitalization (not including costs of readmissions). Since costs of care often differ substantially from charges,¹⁵ costs were calculated using cost-to-charge ratios for each hospital (annual total operating expenses/total gross patient revenue, as reported to the OSHPD).¹⁶ Costs were subdivided into categories representing $5,000 increments and a top category of >$40,000. Outlier hospitals for costs were defined as those for whom the cost random effect was either greater than the third quartile of the distribution of values by more than 1.5 times the interquartile range or less than the first quartile of the distribution of values by more than 1.5 times the interquartile range.¹⁷

ANALYSIS

Primary Analysis

For our primary analysis of 30-day hospital readmission rates, we used hierarchical logistic regression models with hospitals as random effects, adjusting for patient age, sex, and the presence and number of body systems affected by chronic conditions.⁸ These 4 patient characteristics were selected by the NQF measure developers “because distributions of these characteristics vary across hospitals, and although they are associated with readmission risk, they are independent of hospital quality of care.”¹⁰

Because the Centers for Medicare and Medicaid Services (CMS) are in the process of selecting pediatric quality measures for meaningful use reporting,¹⁸ we utilized CMS hospital readmissions methodology to calculate risk-adjusted rates and identify outlier hospitals. The CMS modeling strategy stabilizes performance estimates for low-volume hospitals and avoids penalizing these hospitals for high readmission rates that may be due to chance (random effects logistic model to obtain best linear unbiased predictions). This is particularly important in pediatrics, given the low pediatric volumes in many hospitals admitting children.^4,19 We then identified outlier hospitals for the 30-day readmission rate using CMS methodology (hospital’s adjusted readmission rate confidence interval does not overlap the overall group mean rate).^{5, 4} CMS uses this approach for public reporting on HospitalCompare.²⁰

Sensitivity Analyses

We tested several broadening variations of the NQF measure: (1) addition of children admitted with a primary diagnosis of asthma (without requiring LRI as a secondary diagnosis) or a secondary diagnosis of asthma exacerbation (LRIA), (2) inclusion of 30-day ED revisits as an outcome, and (3) merging of 3 years of data. These analyses were all performed using the same modeling strategy as in our primary analysis.

Secondary Outcome Analyses

Our analysis of hospital costs used costs for index admissions over 3 years (2012–2014) and included admissions for asthma. We used hierarchical regression models with hospitals as random effects, adjusting for age, gender, and the presence and number of chronic conditions. The distribution of cost values was highly skewed, so ordinal models were selected after several other modeling approaches failed (log transformation linear model, gamma model, Poisson model, zero-truncated Poisson model).

The relationship between hospital-level costs and hospital-level 30-day readmission or ED revisit rates was analyzed using Spearman’s rank correlation coefficient. Statistical analysis was performed using SAS version 9.4 software (SAS Institute; Cary, North Carolina).

Primary Analysis of 30-day Readmissions (per National Quality Forum Measure)

Our analysis of the 2014 OSHPD database using the specifications of the NQF Pediatric LRI Readmission Measure included a total of 5550 hospitalizations from 174 hospitals, with a mean of 12 eligible hospitalizations per hospital. The mean risk-adjusted readmission rate was 6.5% (362 readmissions). There were no hospitals that were considered outliers based on the risk-adjusted readmission rates (Table 1).

Sensitivity Analyses (Broadening Definitions of National Quality Forum Measure)

We report our testing of the broadened variations of the NQF measure in Table 1. Broadening the population to include children with asthma as a primary diagnosis and children with asthma exacerbations as a secondary diagnosis (LRIA) increased the size of our analysis to 8402 hospitalizations from 190 hospitals. The mean risk-adjusted readmission rate was 5.5%, and no outlier hospitals were identified.

Using the same inclusion criteria of the NQF measure but including 30-day ED revisits as an outcome, we analyzed a total of 5500 hospitalizations from 174 hospitals. The mean risk-adjusted event rate was higher at 7.9%, but there were still no outlier hospitals identified.

Using the broadened population definition (LRIA) and including 30-day ED revisits as an outcome, we analyzed a total of 8402 hospitalizations from 190 hospitals. The mean risk-adjusted event rate was 6.8%, but there were still no outlier hospitals identified.

In our final iteration, we merged 3 years of hospital data (2012-2014) using the broader population definition (LRIA) and including 30-day ED revisits as an outcome. This resulted in 27,873 admissions from 239 hospitals for this analysis, with a mean of 28 eligible hospitalizations per hospital. The mean risk-adjusted event rate was 6.7%, and this approach identified 2 high-performing (risk-adjusted rates: 3.6-5.3) and 7 low-performing hospitals (risk-adjusted rates: 10.1-15.9).

Table 2 presents the demographics of children included in this analysis. Children who had readmissions/revisits were younger, more likely to be white, less likely to have private insurance, and more likely to have a greater number of chronic conditions compared to children without readmissions/revisits.

Secondary Outcome: Hospital Costs

In the analysis of hospital-level costs, we found only 1 outlier high-cost hospital. There was a 20% probability of a hospital respiratory admission costing ≥$40,000 at this hospital. We found no overall relationship between hospital 30-day respiratory readmission rate and hospital costs (Figure 1). However, the hospitals that were outliers for low readmission rates also had low probabilities of excessive hospital costs (3% probability of costs >$40,000; Figure 2).

We used a nationally endorsed pediatric quality measure to evaluate hospital performance, defined as 30-day readmission rates for children with respiratory illness. We examined all-payer data from California, which is the most populous state in the country and home to 1 in 8 American children. In this large California dataset, we were unable to identify meaningful variation in hospital performance due to low hospital volumes and event rates. However, when we broadened the measure definition, we were able to identify performance variation. Our findings underscore the importance of testing and potentially modifying existing quality measures in order to more accurately capture the quality of care delivered at hospitals with lower volumes of pediatric patients.²¹

The ability of a nationally-endorsed pediatric respiratory readmissions measure to meaningfully identify variation in hospital performance is limited. General hospitals, which provide the majority of pediatric care for common conditions such as LRI, likely cannot be accurately evaluated using national pediatric quality metrics as they are currently designed. Modifying measures in order to increase hospital-level pediatric patient volumes may facilitate more meaningful evaluation of the quality of pediatric care in general hospitals and identification of exceptional hospitals for understanding best practices in pediatric inpatient care.

Disclosures

Regina Lam consulted for Proximity Health doing market research during the course of developing this manuscript, but this work did not involve any content related to quality metrics, and this entity did not play any role in the development of this manuscript. The remaining authors have no conflicts of interest relevant to this article to disclose.

Funding

Supported by the Agency for Healthcare Research and Quality (K08 HS24592 to SVK and U18HS25297 to MDC and NSB) and the National Institute of Child Health and Human Development (K23HD065836 to NSB). The funding agency played no role in the study design; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the manuscript for publication.

Respiratory illnesses are the leading causes of pediatric hospitalizations in the United States.¹ The 30-day hospital readmission rate for respiratory illnesses is being considered for implementation as a national hospital performance measure, as it may be an indicator of lower quality care (eg, poor hospital management of disease, inadequate patient/caretaker education prior to discharge). In adult populations, readmissions can be used to reliably identify variation in hospital performance and successfully drive efforts to improve the value of care.^{2, 3} In contrast, there are persistent concerns about using pediatric readmissions to identify variation in hospital performance, largely due to lower patient volumes.^4-7 To increase the value of pediatric hospital care, it is important to develop ways to meaningfully measure quality of care and further, to better understand the relationship between measures of quality and healthcare costs.

In December 2016, the National Quality Forum (NQF) endorsed a Pediatric Lower Respiratory Infection (LRI) Readmission Measure.⁸ This measure was developed by the Pediatric Quality Measurement Program, through the Agency for Healthcare Research and Quality. The goal of this program was to “increase the portfolio of evidence-based, consensus pediatric quality measures available to public and private purchasers of children’s healthcare services, providers, and consumers.”⁹

In anticipation of the national implementation of pediatric readmission measures, we examined whether the Pediatric LRI Readmission Measure could meaningfully identify high and low performers across all types of hospitals admitting children (general hospitals and children’s hospitals) using an all-payer claims database. A recent analysis by Nakamura et al. identified high and low performers using this measure¹⁰ but limited the analysis to hospitals with >50 pediatric LRI admissions per year, an approach that excludes many general hospitals. Since general hospitals provide the majority of care for children hospitalized with respiratory infections,¹¹ we aimed to evaluate the measure in a broadly inclusive analysis that included all hospital types. Because low patient volumes might limit use of the measure,^4,6 we tested several broadened variations of the measure. We also examined the relationship between hospital performance in pediatric LRI readmissions and healthcare costs.

Our analysis is intended to inform utilizers of pediatric quality metrics and policy makers about the feasibility of using these metrics to publicly report hospital performance and/or identify exceptional hospitals for understanding best practices in pediatric inpatient care.¹²

Study Design and Data Source

We conducted an observational, retrospective cohort analysis using the 2012-2014 California Office of Statewide Health Planning and Development (OSHPD) nonpublic inpatient and emergency department databases.¹³ The OSHPD databases are compiled annually through mandatory reporting by all licensed nonfederal hospitals in California. The databases contain demographic (eg, age, gender) and utilization data (eg, charges) and can track readmissions to hospitals other than the index hospital. The databases capture administrative claims from approximately 450 hospitals, composed of 16 million inpatients, emergency department patients, and ambulatory surgery patients annually. Data quality is monitored through the California OSHPD.

Study Population

Our study included children aged ≤18 years with LRI, defined using the NQF Pediatric LRI Readmissions Measure: a primary diagnosis of bronchiolitis, influenza, or pneumonia, or a secondary diagnosis of bronchiolitis, influenza, or pneumonia, with a primary diagnosis of asthma, respiratory failure, sepsis, or bacteremia.⁸ International classification of Diseases, 9^th edition (ICD-9) diagnostic codes used are in Appendix 1.

Per the NQF measure specifications,⁸ records were excluded if they were from hospitals with <80% of records complete with core elements (unique patient identifier, admission date, end-of-service date, and ICD-9 primary diagnosis code). In addition, records were excluded for the following reasons: (1) individual record missing core elements, (2) discharge disposition “death,” (3) 30-day follow-up data not available, (4) primary “newborn” or mental health diagnosis, or (5) primary ICD-9 procedure code for a planned procedure or chemotherapy.

Patient characteristics for hospital admissions with and without 30-day readmissions or 30-day emergency department (ED) revisits were summarized. For the continuous variable age, mean and standard deviation for each group were calculated. For categorical variables (sex, race, payer, and number of chronic conditions), numbers and proportions were determined. Univariate tests of comparison were carried out using the Student’s t test for age and chi-square tests for all categorical variables. Categories of payer with small values were combined for ease of description (categories combined into “other:” workers’ compensation, county indigent programs, other government, other indigent, self-pay, other payer). We identified chronic conditions using the Agency for Healthcare Research and Quality Chronic Condition Indicator (CCI) system, which classifies ICD-9-CM diagnosis codes as chronic or acute and places each code into 1 of 18 mutually exclusive categories (organ systems, disease categories, or other categories). The case-mix adjustment model incorporates a binary variable for each CCI category (0-1, 2, 3, or >4 chronic conditions) per the NQF measure specifications.⁸ This study was approved by the University of California, San Francisco Institutional Review Board.

Outcomes

Our primary outcome was the hospital-level rate of 30-day readmission after hospital discharge, consistent with the NQF measure.⁸ We identified outlier hospitals for 30-day readmission rate using the Centers for Medicare and Medicaid Services (CMS) methodology, which defines outlier hospitals as those for whom adjusted readmission rate confidence intervals do not overlap with the overall group mean rate.^{5, 14}

We also determined the hospital-level average cost per index hospitalization (not including costs of readmissions). Since costs of care often differ substantially from charges,¹⁵ costs were calculated using cost-to-charge ratios for each hospital (annual total operating expenses/total gross patient revenue, as reported to the OSHPD).¹⁶ Costs were subdivided into categories representing $5,000 increments and a top category of >$40,000. Outlier hospitals for costs were defined as those for whom the cost random effect was either greater than the third quartile of the distribution of values by more than 1.5 times the interquartile range or less than the first quartile of the distribution of values by more than 1.5 times the interquartile range.¹⁷

ANALYSIS

Primary Analysis

For our primary analysis of 30-day hospital readmission rates, we used hierarchical logistic regression models with hospitals as random effects, adjusting for patient age, sex, and the presence and number of body systems affected by chronic conditions.⁸ These 4 patient characteristics were selected by the NQF measure developers “because distributions of these characteristics vary across hospitals, and although they are associated with readmission risk, they are independent of hospital quality of care.”¹⁰

Because the Centers for Medicare and Medicaid Services (CMS) are in the process of selecting pediatric quality measures for meaningful use reporting,¹⁸ we utilized CMS hospital readmissions methodology to calculate risk-adjusted rates and identify outlier hospitals. The CMS modeling strategy stabilizes performance estimates for low-volume hospitals and avoids penalizing these hospitals for high readmission rates that may be due to chance (random effects logistic model to obtain best linear unbiased predictions). This is particularly important in pediatrics, given the low pediatric volumes in many hospitals admitting children.^4,19 We then identified outlier hospitals for the 30-day readmission rate using CMS methodology (hospital’s adjusted readmission rate confidence interval does not overlap the overall group mean rate).^{5, 4} CMS uses this approach for public reporting on HospitalCompare.²⁰

Sensitivity Analyses

We tested several broadening variations of the NQF measure: (1) addition of children admitted with a primary diagnosis of asthma (without requiring LRI as a secondary diagnosis) or a secondary diagnosis of asthma exacerbation (LRIA), (2) inclusion of 30-day ED revisits as an outcome, and (3) merging of 3 years of data. These analyses were all performed using the same modeling strategy as in our primary analysis.

Secondary Outcome Analyses

Our analysis of hospital costs used costs for index admissions over 3 years (2012–2014) and included admissions for asthma. We used hierarchical regression models with hospitals as random effects, adjusting for age, gender, and the presence and number of chronic conditions. The distribution of cost values was highly skewed, so ordinal models were selected after several other modeling approaches failed (log transformation linear model, gamma model, Poisson model, zero-truncated Poisson model).

The relationship between hospital-level costs and hospital-level 30-day readmission or ED revisit rates was analyzed using Spearman’s rank correlation coefficient. Statistical analysis was performed using SAS version 9.4 software (SAS Institute; Cary, North Carolina).

Primary Analysis of 30-day Readmissions (per National Quality Forum Measure)

Our analysis of the 2014 OSHPD database using the specifications of the NQF Pediatric LRI Readmission Measure included a total of 5550 hospitalizations from 174 hospitals, with a mean of 12 eligible hospitalizations per hospital. The mean risk-adjusted readmission rate was 6.5% (362 readmissions). There were no hospitals that were considered outliers based on the risk-adjusted readmission rates (Table 1).

Sensitivity Analyses (Broadening Definitions of National Quality Forum Measure)

We report our testing of the broadened variations of the NQF measure in Table 1. Broadening the population to include children with asthma as a primary diagnosis and children with asthma exacerbations as a secondary diagnosis (LRIA) increased the size of our analysis to 8402 hospitalizations from 190 hospitals. The mean risk-adjusted readmission rate was 5.5%, and no outlier hospitals were identified.

Using the same inclusion criteria of the NQF measure but including 30-day ED revisits as an outcome, we analyzed a total of 5500 hospitalizations from 174 hospitals. The mean risk-adjusted event rate was higher at 7.9%, but there were still no outlier hospitals identified.

Using the broadened population definition (LRIA) and including 30-day ED revisits as an outcome, we analyzed a total of 8402 hospitalizations from 190 hospitals. The mean risk-adjusted event rate was 6.8%, but there were still no outlier hospitals identified.

In our final iteration, we merged 3 years of hospital data (2012-2014) using the broader population definition (LRIA) and including 30-day ED revisits as an outcome. This resulted in 27,873 admissions from 239 hospitals for this analysis, with a mean of 28 eligible hospitalizations per hospital. The mean risk-adjusted event rate was 6.7%, and this approach identified 2 high-performing (risk-adjusted rates: 3.6-5.3) and 7 low-performing hospitals (risk-adjusted rates: 10.1-15.9).

Table 2 presents the demographics of children included in this analysis. Children who had readmissions/revisits were younger, more likely to be white, less likely to have private insurance, and more likely to have a greater number of chronic conditions compared to children without readmissions/revisits.

Secondary Outcome: Hospital Costs

In the analysis of hospital-level costs, we found only 1 outlier high-cost hospital. There was a 20% probability of a hospital respiratory admission costing ≥$40,000 at this hospital. We found no overall relationship between hospital 30-day respiratory readmission rate and hospital costs (Figure 1). However, the hospitals that were outliers for low readmission rates also had low probabilities of excessive hospital costs (3% probability of costs >$40,000; Figure 2).

We used a nationally endorsed pediatric quality measure to evaluate hospital performance, defined as 30-day readmission rates for children with respiratory illness. We examined all-payer data from California, which is the most populous state in the country and home to 1 in 8 American children. In this large California dataset, we were unable to identify meaningful variation in hospital performance due to low hospital volumes and event rates. However, when we broadened the measure definition, we were able to identify performance variation. Our findings underscore the importance of testing and potentially modifying existing quality measures in order to more accurately capture the quality of care delivered at hospitals with lower volumes of pediatric patients.²¹

The ability of a nationally-endorsed pediatric respiratory readmissions measure to meaningfully identify variation in hospital performance is limited. General hospitals, which provide the majority of pediatric care for common conditions such as LRI, likely cannot be accurately evaluated using national pediatric quality metrics as they are currently designed. Modifying measures in order to increase hospital-level pediatric patient volumes may facilitate more meaningful evaluation of the quality of pediatric care in general hospitals and identification of exceptional hospitals for understanding best practices in pediatric inpatient care.

Disclosures

Regina Lam consulted for Proximity Health doing market research during the course of developing this manuscript, but this work did not involve any content related to quality metrics, and this entity did not play any role in the development of this manuscript. The remaining authors have no conflicts of interest relevant to this article to disclose.

Funding

Supported by the Agency for Healthcare Research and Quality (K08 HS24592 to SVK and U18HS25297 to MDC and NSB) and the National Institute of Child Health and Human Development (K23HD065836 to NSB). The funding agency played no role in the study design; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the manuscript for publication.

Respiratory illnesses are the leading causes of pediatric hospitalizations in the United States.¹ The 30-day hospital readmission rate for respiratory illnesses is being considered for implementation as a national hospital performance measure, as it may be an indicator of lower quality care (eg, poor hospital management of disease, inadequate patient/caretaker education prior to discharge). In adult populations, readmissions can be used to reliably identify variation in hospital performance and successfully drive efforts to improve the value of care.^{2, 3} In contrast, there are persistent concerns about using pediatric readmissions to identify variation in hospital performance, largely due to lower patient volumes.^4-7 To increase the value of pediatric hospital care, it is important to develop ways to meaningfully measure quality of care and further, to better understand the relationship between measures of quality and healthcare costs.

In December 2016, the National Quality Forum (NQF) endorsed a Pediatric Lower Respiratory Infection (LRI) Readmission Measure.⁸ This measure was developed by the Pediatric Quality Measurement Program, through the Agency for Healthcare Research and Quality. The goal of this program was to “increase the portfolio of evidence-based, consensus pediatric quality measures available to public and private purchasers of children’s healthcare services, providers, and consumers.”⁹

In anticipation of the national implementation of pediatric readmission measures, we examined whether the Pediatric LRI Readmission Measure could meaningfully identify high and low performers across all types of hospitals admitting children (general hospitals and children’s hospitals) using an all-payer claims database. A recent analysis by Nakamura et al. identified high and low performers using this measure¹⁰ but limited the analysis to hospitals with >50 pediatric LRI admissions per year, an approach that excludes many general hospitals. Since general hospitals provide the majority of care for children hospitalized with respiratory infections,¹¹ we aimed to evaluate the measure in a broadly inclusive analysis that included all hospital types. Because low patient volumes might limit use of the measure,^4,6 we tested several broadened variations of the measure. We also examined the relationship between hospital performance in pediatric LRI readmissions and healthcare costs.

Our analysis is intended to inform utilizers of pediatric quality metrics and policy makers about the feasibility of using these metrics to publicly report hospital performance and/or identify exceptional hospitals for understanding best practices in pediatric inpatient care.¹²

Study Design and Data Source

We conducted an observational, retrospective cohort analysis using the 2012-2014 California Office of Statewide Health Planning and Development (OSHPD) nonpublic inpatient and emergency department databases.¹³ The OSHPD databases are compiled annually through mandatory reporting by all licensed nonfederal hospitals in California. The databases contain demographic (eg, age, gender) and utilization data (eg, charges) and can track readmissions to hospitals other than the index hospital. The databases capture administrative claims from approximately 450 hospitals, composed of 16 million inpatients, emergency department patients, and ambulatory surgery patients annually. Data quality is monitored through the California OSHPD.

Study Population

Our study included children aged ≤18 years with LRI, defined using the NQF Pediatric LRI Readmissions Measure: a primary diagnosis of bronchiolitis, influenza, or pneumonia, or a secondary diagnosis of bronchiolitis, influenza, or pneumonia, with a primary diagnosis of asthma, respiratory failure, sepsis, or bacteremia.⁸ International classification of Diseases, 9^th edition (ICD-9) diagnostic codes used are in Appendix 1.

Per the NQF measure specifications,⁸ records were excluded if they were from hospitals with <80% of records complete with core elements (unique patient identifier, admission date, end-of-service date, and ICD-9 primary diagnosis code). In addition, records were excluded for the following reasons: (1) individual record missing core elements, (2) discharge disposition “death,” (3) 30-day follow-up data not available, (4) primary “newborn” or mental health diagnosis, or (5) primary ICD-9 procedure code for a planned procedure or chemotherapy.

Patient characteristics for hospital admissions with and without 30-day readmissions or 30-day emergency department (ED) revisits were summarized. For the continuous variable age, mean and standard deviation for each group were calculated. For categorical variables (sex, race, payer, and number of chronic conditions), numbers and proportions were determined. Univariate tests of comparison were carried out using the Student’s t test for age and chi-square tests for all categorical variables. Categories of payer with small values were combined for ease of description (categories combined into “other:” workers’ compensation, county indigent programs, other government, other indigent, self-pay, other payer). We identified chronic conditions using the Agency for Healthcare Research and Quality Chronic Condition Indicator (CCI) system, which classifies ICD-9-CM diagnosis codes as chronic or acute and places each code into 1 of 18 mutually exclusive categories (organ systems, disease categories, or other categories). The case-mix adjustment model incorporates a binary variable for each CCI category (0-1, 2, 3, or >4 chronic conditions) per the NQF measure specifications.⁸ This study was approved by the University of California, San Francisco Institutional Review Board.

Outcomes

Our primary outcome was the hospital-level rate of 30-day readmission after hospital discharge, consistent with the NQF measure.⁸ We identified outlier hospitals for 30-day readmission rate using the Centers for Medicare and Medicaid Services (CMS) methodology, which defines outlier hospitals as those for whom adjusted readmission rate confidence intervals do not overlap with the overall group mean rate.^{5, 14}

We also determined the hospital-level average cost per index hospitalization (not including costs of readmissions). Since costs of care often differ substantially from charges,¹⁵ costs were calculated using cost-to-charge ratios for each hospital (annual total operating expenses/total gross patient revenue, as reported to the OSHPD).¹⁶ Costs were subdivided into categories representing $5,000 increments and a top category of >$40,000. Outlier hospitals for costs were defined as those for whom the cost random effect was either greater than the third quartile of the distribution of values by more than 1.5 times the interquartile range or less than the first quartile of the distribution of values by more than 1.5 times the interquartile range.¹⁷

ANALYSIS

Primary Analysis

For our primary analysis of 30-day hospital readmission rates, we used hierarchical logistic regression models with hospitals as random effects, adjusting for patient age, sex, and the presence and number of body systems affected by chronic conditions.⁸ These 4 patient characteristics were selected by the NQF measure developers “because distributions of these characteristics vary across hospitals, and although they are associated with readmission risk, they are independent of hospital quality of care.”¹⁰

Because the Centers for Medicare and Medicaid Services (CMS) are in the process of selecting pediatric quality measures for meaningful use reporting,¹⁸ we utilized CMS hospital readmissions methodology to calculate risk-adjusted rates and identify outlier hospitals. The CMS modeling strategy stabilizes performance estimates for low-volume hospitals and avoids penalizing these hospitals for high readmission rates that may be due to chance (random effects logistic model to obtain best linear unbiased predictions). This is particularly important in pediatrics, given the low pediatric volumes in many hospitals admitting children.^4,19 We then identified outlier hospitals for the 30-day readmission rate using CMS methodology (hospital’s adjusted readmission rate confidence interval does not overlap the overall group mean rate).^{5, 4} CMS uses this approach for public reporting on HospitalCompare.²⁰

Sensitivity Analyses

We tested several broadening variations of the NQF measure: (1) addition of children admitted with a primary diagnosis of asthma (without requiring LRI as a secondary diagnosis) or a secondary diagnosis of asthma exacerbation (LRIA), (2) inclusion of 30-day ED revisits as an outcome, and (3) merging of 3 years of data. These analyses were all performed using the same modeling strategy as in our primary analysis.

Secondary Outcome Analyses

Our analysis of hospital costs used costs for index admissions over 3 years (2012–2014) and included admissions for asthma. We used hierarchical regression models with hospitals as random effects, adjusting for age, gender, and the presence and number of chronic conditions. The distribution of cost values was highly skewed, so ordinal models were selected after several other modeling approaches failed (log transformation linear model, gamma model, Poisson model, zero-truncated Poisson model).

The relationship between hospital-level costs and hospital-level 30-day readmission or ED revisit rates was analyzed using Spearman’s rank correlation coefficient. Statistical analysis was performed using SAS version 9.4 software (SAS Institute; Cary, North Carolina).

Primary Analysis of 30-day Readmissions (per National Quality Forum Measure)

Our analysis of the 2014 OSHPD database using the specifications of the NQF Pediatric LRI Readmission Measure included a total of 5550 hospitalizations from 174 hospitals, with a mean of 12 eligible hospitalizations per hospital. The mean risk-adjusted readmission rate was 6.5% (362 readmissions). There were no hospitals that were considered outliers based on the risk-adjusted readmission rates (Table 1).

Sensitivity Analyses (Broadening Definitions of National Quality Forum Measure)

We report our testing of the broadened variations of the NQF measure in Table 1. Broadening the population to include children with asthma as a primary diagnosis and children with asthma exacerbations as a secondary diagnosis (LRIA) increased the size of our analysis to 8402 hospitalizations from 190 hospitals. The mean risk-adjusted readmission rate was 5.5%, and no outlier hospitals were identified.

Using the same inclusion criteria of the NQF measure but including 30-day ED revisits as an outcome, we analyzed a total of 5500 hospitalizations from 174 hospitals. The mean risk-adjusted event rate was higher at 7.9%, but there were still no outlier hospitals identified.

Using the broadened population definition (LRIA) and including 30-day ED revisits as an outcome, we analyzed a total of 8402 hospitalizations from 190 hospitals. The mean risk-adjusted event rate was 6.8%, but there were still no outlier hospitals identified.

In our final iteration, we merged 3 years of hospital data (2012-2014) using the broader population definition (LRIA) and including 30-day ED revisits as an outcome. This resulted in 27,873 admissions from 239 hospitals for this analysis, with a mean of 28 eligible hospitalizations per hospital. The mean risk-adjusted event rate was 6.7%, and this approach identified 2 high-performing (risk-adjusted rates: 3.6-5.3) and 7 low-performing hospitals (risk-adjusted rates: 10.1-15.9).

Table 2 presents the demographics of children included in this analysis. Children who had readmissions/revisits were younger, more likely to be white, less likely to have private insurance, and more likely to have a greater number of chronic conditions compared to children without readmissions/revisits.

Secondary Outcome: Hospital Costs

In the analysis of hospital-level costs, we found only 1 outlier high-cost hospital. There was a 20% probability of a hospital respiratory admission costing ≥$40,000 at this hospital. We found no overall relationship between hospital 30-day respiratory readmission rate and hospital costs (Figure 1). However, the hospitals that were outliers for low readmission rates also had low probabilities of excessive hospital costs (3% probability of costs >$40,000; Figure 2).

We used a nationally endorsed pediatric quality measure to evaluate hospital performance, defined as 30-day readmission rates for children with respiratory illness. We examined all-payer data from California, which is the most populous state in the country and home to 1 in 8 American children. In this large California dataset, we were unable to identify meaningful variation in hospital performance due to low hospital volumes and event rates. However, when we broadened the measure definition, we were able to identify performance variation. Our findings underscore the importance of testing and potentially modifying existing quality measures in order to more accurately capture the quality of care delivered at hospitals with lower volumes of pediatric patients.²¹

The ability of a nationally-endorsed pediatric respiratory readmissions measure to meaningfully identify variation in hospital performance is limited. General hospitals, which provide the majority of pediatric care for common conditions such as LRI, likely cannot be accurately evaluated using national pediatric quality metrics as they are currently designed. Modifying measures in order to increase hospital-level pediatric patient volumes may facilitate more meaningful evaluation of the quality of pediatric care in general hospitals and identification of exceptional hospitals for understanding best practices in pediatric inpatient care.

Disclosures

Regina Lam consulted for Proximity Health doing market research during the course of developing this manuscript, but this work did not involve any content related to quality metrics, and this entity did not play any role in the development of this manuscript. The remaining authors have no conflicts of interest relevant to this article to disclose.

Funding

Supported by the Agency for Healthcare Research and Quality (K08 HS24592 to SVK and U18HS25297 to MDC and NSB) and the National Institute of Child Health and Human Development (K23HD065836 to NSB). The funding agency played no role in the study design; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the manuscript for publication.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.

Transport medicine (TM) involves the provision of care to patients who require transfer to a healthcare facility that can deliver definitive treatment.¹ Pediatric interfacility transport occurs in approximately 10% of nonneonatal, nonpregnancy pediatric hospitalizations in the United States.² Studies document a decline in resident participation in pediatric transports and variability in curricular content.^3,4As a result, pediatric hospitalists, who often serve as the referring, accepting, transport, and/or medical control physician during interfacility transports,^5,6 may have gaps in training related to TM.

The Pediatric Hospital Medicine (PHM) Core Competencies include “Transport of the Critically Ill Child.”⁷ Additionally, the Curriculum Committee of the PHM Fellowship Directors Council proposed a curricular framework that includes a required clinical experience in “Care and Stabilization of the Critically Ill Child,”⁸ which can occur in a variety of practice settings, including TM. TM is also listed as a potential elective rotation.

In 2014, 60% of PHM fellowships included a required or optional TM rotation.⁹ A recent study of pediatric emergency, critical care, and neonatal medicine fellowships revealed a paucity of formal or published TM curricula in these programs.¹⁰ Furthermore, no standard or published TM curricula have been established for PHM fellowships. The primary objective of our study is to determine attitudes regarding TM training among PHM fellows, recent PHM fellowship graduates, and PHM fellowship program directors (PDs). The secondary objective is to identify how the perspectives of these fellowship stakeholders could influence the design of a TM curriculum.

This cross-sectional study focused on 3 stakeholder groups related to PHM fellowships. The subjects included in the study were physicians enrolled in a PHM fellowship (fellow) during the 2015-2016 academic year, graduates of fellowship (graduate) between 2010 and 2015, and fellowship program directors (PD). Unique web-based, anonymous surveys for each group were developed, reviewed by content and methodology experts, and piloted with local pediatric hospitalists. Surveys consisted of unfolding multiple-choice questions and ranking items along Likert scales and the Dreyfus model.

Questions were designed to elicit demographic data, perspectives, and experience related to TM education in PHM fellowships across all respondent groups. Depending on the context, identical or similar questions were asked among the groups. For example, all groups were asked to prioritize learning objectives for a TM rotation. Graduates and PDs reported the most effective teaching methods for use during a TM rotation. Fellows rated their own interest in a TM elective, and PDs were asked to rate the level of interest among their fellows.

Participant contact information was obtained from a website (phmfellows.org) and databases of fellows and graduates, which are maintained by the PHM Fellowship Directors Council (personal communication, Jayne Truckenbrod, DO; February 2, 2017). Between February and April 2016, the participants were individually emailed a link to their respective surveys, and 3 reminder e-mails were sent to nonresponders. The survey was administered through SurveyMonkey (www.surveymonkey.com).

SPSS (IBM SPSS Statistics, IBM Corporation, Armonk, New York) was used for statistical analysis. Descriptive data were presented using mean and standard deviation. Comparisons among fellows, graduates, and PDs were conducted using one-way analyses of variance or Mann-Whitney U test. Frequency of application and self-evaluation of core competency skills before and after the rotation were evaluated using paired sample t-tests. The study protocol was deemed exempt from review by our local Institutional Review Board.

Forty of 70 (57%) fellows, 32 of 87 graduates (37%), and 14 of 32 PDs (44%) responded to the survey. The majority of the participants described their respective programs as 2 years in duration (59% for fellows, 56% for graduates, and 85% for PDs). Most programs (85%) were based at children’s hospitals. Most graduates (84%) practiced in a children’s hospital, and 12% of them practiced in a community site or a combination of sites.

Both fellows and graduates reported limited involvement in several aspects of TM prior to fellowship. Fellows’ interest in completing a TM rotation during fellowship is greater than the interest as perceived by PDs (3.03+1.00 vs. 2.38+1.19, P = .061). Prior TM exposure in residency or perceived proficiency in TM was not associated with lack of interest. Twenty-five percent of graduates completed a TM rotation during PHM fellowship. Many graduates agreed (41%) or strongly agreed (16%) with the statement “I recommend participating in a TM rotation during PHM fellowship.” Graduates who had completed a TM rotation were more likely to agree with this statement (P = .001).

Survey respondents demonstrate variable levels of interest and engagement in TM training; in particular, fellows and graduates often reported greater interest and value in a TM rotation than PDs. Similar to fellows in related fields,¹⁰ PHM fellows and graduates selected clinical topics as the most essential elements of TM training. In accordance with the literature, our findings suggest that direct patient care, one-on-one instruction, and simulation would be appropriate and popular methods for delivering this type of educational content.^10,11

Curriculum design for a TM rotation should reinforce clinical PHM competencies related to TM while focusing on topics that are specific to the transport environment, such as methods of interfacility transport, handoffs, transitions of care, and team leadership.^2,7,12 Trainee comfort level with different forms of transport (eg, fear of flying, motion sickness) and local and state policies regarding interfacility transfer should also be considered. In addition, fellows could engage in clinical research and quality improvement projects related to TM given the overall paucity of literature in the field.¹³

Several reasons can explain why fellows and graduates place a greater value on a TM rotation than PDs. Fellows and graduates may perceive inherent value in gaining particular knowledge and skills, such as greater understanding of the logistics and personnel involved in transferring patients and experience working with a healthcare team in a unique and dynamic setting.^3,10,14Meanwhile, PDs may have had limited personal exposure to TM or may underestimate the limited exposure to TM for fellows while in residency.³

PDs may not be aware of the extent of participation in elements of transport among graduates. A recent workforce survey of pediatric interfacility transport systems indicated that although medical directors are from the fields of emergency, critical care, and neonatal medicine, 20% of medical control physicians are pediatric hospitalists.⁴ Given that the majority of PHM fellowships are based at children’s hospitals and transport teams are often associated with intensive care or emergency medicine units, PDs may have limited exposure to transport systems that incorporate hospitalists.

Pediatric hospitalists at all practice sites must have clinical and systems skills related to TM. However, the scope of practice for those working at community sites may be more likely to include distinct elements of TM.⁶ Currently, most fellowship graduates work at free-standing children’s or university-affiliated hospitals and have pursued careers in academic medicine.¹⁵ As the field evolves, the number of fellowship-trained pediatric hospitalists working at community sites may increase, making the acquisition of skills relevant to TM during fellowship training more crucial.

This study has several limitations. We attempted to identify all recent PHM fellowship graduates, but sampling bias may exist. Response bias may have been introduced by the self-reporting of skill and proficiency as well as by the small sample size and response rate for some stakeholder groups. The latter may be exacerbated by the fact that we do not have data on the degree or distribution of program representation among the fellow and graduate groups, given the lack of identifying information collected. Finally, we did not collect specific information about existing TM curricula in PHM fellowships.

We report a variable level of interest and engagement in TM among fellowship stakeholders, even though “Transport of the Critically Ill Child” is a PHM Core Competency. Fellows are interested in TM but unsure of its relevance to a PHM career. Graduates support the acquisition of transport skills during fellowship training. We found agreement about the opportunity to teach core PHM knowledge and skills through a TM experience. Formal curricula, locally and nationally, could improve trainees’ transport skills and provide a means for addressing an essential component of the proposed PHM fellowship curricular framework.

The authors would like to thank Tony Woodward, MD for reviewing the survey tools; Sheree Schrager, PhD and Margaret Trost, MD for their valuable insights into the results; and Grant Christman, MD for reviewing the manuscript.

Disclosures

The authors declare no potential conflicts of interest.

Funding

No funding was secured for this study.

Transport medicine (TM) involves the provision of care to patients who require transfer to a healthcare facility that can deliver definitive treatment.¹ Pediatric interfacility transport occurs in approximately 10% of nonneonatal, nonpregnancy pediatric hospitalizations in the United States.² Studies document a decline in resident participation in pediatric transports and variability in curricular content.^3,4As a result, pediatric hospitalists, who often serve as the referring, accepting, transport, and/or medical control physician during interfacility transports,^5,6 may have gaps in training related to TM.

The Pediatric Hospital Medicine (PHM) Core Competencies include “Transport of the Critically Ill Child.”⁷ Additionally, the Curriculum Committee of the PHM Fellowship Directors Council proposed a curricular framework that includes a required clinical experience in “Care and Stabilization of the Critically Ill Child,”⁸ which can occur in a variety of practice settings, including TM. TM is also listed as a potential elective rotation.

In 2014, 60% of PHM fellowships included a required or optional TM rotation.⁹ A recent study of pediatric emergency, critical care, and neonatal medicine fellowships revealed a paucity of formal or published TM curricula in these programs.¹⁰ Furthermore, no standard or published TM curricula have been established for PHM fellowships. The primary objective of our study is to determine attitudes regarding TM training among PHM fellows, recent PHM fellowship graduates, and PHM fellowship program directors (PDs). The secondary objective is to identify how the perspectives of these fellowship stakeholders could influence the design of a TM curriculum.

This cross-sectional study focused on 3 stakeholder groups related to PHM fellowships. The subjects included in the study were physicians enrolled in a PHM fellowship (fellow) during the 2015-2016 academic year, graduates of fellowship (graduate) between 2010 and 2015, and fellowship program directors (PD). Unique web-based, anonymous surveys for each group were developed, reviewed by content and methodology experts, and piloted with local pediatric hospitalists. Surveys consisted of unfolding multiple-choice questions and ranking items along Likert scales and the Dreyfus model.

Questions were designed to elicit demographic data, perspectives, and experience related to TM education in PHM fellowships across all respondent groups. Depending on the context, identical or similar questions were asked among the groups. For example, all groups were asked to prioritize learning objectives for a TM rotation. Graduates and PDs reported the most effective teaching methods for use during a TM rotation. Fellows rated their own interest in a TM elective, and PDs were asked to rate the level of interest among their fellows.

Participant contact information was obtained from a website (phmfellows.org) and databases of fellows and graduates, which are maintained by the PHM Fellowship Directors Council (personal communication, Jayne Truckenbrod, DO; February 2, 2017). Between February and April 2016, the participants were individually emailed a link to their respective surveys, and 3 reminder e-mails were sent to nonresponders. The survey was administered through SurveyMonkey (www.surveymonkey.com).

SPSS (IBM SPSS Statistics, IBM Corporation, Armonk, New York) was used for statistical analysis. Descriptive data were presented using mean and standard deviation. Comparisons among fellows, graduates, and PDs were conducted using one-way analyses of variance or Mann-Whitney U test. Frequency of application and self-evaluation of core competency skills before and after the rotation were evaluated using paired sample t-tests. The study protocol was deemed exempt from review by our local Institutional Review Board.

Forty of 70 (57%) fellows, 32 of 87 graduates (37%), and 14 of 32 PDs (44%) responded to the survey. The majority of the participants described their respective programs as 2 years in duration (59% for fellows, 56% for graduates, and 85% for PDs). Most programs (85%) were based at children’s hospitals. Most graduates (84%) practiced in a children’s hospital, and 12% of them practiced in a community site or a combination of sites.

Both fellows and graduates reported limited involvement in several aspects of TM prior to fellowship. Fellows’ interest in completing a TM rotation during fellowship is greater than the interest as perceived by PDs (3.03+1.00 vs. 2.38+1.19, P = .061). Prior TM exposure in residency or perceived proficiency in TM was not associated with lack of interest. Twenty-five percent of graduates completed a TM rotation during PHM fellowship. Many graduates agreed (41%) or strongly agreed (16%) with the statement “I recommend participating in a TM rotation during PHM fellowship.” Graduates who had completed a TM rotation were more likely to agree with this statement (P = .001).

Survey respondents demonstrate variable levels of interest and engagement in TM training; in particular, fellows and graduates often reported greater interest and value in a TM rotation than PDs. Similar to fellows in related fields,¹⁰ PHM fellows and graduates selected clinical topics as the most essential elements of TM training. In accordance with the literature, our findings suggest that direct patient care, one-on-one instruction, and simulation would be appropriate and popular methods for delivering this type of educational content.^10,11

Curriculum design for a TM rotation should reinforce clinical PHM competencies related to TM while focusing on topics that are specific to the transport environment, such as methods of interfacility transport, handoffs, transitions of care, and team leadership.^2,7,12 Trainee comfort level with different forms of transport (eg, fear of flying, motion sickness) and local and state policies regarding interfacility transfer should also be considered. In addition, fellows could engage in clinical research and quality improvement projects related to TM given the overall paucity of literature in the field.¹³

Several reasons can explain why fellows and graduates place a greater value on a TM rotation than PDs. Fellows and graduates may perceive inherent value in gaining particular knowledge and skills, such as greater understanding of the logistics and personnel involved in transferring patients and experience working with a healthcare team in a unique and dynamic setting.^3,10,14Meanwhile, PDs may have had limited personal exposure to TM or may underestimate the limited exposure to TM for fellows while in residency.³

PDs may not be aware of the extent of participation in elements of transport among graduates. A recent workforce survey of pediatric interfacility transport systems indicated that although medical directors are from the fields of emergency, critical care, and neonatal medicine, 20% of medical control physicians are pediatric hospitalists.⁴ Given that the majority of PHM fellowships are based at children’s hospitals and transport teams are often associated with intensive care or emergency medicine units, PDs may have limited exposure to transport systems that incorporate hospitalists.

Pediatric hospitalists at all practice sites must have clinical and systems skills related to TM. However, the scope of practice for those working at community sites may be more likely to include distinct elements of TM.⁶ Currently, most fellowship graduates work at free-standing children’s or university-affiliated hospitals and have pursued careers in academic medicine.¹⁵ As the field evolves, the number of fellowship-trained pediatric hospitalists working at community sites may increase, making the acquisition of skills relevant to TM during fellowship training more crucial.

This study has several limitations. We attempted to identify all recent PHM fellowship graduates, but sampling bias may exist. Response bias may have been introduced by the self-reporting of skill and proficiency as well as by the small sample size and response rate for some stakeholder groups. The latter may be exacerbated by the fact that we do not have data on the degree or distribution of program representation among the fellow and graduate groups, given the lack of identifying information collected. Finally, we did not collect specific information about existing TM curricula in PHM fellowships.

We report a variable level of interest and engagement in TM among fellowship stakeholders, even though “Transport of the Critically Ill Child” is a PHM Core Competency. Fellows are interested in TM but unsure of its relevance to a PHM career. Graduates support the acquisition of transport skills during fellowship training. We found agreement about the opportunity to teach core PHM knowledge and skills through a TM experience. Formal curricula, locally and nationally, could improve trainees’ transport skills and provide a means for addressing an essential component of the proposed PHM fellowship curricular framework.

The authors would like to thank Tony Woodward, MD for reviewing the survey tools; Sheree Schrager, PhD and Margaret Trost, MD for their valuable insights into the results; and Grant Christman, MD for reviewing the manuscript.

Disclosures

The authors declare no potential conflicts of interest.

Funding

No funding was secured for this study.

Transport medicine (TM) involves the provision of care to patients who require transfer to a healthcare facility that can deliver definitive treatment.¹ Pediatric interfacility transport occurs in approximately 10% of nonneonatal, nonpregnancy pediatric hospitalizations in the United States.² Studies document a decline in resident participation in pediatric transports and variability in curricular content.^3,4As a result, pediatric hospitalists, who often serve as the referring, accepting, transport, and/or medical control physician during interfacility transports,^5,6 may have gaps in training related to TM.

The Pediatric Hospital Medicine (PHM) Core Competencies include “Transport of the Critically Ill Child.”⁷ Additionally, the Curriculum Committee of the PHM Fellowship Directors Council proposed a curricular framework that includes a required clinical experience in “Care and Stabilization of the Critically Ill Child,”⁸ which can occur in a variety of practice settings, including TM. TM is also listed as a potential elective rotation.

In 2014, 60% of PHM fellowships included a required or optional TM rotation.⁹ A recent study of pediatric emergency, critical care, and neonatal medicine fellowships revealed a paucity of formal or published TM curricula in these programs.¹⁰ Furthermore, no standard or published TM curricula have been established for PHM fellowships. The primary objective of our study is to determine attitudes regarding TM training among PHM fellows, recent PHM fellowship graduates, and PHM fellowship program directors (PDs). The secondary objective is to identify how the perspectives of these fellowship stakeholders could influence the design of a TM curriculum.

This cross-sectional study focused on 3 stakeholder groups related to PHM fellowships. The subjects included in the study were physicians enrolled in a PHM fellowship (fellow) during the 2015-2016 academic year, graduates of fellowship (graduate) between 2010 and 2015, and fellowship program directors (PD). Unique web-based, anonymous surveys for each group were developed, reviewed by content and methodology experts, and piloted with local pediatric hospitalists. Surveys consisted of unfolding multiple-choice questions and ranking items along Likert scales and the Dreyfus model.

Questions were designed to elicit demographic data, perspectives, and experience related to TM education in PHM fellowships across all respondent groups. Depending on the context, identical or similar questions were asked among the groups. For example, all groups were asked to prioritize learning objectives for a TM rotation. Graduates and PDs reported the most effective teaching methods for use during a TM rotation. Fellows rated their own interest in a TM elective, and PDs were asked to rate the level of interest among their fellows.

Participant contact information was obtained from a website (phmfellows.org) and databases of fellows and graduates, which are maintained by the PHM Fellowship Directors Council (personal communication, Jayne Truckenbrod, DO; February 2, 2017). Between February and April 2016, the participants were individually emailed a link to their respective surveys, and 3 reminder e-mails were sent to nonresponders. The survey was administered through SurveyMonkey (www.surveymonkey.com).

SPSS (IBM SPSS Statistics, IBM Corporation, Armonk, New York) was used for statistical analysis. Descriptive data were presented using mean and standard deviation. Comparisons among fellows, graduates, and PDs were conducted using one-way analyses of variance or Mann-Whitney U test. Frequency of application and self-evaluation of core competency skills before and after the rotation were evaluated using paired sample t-tests. The study protocol was deemed exempt from review by our local Institutional Review Board.

Forty of 70 (57%) fellows, 32 of 87 graduates (37%), and 14 of 32 PDs (44%) responded to the survey. The majority of the participants described their respective programs as 2 years in duration (59% for fellows, 56% for graduates, and 85% for PDs). Most programs (85%) were based at children’s hospitals. Most graduates (84%) practiced in a children’s hospital, and 12% of them practiced in a community site or a combination of sites.

Both fellows and graduates reported limited involvement in several aspects of TM prior to fellowship. Fellows’ interest in completing a TM rotation during fellowship is greater than the interest as perceived by PDs (3.03+1.00 vs. 2.38+1.19, P = .061). Prior TM exposure in residency or perceived proficiency in TM was not associated with lack of interest. Twenty-five percent of graduates completed a TM rotation during PHM fellowship. Many graduates agreed (41%) or strongly agreed (16%) with the statement “I recommend participating in a TM rotation during PHM fellowship.” Graduates who had completed a TM rotation were more likely to agree with this statement (P = .001).

Survey respondents demonstrate variable levels of interest and engagement in TM training; in particular, fellows and graduates often reported greater interest and value in a TM rotation than PDs. Similar to fellows in related fields,¹⁰ PHM fellows and graduates selected clinical topics as the most essential elements of TM training. In accordance with the literature, our findings suggest that direct patient care, one-on-one instruction, and simulation would be appropriate and popular methods for delivering this type of educational content.^10,11

Curriculum design for a TM rotation should reinforce clinical PHM competencies related to TM while focusing on topics that are specific to the transport environment, such as methods of interfacility transport, handoffs, transitions of care, and team leadership.^2,7,12 Trainee comfort level with different forms of transport (eg, fear of flying, motion sickness) and local and state policies regarding interfacility transfer should also be considered. In addition, fellows could engage in clinical research and quality improvement projects related to TM given the overall paucity of literature in the field.¹³

Several reasons can explain why fellows and graduates place a greater value on a TM rotation than PDs. Fellows and graduates may perceive inherent value in gaining particular knowledge and skills, such as greater understanding of the logistics and personnel involved in transferring patients and experience working with a healthcare team in a unique and dynamic setting.^3,10,14Meanwhile, PDs may have had limited personal exposure to TM or may underestimate the limited exposure to TM for fellows while in residency.³

PDs may not be aware of the extent of participation in elements of transport among graduates. A recent workforce survey of pediatric interfacility transport systems indicated that although medical directors are from the fields of emergency, critical care, and neonatal medicine, 20% of medical control physicians are pediatric hospitalists.⁴ Given that the majority of PHM fellowships are based at children’s hospitals and transport teams are often associated with intensive care or emergency medicine units, PDs may have limited exposure to transport systems that incorporate hospitalists.

Pediatric hospitalists at all practice sites must have clinical and systems skills related to TM. However, the scope of practice for those working at community sites may be more likely to include distinct elements of TM.⁶ Currently, most fellowship graduates work at free-standing children’s or university-affiliated hospitals and have pursued careers in academic medicine.¹⁵ As the field evolves, the number of fellowship-trained pediatric hospitalists working at community sites may increase, making the acquisition of skills relevant to TM during fellowship training more crucial.

This study has several limitations. We attempted to identify all recent PHM fellowship graduates, but sampling bias may exist. Response bias may have been introduced by the self-reporting of skill and proficiency as well as by the small sample size and response rate for some stakeholder groups. The latter may be exacerbated by the fact that we do not have data on the degree or distribution of program representation among the fellow and graduate groups, given the lack of identifying information collected. Finally, we did not collect specific information about existing TM curricula in PHM fellowships.

We report a variable level of interest and engagement in TM among fellowship stakeholders, even though “Transport of the Critically Ill Child” is a PHM Core Competency. Fellows are interested in TM but unsure of its relevance to a PHM career. Graduates support the acquisition of transport skills during fellowship training. We found agreement about the opportunity to teach core PHM knowledge and skills through a TM experience. Formal curricula, locally and nationally, could improve trainees’ transport skills and provide a means for addressing an essential component of the proposed PHM fellowship curricular framework.

The authors would like to thank Tony Woodward, MD for reviewing the survey tools; Sheree Schrager, PhD and Margaret Trost, MD for their valuable insights into the results; and Grant Christman, MD for reviewing the manuscript.

Disclosures

The authors declare no potential conflicts of interest.

Funding

No funding was secured for this study.