Hospital Medicine

Hospitalized patients with pneumonia who develop a complicated parapneumonic effusion or empyema need to undergo chest tube placement.

WHAT IS PARAPNEUMONIC EFFUSION?

Parapneumonic effusion is a pleural effusion that forms concurrently with bacterial or viral pneumonia. Up to 40% of patients hospitalized with pneumonia develop a parapneumonic effusion.¹ The effusion progresses through a continuum of 3 stages: uncomplicated, complicated, and empyema.

Uncomplicated parapneumonic effusion is an exudative effusion without bacteria or pus that is caused by movement of fluid and neutrophils into the pleural space. Pneumonia itself causes an increase in interstitial fluid and capillary leakage. The effusion becomes complicated as a result of bacteria invading the pleural space, causing a further increase in neutrophils in the pleural fluid. Empyema is defined as the presence of frank pus in the pleural space.

CLINICAL SIGNIFICANCE

According to the US Centers for Disease Control and Prevention, pneumonia accounts for 674,000 emergency department visits each year; of the patients hospitalized, up to 40% develop a parapneumonic effusion.² The only study done on rates of death associated with parapneumonic effusion showed that, compared with patients with no effusion, the risk of death was 3.7 times higher with a unilateral effusion and 6.5 times higher with bilateral effusions.³

INITIAL EVALUATION

The initial evaluation of suspected parapneumonic effusion should include chest radiography with lateral or decubitus views, followed by thoracentesis if indicated. If thoracentesis is performed, the fluid should be tested as follows:

• Gram stain
• Appropriate cultures based on clinical scenario (eg, aerobic, anaerobic, fungal)
• Total protein in pleural fluid and serum
• Lactate dehydrogenase (LDH) in pleural fluid and serum
• Glucose
• pH.

CLASSIFICATION OF EFFUSIONS

When pleural fluid is obtained, the total protein and LDH levels are used to categorize the effusion as either transudative or exudative based on the Light criteria.⁴ An effusion is confirmed as exudative when 1 of the following 3 criteria is met:

• The ratio of pleural fluid protein to serum protein is greater than 0.5
• The ratio of pleural fluid LDH to serum LDH is greater than 0.6
• The pleural fluid LDH is greater than two-thirds the upper limit of normal for the serum LDH.

Once an effusion is characterized as exudative, the American College of Chest Physicians consensus guidelines are used for further classification and prognostication.⁵ The guidelines estimate the risk of poor outcome by dividing the effusions based on their characteristics: the anatomy of the effusion, the bacteriology of the fluid, and the chemistry of the pleural fluid (Table 1). Based on these guidelines, effusions are divided into 4 categories. Category 1 and 2 effusions are considered uncomplicated, category 3 is considered complicated, and category 4 is empyema. The outcomes of interest are prolonged hospitalization, prolonged systemic toxicity, increased ventilator impairment, increased morbidity rate, and increased mortality rate.⁵

Category 1 effusions are defined as free- flowing fluid with a thickness of less than 10 mm on any imaging modality. Thoracentesis for pleural fluid analysis is not required. The prognosis is very good.

Category 2 effusions are defined as free- flowing fluid with a thickness greater than 10 mm and less than 50% of the hemithorax. Thoracentesis is typically done because of the size of the effusion, but Gram stain and culture of the pleural fluid are usually negative, and the pH is at least 7.2. The prognosis is good.

Category 3 effusions are considered complicated because the anatomy of the pleural space becomes altered or because bacteria have invaded the pleural space. The effusion is larger than 50% of the hemithorax or is loculated, or the parietal pleura is thickened. Since the bacteria have invaded the pleural space, Gram stain or culture of pleural fluid may be positive, the pleural fluid pH may be less than 7.2, or the glucose level of the fluid may be less than 60 mg/dL. The prognosis for category 3 is poor.

Category 4 effusions are defined as empyema. The only characteristic that separates this from category 3 is frank pus in the pleural space. The prognosis is very poor.

 

 

TO PLACE A CHEST TUBE OR NOT

For category 1 or 2 effusions, treatment with antibiotics alone is typically enough. Category 3 effusions usually do not respond to antibiotics alone and may require complete drainage of the fluid with or without a chest tube depending on whether loculations are present, as loculations are difficult to drain with a chest tube. Category 4 effusions require both antibiotics and chest tube placement.

WHAT TYPE OF CHEST TUBE?

Studies have shown that small-bore chest tubes (< 20 F) are as efficacious as larger tubes (≥ 20 F) for the treatment of complicated parapneumonic effusion and empyema.^6,7 Studies have also shown that the size of the tube makes no difference in the time needed to drain the effusion, the length of hospital stay, or the complication rate.^8,9 Based on these studies, a small-bore chest tube should be placed first when clinically appropriate. When a chest tube is placed for empyema, computed tomography should be performed within 24 hours to confirm proper tube placement.

ADVANCED THERAPIES FOR EMPYEMA

Empyema treatment fails when antibiotic coverage is inadequate or when a loculation is not drained appropriately. Options if treatment fails include instillation of fibrinolytics into the pleural space, video-assisted thora­scopic surgery, and decortication.

The role of fibrinolytics has not been well-established, but fibrinolytics should be considered in loculated effusions or empyema, or if drainage of the effusion slows.¹⁰ Video-assisted thora­scopic surgery is reserved for effusions that are incompletely drained with a chest tube with or without fibrinolytics; studies have shown shorter hospital length of stay and higher treatment efficacy when this is performed earlier for loculated effusions.¹¹ Decortication is reserved for symptomatic patients who have a thickened pleura more than 6 months after the initial infection.¹² Timing for each of these procedures is not clearly defined and so must be individualized.

TAKE-AWAY POINTS

• Parapneumonic effusion occurs concurrently with pneumonia and with a high frequency.¹
• Effusions are associated with an increased risk of death.³
• Categorizing the effusion helps guide treatment.
• Chest tubes should be placed for some cases of complicated effusion and for all cases of empyema.
• A small-bore chest tube (< 20 F) should be tried first.

Hospitalized patients with pneumonia who develop a complicated parapneumonic effusion or empyema need to undergo chest tube placement.

WHAT IS PARAPNEUMONIC EFFUSION?

Parapneumonic effusion is a pleural effusion that forms concurrently with bacterial or viral pneumonia. Up to 40% of patients hospitalized with pneumonia develop a parapneumonic effusion.¹ The effusion progresses through a continuum of 3 stages: uncomplicated, complicated, and empyema.

Uncomplicated parapneumonic effusion is an exudative effusion without bacteria or pus that is caused by movement of fluid and neutrophils into the pleural space. Pneumonia itself causes an increase in interstitial fluid and capillary leakage. The effusion becomes complicated as a result of bacteria invading the pleural space, causing a further increase in neutrophils in the pleural fluid. Empyema is defined as the presence of frank pus in the pleural space.

CLINICAL SIGNIFICANCE

According to the US Centers for Disease Control and Prevention, pneumonia accounts for 674,000 emergency department visits each year; of the patients hospitalized, up to 40% develop a parapneumonic effusion.² The only study done on rates of death associated with parapneumonic effusion showed that, compared with patients with no effusion, the risk of death was 3.7 times higher with a unilateral effusion and 6.5 times higher with bilateral effusions.³

INITIAL EVALUATION

The initial evaluation of suspected parapneumonic effusion should include chest radiography with lateral or decubitus views, followed by thoracentesis if indicated. If thoracentesis is performed, the fluid should be tested as follows:

• Gram stain
• Appropriate cultures based on clinical scenario (eg, aerobic, anaerobic, fungal)
• Total protein in pleural fluid and serum
• Lactate dehydrogenase (LDH) in pleural fluid and serum
• Glucose
• pH.

CLASSIFICATION OF EFFUSIONS

When pleural fluid is obtained, the total protein and LDH levels are used to categorize the effusion as either transudative or exudative based on the Light criteria.⁴ An effusion is confirmed as exudative when 1 of the following 3 criteria is met:

• The ratio of pleural fluid protein to serum protein is greater than 0.5
• The ratio of pleural fluid LDH to serum LDH is greater than 0.6
• The pleural fluid LDH is greater than two-thirds the upper limit of normal for the serum LDH.

Once an effusion is characterized as exudative, the American College of Chest Physicians consensus guidelines are used for further classification and prognostication.⁵ The guidelines estimate the risk of poor outcome by dividing the effusions based on their characteristics: the anatomy of the effusion, the bacteriology of the fluid, and the chemistry of the pleural fluid (Table 1). Based on these guidelines, effusions are divided into 4 categories. Category 1 and 2 effusions are considered uncomplicated, category 3 is considered complicated, and category 4 is empyema. The outcomes of interest are prolonged hospitalization, prolonged systemic toxicity, increased ventilator impairment, increased morbidity rate, and increased mortality rate.⁵

Category 1 effusions are defined as free- flowing fluid with a thickness of less than 10 mm on any imaging modality. Thoracentesis for pleural fluid analysis is not required. The prognosis is very good.

Category 2 effusions are defined as free- flowing fluid with a thickness greater than 10 mm and less than 50% of the hemithorax. Thoracentesis is typically done because of the size of the effusion, but Gram stain and culture of the pleural fluid are usually negative, and the pH is at least 7.2. The prognosis is good.

Category 3 effusions are considered complicated because the anatomy of the pleural space becomes altered or because bacteria have invaded the pleural space. The effusion is larger than 50% of the hemithorax or is loculated, or the parietal pleura is thickened. Since the bacteria have invaded the pleural space, Gram stain or culture of pleural fluid may be positive, the pleural fluid pH may be less than 7.2, or the glucose level of the fluid may be less than 60 mg/dL. The prognosis for category 3 is poor.

Category 4 effusions are defined as empyema. The only characteristic that separates this from category 3 is frank pus in the pleural space. The prognosis is very poor.

 

 

TO PLACE A CHEST TUBE OR NOT

For category 1 or 2 effusions, treatment with antibiotics alone is typically enough. Category 3 effusions usually do not respond to antibiotics alone and may require complete drainage of the fluid with or without a chest tube depending on whether loculations are present, as loculations are difficult to drain with a chest tube. Category 4 effusions require both antibiotics and chest tube placement.

WHAT TYPE OF CHEST TUBE?

Studies have shown that small-bore chest tubes (< 20 F) are as efficacious as larger tubes (≥ 20 F) for the treatment of complicated parapneumonic effusion and empyema.^6,7 Studies have also shown that the size of the tube makes no difference in the time needed to drain the effusion, the length of hospital stay, or the complication rate.^8,9 Based on these studies, a small-bore chest tube should be placed first when clinically appropriate. When a chest tube is placed for empyema, computed tomography should be performed within 24 hours to confirm proper tube placement.

ADVANCED THERAPIES FOR EMPYEMA

Empyema treatment fails when antibiotic coverage is inadequate or when a loculation is not drained appropriately. Options if treatment fails include instillation of fibrinolytics into the pleural space, video-assisted thora­scopic surgery, and decortication.

The role of fibrinolytics has not been well-established, but fibrinolytics should be considered in loculated effusions or empyema, or if drainage of the effusion slows.¹⁰ Video-assisted thora­scopic surgery is reserved for effusions that are incompletely drained with a chest tube with or without fibrinolytics; studies have shown shorter hospital length of stay and higher treatment efficacy when this is performed earlier for loculated effusions.¹¹ Decortication is reserved for symptomatic patients who have a thickened pleura more than 6 months after the initial infection.¹² Timing for each of these procedures is not clearly defined and so must be individualized.

TAKE-AWAY POINTS

• Parapneumonic effusion occurs concurrently with pneumonia and with a high frequency.¹
• Effusions are associated with an increased risk of death.³
• Categorizing the effusion helps guide treatment.
• Chest tubes should be placed for some cases of complicated effusion and for all cases of empyema.
• A small-bore chest tube (< 20 F) should be tried first.

Hospitalized patients with pneumonia who develop a complicated parapneumonic effusion or empyema need to undergo chest tube placement.

WHAT IS PARAPNEUMONIC EFFUSION?

Parapneumonic effusion is a pleural effusion that forms concurrently with bacterial or viral pneumonia. Up to 40% of patients hospitalized with pneumonia develop a parapneumonic effusion.¹ The effusion progresses through a continuum of 3 stages: uncomplicated, complicated, and empyema.

Uncomplicated parapneumonic effusion is an exudative effusion without bacteria or pus that is caused by movement of fluid and neutrophils into the pleural space. Pneumonia itself causes an increase in interstitial fluid and capillary leakage. The effusion becomes complicated as a result of bacteria invading the pleural space, causing a further increase in neutrophils in the pleural fluid. Empyema is defined as the presence of frank pus in the pleural space.

CLINICAL SIGNIFICANCE

According to the US Centers for Disease Control and Prevention, pneumonia accounts for 674,000 emergency department visits each year; of the patients hospitalized, up to 40% develop a parapneumonic effusion.² The only study done on rates of death associated with parapneumonic effusion showed that, compared with patients with no effusion, the risk of death was 3.7 times higher with a unilateral effusion and 6.5 times higher with bilateral effusions.³

INITIAL EVALUATION

The initial evaluation of suspected parapneumonic effusion should include chest radiography with lateral or decubitus views, followed by thoracentesis if indicated. If thoracentesis is performed, the fluid should be tested as follows:

• Gram stain
• Appropriate cultures based on clinical scenario (eg, aerobic, anaerobic, fungal)
• Total protein in pleural fluid and serum
• Lactate dehydrogenase (LDH) in pleural fluid and serum
• Glucose
• pH.

CLASSIFICATION OF EFFUSIONS

When pleural fluid is obtained, the total protein and LDH levels are used to categorize the effusion as either transudative or exudative based on the Light criteria.⁴ An effusion is confirmed as exudative when 1 of the following 3 criteria is met:

• The ratio of pleural fluid protein to serum protein is greater than 0.5
• The ratio of pleural fluid LDH to serum LDH is greater than 0.6
• The pleural fluid LDH is greater than two-thirds the upper limit of normal for the serum LDH.

Once an effusion is characterized as exudative, the American College of Chest Physicians consensus guidelines are used for further classification and prognostication.⁵ The guidelines estimate the risk of poor outcome by dividing the effusions based on their characteristics: the anatomy of the effusion, the bacteriology of the fluid, and the chemistry of the pleural fluid (Table 1). Based on these guidelines, effusions are divided into 4 categories. Category 1 and 2 effusions are considered uncomplicated, category 3 is considered complicated, and category 4 is empyema. The outcomes of interest are prolonged hospitalization, prolonged systemic toxicity, increased ventilator impairment, increased morbidity rate, and increased mortality rate.⁵

Category 1 effusions are defined as free- flowing fluid with a thickness of less than 10 mm on any imaging modality. Thoracentesis for pleural fluid analysis is not required. The prognosis is very good.

Category 2 effusions are defined as free- flowing fluid with a thickness greater than 10 mm and less than 50% of the hemithorax. Thoracentesis is typically done because of the size of the effusion, but Gram stain and culture of the pleural fluid are usually negative, and the pH is at least 7.2. The prognosis is good.

Category 3 effusions are considered complicated because the anatomy of the pleural space becomes altered or because bacteria have invaded the pleural space. The effusion is larger than 50% of the hemithorax or is loculated, or the parietal pleura is thickened. Since the bacteria have invaded the pleural space, Gram stain or culture of pleural fluid may be positive, the pleural fluid pH may be less than 7.2, or the glucose level of the fluid may be less than 60 mg/dL. The prognosis for category 3 is poor.

Category 4 effusions are defined as empyema. The only characteristic that separates this from category 3 is frank pus in the pleural space. The prognosis is very poor.

 

 

TO PLACE A CHEST TUBE OR NOT

For category 1 or 2 effusions, treatment with antibiotics alone is typically enough. Category 3 effusions usually do not respond to antibiotics alone and may require complete drainage of the fluid with or without a chest tube depending on whether loculations are present, as loculations are difficult to drain with a chest tube. Category 4 effusions require both antibiotics and chest tube placement.

WHAT TYPE OF CHEST TUBE?

Studies have shown that small-bore chest tubes (< 20 F) are as efficacious as larger tubes (≥ 20 F) for the treatment of complicated parapneumonic effusion and empyema.^6,7 Studies have also shown that the size of the tube makes no difference in the time needed to drain the effusion, the length of hospital stay, or the complication rate.^8,9 Based on these studies, a small-bore chest tube should be placed first when clinically appropriate. When a chest tube is placed for empyema, computed tomography should be performed within 24 hours to confirm proper tube placement.

ADVANCED THERAPIES FOR EMPYEMA

Empyema treatment fails when antibiotic coverage is inadequate or when a loculation is not drained appropriately. Options if treatment fails include instillation of fibrinolytics into the pleural space, video-assisted thora­scopic surgery, and decortication.

The role of fibrinolytics has not been well-established, but fibrinolytics should be considered in loculated effusions or empyema, or if drainage of the effusion slows.¹⁰ Video-assisted thora­scopic surgery is reserved for effusions that are incompletely drained with a chest tube with or without fibrinolytics; studies have shown shorter hospital length of stay and higher treatment efficacy when this is performed earlier for loculated effusions.¹¹ Decortication is reserved for symptomatic patients who have a thickened pleura more than 6 months after the initial infection.¹² Timing for each of these procedures is not clearly defined and so must be individualized.

TAKE-AWAY POINTS

• Parapneumonic effusion occurs concurrently with pneumonia and with a high frequency.¹
• Effusions are associated with an increased risk of death.³
• Categorizing the effusion helps guide treatment.
• Chest tubes should be placed for some cases of complicated effusion and for all cases of empyema.
• A small-bore chest tube (< 20 F) should be tried first.

Peripherally inserted central catheters (PICCs) have evolved since their inception in the early 1970s and are used with increasing frequency for pediatric inpatients and outpatients.^1-3 Emerging literature, including a meta-analysis of international observational studies,⁴ reports PICC failure (complications necessitating premature removal) occurs in up to 30% of PICCs, most commonly due to infection, thrombosis, occlusion, and fracture.^4-7 Raffini et al.⁷ report the increasing incidence of pediatric PICC-related thrombosis increases morbidity and mortality⁸ and negatively impacts future vessel health and preservation.⁹

PICCs have progressed from relatively simple, silicone-based catheters with an external clamp to chemically engineered polyurethane with pressure-activated valves placed at the proximal or distal catheter hub with the intent to reduce occlusion.¹⁰ Further modernization of PICC material occurred with the incorporation of antithrombogenic (AT) material (Endexo^®). These PICCs are designed to contain a nonstick polymer, which is designed to reduce the adherence of blood components (platelets and clotting factors) and inhibit thrombus formation (and hence prevent deep vein thrombosis andocclusion, as well as inhibit microbial biofilm attachment [and subsequent infection]).¹¹

In addition to new materials, other aspects of this PICC design have been the addition of a pressure-activated safety valve (PASV^®) built into the proximal hub. Pressure-activated valve technology promises to prevent catheter occlusion by reducing blood reflux into the PICC; the valve opens with pressure during infusion and aspiration and remains closed with normal venous pressure, circumventing the need for clinicians to manually clamp the PICC and reducing human error and the potential for thrombosis, occlusion, and fracture development.¹² Hoffer et al.¹³ reported half as many occlusions of valved PICCs (3.3%) compared with nonvalved or clamped PICCs (7.1%); although not statistically significant (P = .10), perhaps due to the small sample, overall complications, including occlusion and infection, were significantly lessened with the valved PICC (35% vs 79%; P = .02). Comparatively, Pittiruti et al.¹⁴ conducted a trial of 2 types of valved PICCs with an open-ended, nonvalved PICC and found no reduction in PICC occlusion or catheter malfunction.

Today, PICC use is common for patients who require short-to-medium intravenous therapy. PICCs are increasingly recognized for their significant complications, including thrombosis and infection.¹⁵ Novel PICC technology, including the incorporation of AT material such as Endexo^® and PASV^®, may reduce complications; however, the clinical efficacy, cost-effectiveness, and acceptability of these innovations have not been tested through randomized trials in pediatric patients. In accordance with Medical Research Council guidelines¹⁶ for developing interventions, we pilot tested the feasibility of the BioFlo^® PICC, including intervention acceptability, compliance, recruitment, and initial estimates of effect, in anticipation of a subsequent full-scale efficacy randomized controlled trial. Our secondary aim was to compare the effectiveness of the BioFlo^® PICC with Endexo^® and PASV^® technology in reducing PICC complications and failure.

METHODS

Design

We undertook a pilot randomized controlled trial comparing the standard polyurethane PICC (with external clamp) with the BioFlo^® PICC (with internal valve) in preventing catheter failure in pediatric patients. The study was prospectively registered with the Australian Clinical Trials Registry (ACTRN12615001290583), and the research protocol was published.¹⁷

Study Setting

The study commenced in March 2016 at the Lady Cilento Children’s Hospital in South Brisbane, Australia, a tertiary-level, specialist, pediatric teaching hospital in Queensland, Australia, providing full-spectrum health services to children and young people from birth to 18 years of age. Recruitment, including data collection, was completed in November 2016.

Sample

The target sample size was 110 participants, 50 participants per group plus 10% for potential attrition, as determined by standard pilot trial sample size recommendations.¹⁸ With ethics approval, the sample size was later increased to 150 participants in order to adequately pilot a microbiological substudy method (published separately).¹⁷ Participants were consecutively recruited if they met the inclusion criteria: PICC insertion, age <18 years, predicted hospital stay >24 hours, single-lumen PICC, and written informed consent by an English-speaking, legal parent or guardian. Patients were excluded if they had a current (<48 hours) blood stream infection (BSI), vessel size <2 mm, could not speak English without an interpreter, required a multilumen PICC, or were previously enrolled in the study.

 

Interventions

Participants were randomized to receive either of the following PICCs: (1) standard care: Cook™ polyurethane, turbo-ject, power-injectable PICC (Cook Medical, Bloomington, IN) or (2) comparison: BioFlo^® polyurethane with Endexo^® technology (AngioDynamics Inc, Queensbury, NY).

Outcomes

The primary outcome was feasibility of a full-efficacy trial established by composite analysis of the elements of eligibility (>70% of patients will be eligible), recruitment (>70% of patients will agree to enroll), retention and attrition (<15% of participants are lost to follow-up or withdraw from the study), protocol adherence (>80% of participants receive their allocated, randomly assigned study product), missing data (<10% of data are missed during data collection), parent and healthcare staff satisfaction, and PICC failure effect size estimates to allow sample size calculations.^18,19 PICC failure was defined as the following complications associated with PICC removal: (1) catheter-associated BSI,^8,20-22 (2) local site infection,²² (3) venous thrombosis,²³ (4) occlusion,^24,25 (5) PICC fracture, or (6) PICC dislodgement.^25,26 Parents (or caregivers) and healthcare staff were asked to rate their level of confidence with the study product and ease of PICC removal by using a 0 to 100 numeric rating scale (NRS) of increasing confidence and/or ease. These data were collected at the time of PICC removal. Operators were also asked to rate their levels of satisfaction with the insertion equipment and ease of PICC insertion immediately upon completion of the insertion procedure (both 0-100 NRS of increasing satisfaction and/or ease). Secondary outcomes included individual PICC complications (eg, occlusion) occurring at any time point during the PICC dwell (including at removal), adverse events, pain, redness at the insertion site, and overall PICC dwell.

Study Procedures

The research nurse (ReN) screened operating theater lists for patients, obtained written informed consent, and initiated the randomization. Randomization was computer generated, and web based via Griffith University (https://www151.griffith.edu.au/random) to ensure allocation concealment until study entry. Patients were randomly assigned in a 1:1 ratio with computer-generated and randomly varied block sizes of 2 and 4. Data were collected by the ReN on the day of insertion, at day 1 postinsertion, then every 2 to 3 days thereafter so that PICCs were checked at least twice per week until study completion. Participants were included in the trial until 12 weeks post-PICC insertion, study withdrawal or PICC removal (whichever came first), with an additional 48 hours follow-up for infection outcomes. Patient review was face to face during the inpatient stay, with discharged patients’ follow-up occurring via outpatient clinics, hospital-in-the-home service, or telephone.

Data collection was via Research Electronic Data Capture (http://project-redcap.org/). The ReN collected data on primary and secondary outcomes by using the predefined criteria. Demographic and clinical data were collected to assess the success of randomization, describe the participant group, and display characteristics known to increase the risk of PICC complication and thrombosis. A blinded radiologist and infectious disease specialist reviewed and diagnosed thrombosis of deep veins and catheter-associated BSI outcomes, respectively.

PICC Procedures

Extensive prestudy education for 2 months prior to trial commencement was provided to all clinicians involved with the insertion and care of PICCs, including the study products. PICCs were inserted in an operating theater environment by a qualified consultant pediatric anesthetist, a senior anesthetic registrar or fellow in an approved anesthetic training program, or pediatric vascular access nurse practitioner. Ultrasound guidance was used to assess a patient’s vasculature and puncture the vessel. The operator chose the PICC size on the basis of clinical judgment of vessel size and patient needs and then inserted the allocated PICC.²⁷ Preferred PICC tip location was the cavoatrial junction. All PICC tip positions were confirmed with a chest x-ray before use.

Postinsertion, PICCs were managed by local interdisciplinary clinicians in accordance with local practice guidelines.^27-31 PICC care and management includes the use of 2% chlorhexidine gluconate in 70% alcohol for site antisepsis and neutral displacement needleless connectors (TUTA Pulse; Medical Australia Limited, Lidcombe, New South Wales, Australia); normal saline was used to flush after medication administration, and if the device was not in use for 6 hours or longer, heparin is instilled with securement via bordered polyurethane dressing (Tegaderm 1616; 3M, St Paul, Minnesota) and a sutureless securement device (Statlock VPPCSP; Bard, Georgia).

Statistical Analyses

Data were exported to Stata 15³² for cleaning and analysis. Data cleaning of outlying figures and missing and implausible data was undertaken prior to analysis. Missing data were not imputed. The PICC was the unit of measurement, and all randomly assigned patients were analyzed on an intention-to-treat basis.³³ Descriptive statistics (frequencies and percentages) were used to ascertain the primary outcome of feasibility for the larger trial. Incidence rates (per 1,000 catheter days) and rate ratios, including 95% confidence intervals (CIs), were calculated. The comparability of groups at baseline was described across demographic, clinical, and device characteristics. Kaplan-Meier survival curves (with log-rank tests) were used to compare PICC failure between study groups over time. Associations between baseline characteristics and failure were described by calculating hazard ratios (HRs). Univariable Cox regression was performed only due to the relatively low number of outcomes. P values of <.05 were considered statistically significant.

 

Ethics

The Children’s Health Service District, Queensland (Human Research Ethics Committee/15/QRCH/164), and Griffith University (2016/077) Human Research Ethics Committees provided ethics and governance approval. Informed consent was obtained from parents or legal guardians, with children providing youth assent if they were 7 years or older, dependent upon cognitive ability.

RESULTS

Participant and PICC Characteristics

Participant and PICC characteristics are described in Table 1. The majority of participant and PICC characteristics were balanced between intervention groups. The mean patient age was 7.3 years (standard deviation 5.0; range 0-18). PICC insertion was most commonly for a respiratory diagnosis (n = 98; 65%). Most PICCs were placed in the basilica vein (n = 115; 79%), with insertion being successful on the first attempt (n = 125; 86%). There was some imbalance (>10% absolute difference between groups) in nurse practitioner and registrar insertions (standard care 35% and 23% vs BioFlo^® 51% and 8%, respectively) and patients with leucocytes <1000 µl (standard care 10% vs BioFlo^® 22%). Optimal PICC tip location at the cavoatrial junction was higher with BioFlo^® than standard care, although this difference was <10%.

Feasibility Outcomes

As shown in Figure 1, the majority of feasibility criteria were met, with 94% of 188 screened patients being eligible to participate and 97% of eligible patients consenting to enroll. Of 150 patients randomly assigned, 4 (1 in standard care and 3 in BioFlo^®) were unable to have a PICC inserted or the procedure was cancelled. Demographic data only were collected for these 4 patients. No participants were lost to follow-up, and no primary outcome data were missing. Staff satisfaction with insertion kit and ease of insertion, ease of removal of the PICC, and parental confidence in the PICC product were similar across both groups (Table 2).

PICC Failure and Complications

In total, 24 of 146 participants (16%) experienced PICC failure. There were 16 (22%) failures of standard care PICCs and 8 (11%) failures of BioFlo^® PICCs. This corresponded to incident rates of 12.6 and 7.3 per 1000 catheter days (incident rate ratio 0.58; 95% CI, 0.21-1.43; P = .172; Table 2). Failure was most commonly from thrombosis (n = 5; 7%) or occlusion (n = 5; 7%) in the standard care group, with lower incidences in the BioFlo^® group (n = 2 [3%] and n = 1 [1%], respectively). Figure 2 displays survival from PICC failure.

Considering the entire PICC dwell, of the 74 standard care patients, 49 (66%) had no complications, 9 (12%) had complications during the dwell but none at removal, 2 (3%) had no complications during the dwell but had a complication (ie, failure) at removal, and 14 (19%) had complications during the dwell and at removal. For the 72 BioFlo^® patients, 61 (85%) had no PICC complications, 3 (4%) had complications during the dwell but none at removal, 4 (5.5%) had no complications during the dwell but had a complication (ie, failure) at removal, and 4 (5.5%) had complications during the dwell and at removal.

More than twice as many standard care patients as BioFlo^® patients had a complication during the PICC dwell, and this difference was statistically significant (25 of 74, 34% vs 11 of 72, 15%; P = .009; Table 2). These results are consistent with the Kaplan-Meier curve, which shows longer complication-free survival with BioFlo^® (Figure 2A and 2B). The median BioFlo^® dwell was 1 day longer (13.8 vs 12.9 days), and the median time to first complication was one day later (4.0 BioFlo^® vs 3.0 standard care; Table 2).

As per supplementary Table 1, univariate Cox regression identified PICC failure as significantly associated with tip placement in the proximal superior vena cava (SVC) compared to the SVC–right atrium junction (HR 2.61; 95% CI, 1.17-5.82; P = .024). Reduced risk of PICC failure was significantly associated with any infusion during the dwell (continuous fluid infusion, P = .007; continuous antibiotic, P = .042; or intermittent infusion, P = .046) compared to no infusion. Other variables potentially influencing the risk of failure included PICC insertion by nurse specialist compared to consultant anesthetist (HR 2.61; 95% CI, 0.85-5.44) or registrar (HR 1.97; 95% CI, 0.57-6.77). These differences were not statistically significant; however, baseline imbalance between study groups for this variable and the feasibility design preclude absolute conclusions.

DISCUSSION

This is the first pilot feasibility trial of new PICC materials and valve design incorporated in the BioFlo^® PICC in the pediatric population. The trial incorporated best practice for randomized trials, including using a concurrent control group, centralized and concealed randomization, predetermined feasibility criteria, and a registered and published trial protocol.¹⁷ As in other studies,^15,24,34 PICC failure and complication prevalence was unacceptably high for this essential device. Standard care PICCs failed twice as often as the new BioFlo^® PICCs (22% vs 11%), which is a clinically important difference. As researchers in a pilot study, we did not expect to detect statistically significant differences; however, we found that overall complications during the dwell occurred significantly more with the standard care than BioFlo^® PICCs (P = .009).

 

BioFlo^® PICC material offers a major advancement in PICC material through the incorporation of AT technologies into catheter materials, such as PICCs. Endexo^® is a low molecular–weight, fluoro-oligomeric additive that self-locates to the top few nanometers of the material surface. When added to power-injectable polyurethane, the additive results in a strong but passive, nonstick, fluorinated surface in the base PICC material. This inhibits platelet adhesion, suppresses protein procoagulant conformation, and thereby reduces thrombus formation in medical devices. Additionally, Endexo^® is not a catheter coating; rather, it is incorporated within the polyurethane of the PICC, thereby ensuring these AT properties are present on the internal, external, and cut surfaces of the PICC. If this technology can reduce complication during treatment and reduce failure from infection, thrombosis, occlusion, fracture, and dislodgement, it will improve patient outcomes considerably and lower health system costs. Previous studies investigating valve technology in PICC design to reduce occlusion have been inconclusive.^12-14,35,36 Occlusion (both partial and complete) was less frequent in our study with the BioFlo^® group (n = 3; 4%) compared to the standard care group (n = 6; 8%). The results of this pilot study suggest that either the Endexo^® material or PASV^® technology has a positive association with occlusion reduction during PICC treatment.

Thrombosis was the primary failure type for the standard care PICCs, comprising one-third of failures. All but one patient with radiologically confirmed thrombosis required the removal of the PICC prior to completion of treatment. The decision to remove the PICC or retain and treat conservatively remained with the treating team. Raffini et al.⁷ found thrombosis to increase in patients with one or more coexisting chronic medical condition. Slightly more standard care than BioFlo^® patients were free of such comorbidities (25% vs 16%), yet standard care patients still had the higher number of thromboses (7% vs 3%). Morgenthaler and Rodriguez³⁷ reported vascular access-associated thrombosis in pediatrics to be less common than in adults but higher in medically complex children. Worryingly, Menendez et al.³⁸ reported pediatric thrombosis to be largely asymptomatic, so the true incidence in our study is likely higher because only radiologically confirmed thromboses were recorded.

Occlusion (partial or complete) was the predominant complication across the study, being associated with one-third of all failures. When occlusion complications during the dwell (some of which were resolved with treatment), in addition to those causing failure, were considered, this number was even greater. Occlusion complications are prevalent and costly. Smith et al.²⁴ reported that occlusion was the most common reason for PICC removal and the most likely complication to delay treatment. Both the BioFlo^® and standard care PICCs are pressure rated with good tensile strength; however, fracture occurred in 4% (n = 3) of standard care PICCs compared to no fractures in BioFlo^® PICCs. Although the numbers are small, it may suggest a superior tensile strength of the BioFlo^® material.

This study reinforces previously published results^24,38 that PICC tip position is important and can influence complications, such as occlusion and thrombosis. In addition, we found a significant association with failure when PICCs did not have a continuous infusion. These findings reinforce the need for optimal tip location at insertion and ongoing flushing and maintenance of PICCs not used for infusions.

Limitations of this study include the small sample size, which was not designed to detect statistical differences in the primary outcome between groups. Despite randomization, there were slight imbalances at baseline for inserter type and leukocyte count, although these were not significantly associated with PICC failure in the Cox regression (data not shown), and thus were unlikely to influence findings. Additionally, a difference of <10% was associated with PICC tip position, favoring the BioFlo^® group. PICC tip position outside the cavoatrial junction was positively associated with failure; therefore, the effect of tip positioning on outcomes is difficult to ascertain given the small sample size and feasibility nature of the study. Further study is warranted to further explore this effect. The population sampled was pediatric medical and surgical inpatients with a vessel size >2 mm attending the operating theater suite for PICC insertion, thereby limiting the study’s generalizability to adults and other populations, including neonates and those with PICCs inserted in the pediatric intensive care unit. The study could not be blinded because study products had to be visible to the clinical and research staff. However, it is unlikely that staff would intentionally sabotage PICCs to bias the study. Blinding was possible for the assessment of blood culture and ultrasound reports to diagnose infection and thrombosis. Strengths of this study included 100% protocol adherence, and no patients were lost to follow-up.

 

CONCLUSION

These results confirm that PICC failure is unacceptably high and suggest that the innovative BioFlo^® PICC material and design holds promise to improve PICC outcomes by reducing complications and overall PICC failure. Trials of this technology are feasible, safe, and acceptable to healthcare staff and parents. Further trials are required, including in other patient populations, to definitively identify clinical, cost-effective methods to prevent PICC failure and improve reliability during treatment.

Acknowledgments

The authors thank the children and parents of Lady Cilento Children’s Hospital for participating in this important research. A special thank you goes to the nurses within the Vascular Assessment and Management Service and to Karen Turner, Julieta Woosley, and Anna Dean for their efforts in data collecting and ensuring protocol adherence.

Disclosure

Griffith University has received unrestricted, investigator-initiated research or educational grants to support the research of T. K., A. J. U., and C. R. M. from product manufacturers 3M, Adhezion Inc, AngioDynamics, Bard Medical, Baxter, B. Braun Medical Inc, Becton Dickinson, CareFusion, Centurion Medical Products, Cook Medical, Entrotech, FloMedical, ICU Medical Inc, Medical Australia Limited, Medtronic, Smiths Medical, and Teleflex. Griffith University has received consultancy payments on behalf of C. R. M., A. J. U., and T. K. from manufacturers 3M, AngioDynamics, Bard Medical, B. Braun Medical Inc, Becton Dickinson, CareFusion, Mayo Healthcare Inc, ResQDevices, and Smiths Medical. AngioDynamics (the BioFlo^® PICC manufacturer) provided partial funds to undertake this research via an unrestricted donation to Griffith University (but not the study authors). Queensland Health provided in-kind support to fund the remainder of the trial. The funders had no role in the study design, collection, analysis, or interpretation of the data, writing of the report, or decision to submit the article for publication.

Peripherally inserted central catheters (PICCs) have evolved since their inception in the early 1970s and are used with increasing frequency for pediatric inpatients and outpatients.^1-3 Emerging literature, including a meta-analysis of international observational studies,⁴ reports PICC failure (complications necessitating premature removal) occurs in up to 30% of PICCs, most commonly due to infection, thrombosis, occlusion, and fracture.^4-7 Raffini et al.⁷ report the increasing incidence of pediatric PICC-related thrombosis increases morbidity and mortality⁸ and negatively impacts future vessel health and preservation.⁹

PICCs have progressed from relatively simple, silicone-based catheters with an external clamp to chemically engineered polyurethane with pressure-activated valves placed at the proximal or distal catheter hub with the intent to reduce occlusion.¹⁰ Further modernization of PICC material occurred with the incorporation of antithrombogenic (AT) material (Endexo^®). These PICCs are designed to contain a nonstick polymer, which is designed to reduce the adherence of blood components (platelets and clotting factors) and inhibit thrombus formation (and hence prevent deep vein thrombosis andocclusion, as well as inhibit microbial biofilm attachment [and subsequent infection]).¹¹

In addition to new materials, other aspects of this PICC design have been the addition of a pressure-activated safety valve (PASV^®) built into the proximal hub. Pressure-activated valve technology promises to prevent catheter occlusion by reducing blood reflux into the PICC; the valve opens with pressure during infusion and aspiration and remains closed with normal venous pressure, circumventing the need for clinicians to manually clamp the PICC and reducing human error and the potential for thrombosis, occlusion, and fracture development.¹² Hoffer et al.¹³ reported half as many occlusions of valved PICCs (3.3%) compared with nonvalved or clamped PICCs (7.1%); although not statistically significant (P = .10), perhaps due to the small sample, overall complications, including occlusion and infection, were significantly lessened with the valved PICC (35% vs 79%; P = .02). Comparatively, Pittiruti et al.¹⁴ conducted a trial of 2 types of valved PICCs with an open-ended, nonvalved PICC and found no reduction in PICC occlusion or catheter malfunction.

Today, PICC use is common for patients who require short-to-medium intravenous therapy. PICCs are increasingly recognized for their significant complications, including thrombosis and infection.¹⁵ Novel PICC technology, including the incorporation of AT material such as Endexo^® and PASV^®, may reduce complications; however, the clinical efficacy, cost-effectiveness, and acceptability of these innovations have not been tested through randomized trials in pediatric patients. In accordance with Medical Research Council guidelines¹⁶ for developing interventions, we pilot tested the feasibility of the BioFlo^® PICC, including intervention acceptability, compliance, recruitment, and initial estimates of effect, in anticipation of a subsequent full-scale efficacy randomized controlled trial. Our secondary aim was to compare the effectiveness of the BioFlo^® PICC with Endexo^® and PASV^® technology in reducing PICC complications and failure.

METHODS

Design

We undertook a pilot randomized controlled trial comparing the standard polyurethane PICC (with external clamp) with the BioFlo^® PICC (with internal valve) in preventing catheter failure in pediatric patients. The study was prospectively registered with the Australian Clinical Trials Registry (ACTRN12615001290583), and the research protocol was published.¹⁷

Study Setting

The study commenced in March 2016 at the Lady Cilento Children’s Hospital in South Brisbane, Australia, a tertiary-level, specialist, pediatric teaching hospital in Queensland, Australia, providing full-spectrum health services to children and young people from birth to 18 years of age. Recruitment, including data collection, was completed in November 2016.

Sample

The target sample size was 110 participants, 50 participants per group plus 10% for potential attrition, as determined by standard pilot trial sample size recommendations.¹⁸ With ethics approval, the sample size was later increased to 150 participants in order to adequately pilot a microbiological substudy method (published separately).¹⁷ Participants were consecutively recruited if they met the inclusion criteria: PICC insertion, age <18 years, predicted hospital stay >24 hours, single-lumen PICC, and written informed consent by an English-speaking, legal parent or guardian. Patients were excluded if they had a current (<48 hours) blood stream infection (BSI), vessel size <2 mm, could not speak English without an interpreter, required a multilumen PICC, or were previously enrolled in the study.

 

Interventions

Participants were randomized to receive either of the following PICCs: (1) standard care: Cook™ polyurethane, turbo-ject, power-injectable PICC (Cook Medical, Bloomington, IN) or (2) comparison: BioFlo^® polyurethane with Endexo^® technology (AngioDynamics Inc, Queensbury, NY).

Outcomes

The primary outcome was feasibility of a full-efficacy trial established by composite analysis of the elements of eligibility (>70% of patients will be eligible), recruitment (>70% of patients will agree to enroll), retention and attrition (<15% of participants are lost to follow-up or withdraw from the study), protocol adherence (>80% of participants receive their allocated, randomly assigned study product), missing data (<10% of data are missed during data collection), parent and healthcare staff satisfaction, and PICC failure effect size estimates to allow sample size calculations.^18,19 PICC failure was defined as the following complications associated with PICC removal: (1) catheter-associated BSI,^8,20-22 (2) local site infection,²² (3) venous thrombosis,²³ (4) occlusion,^24,25 (5) PICC fracture, or (6) PICC dislodgement.^25,26 Parents (or caregivers) and healthcare staff were asked to rate their level of confidence with the study product and ease of PICC removal by using a 0 to 100 numeric rating scale (NRS) of increasing confidence and/or ease. These data were collected at the time of PICC removal. Operators were also asked to rate their levels of satisfaction with the insertion equipment and ease of PICC insertion immediately upon completion of the insertion procedure (both 0-100 NRS of increasing satisfaction and/or ease). Secondary outcomes included individual PICC complications (eg, occlusion) occurring at any time point during the PICC dwell (including at removal), adverse events, pain, redness at the insertion site, and overall PICC dwell.

Study Procedures

The research nurse (ReN) screened operating theater lists for patients, obtained written informed consent, and initiated the randomization. Randomization was computer generated, and web based via Griffith University (https://www151.griffith.edu.au/random) to ensure allocation concealment until study entry. Patients were randomly assigned in a 1:1 ratio with computer-generated and randomly varied block sizes of 2 and 4. Data were collected by the ReN on the day of insertion, at day 1 postinsertion, then every 2 to 3 days thereafter so that PICCs were checked at least twice per week until study completion. Participants were included in the trial until 12 weeks post-PICC insertion, study withdrawal or PICC removal (whichever came first), with an additional 48 hours follow-up for infection outcomes. Patient review was face to face during the inpatient stay, with discharged patients’ follow-up occurring via outpatient clinics, hospital-in-the-home service, or telephone.

Data collection was via Research Electronic Data Capture (http://project-redcap.org/). The ReN collected data on primary and secondary outcomes by using the predefined criteria. Demographic and clinical data were collected to assess the success of randomization, describe the participant group, and display characteristics known to increase the risk of PICC complication and thrombosis. A blinded radiologist and infectious disease specialist reviewed and diagnosed thrombosis of deep veins and catheter-associated BSI outcomes, respectively.

PICC Procedures

Extensive prestudy education for 2 months prior to trial commencement was provided to all clinicians involved with the insertion and care of PICCs, including the study products. PICCs were inserted in an operating theater environment by a qualified consultant pediatric anesthetist, a senior anesthetic registrar or fellow in an approved anesthetic training program, or pediatric vascular access nurse practitioner. Ultrasound guidance was used to assess a patient’s vasculature and puncture the vessel. The operator chose the PICC size on the basis of clinical judgment of vessel size and patient needs and then inserted the allocated PICC.²⁷ Preferred PICC tip location was the cavoatrial junction. All PICC tip positions were confirmed with a chest x-ray before use.

Postinsertion, PICCs were managed by local interdisciplinary clinicians in accordance with local practice guidelines.^27-31 PICC care and management includes the use of 2% chlorhexidine gluconate in 70% alcohol for site antisepsis and neutral displacement needleless connectors (TUTA Pulse; Medical Australia Limited, Lidcombe, New South Wales, Australia); normal saline was used to flush after medication administration, and if the device was not in use for 6 hours or longer, heparin is instilled with securement via bordered polyurethane dressing (Tegaderm 1616; 3M, St Paul, Minnesota) and a sutureless securement device (Statlock VPPCSP; Bard, Georgia).

Statistical Analyses

Data were exported to Stata 15³² for cleaning and analysis. Data cleaning of outlying figures and missing and implausible data was undertaken prior to analysis. Missing data were not imputed. The PICC was the unit of measurement, and all randomly assigned patients were analyzed on an intention-to-treat basis.³³ Descriptive statistics (frequencies and percentages) were used to ascertain the primary outcome of feasibility for the larger trial. Incidence rates (per 1,000 catheter days) and rate ratios, including 95% confidence intervals (CIs), were calculated. The comparability of groups at baseline was described across demographic, clinical, and device characteristics. Kaplan-Meier survival curves (with log-rank tests) were used to compare PICC failure between study groups over time. Associations between baseline characteristics and failure were described by calculating hazard ratios (HRs). Univariable Cox regression was performed only due to the relatively low number of outcomes. P values of <.05 were considered statistically significant.

 

Ethics

The Children’s Health Service District, Queensland (Human Research Ethics Committee/15/QRCH/164), and Griffith University (2016/077) Human Research Ethics Committees provided ethics and governance approval. Informed consent was obtained from parents or legal guardians, with children providing youth assent if they were 7 years or older, dependent upon cognitive ability.

RESULTS

Participant and PICC Characteristics

Participant and PICC characteristics are described in Table 1. The majority of participant and PICC characteristics were balanced between intervention groups. The mean patient age was 7.3 years (standard deviation 5.0; range 0-18). PICC insertion was most commonly for a respiratory diagnosis (n = 98; 65%). Most PICCs were placed in the basilica vein (n = 115; 79%), with insertion being successful on the first attempt (n = 125; 86%). There was some imbalance (>10% absolute difference between groups) in nurse practitioner and registrar insertions (standard care 35% and 23% vs BioFlo^® 51% and 8%, respectively) and patients with leucocytes <1000 µl (standard care 10% vs BioFlo^® 22%). Optimal PICC tip location at the cavoatrial junction was higher with BioFlo^® than standard care, although this difference was <10%.

Feasibility Outcomes

As shown in Figure 1, the majority of feasibility criteria were met, with 94% of 188 screened patients being eligible to participate and 97% of eligible patients consenting to enroll. Of 150 patients randomly assigned, 4 (1 in standard care and 3 in BioFlo^®) were unable to have a PICC inserted or the procedure was cancelled. Demographic data only were collected for these 4 patients. No participants were lost to follow-up, and no primary outcome data were missing. Staff satisfaction with insertion kit and ease of insertion, ease of removal of the PICC, and parental confidence in the PICC product were similar across both groups (Table 2).

PICC Failure and Complications

In total, 24 of 146 participants (16%) experienced PICC failure. There were 16 (22%) failures of standard care PICCs and 8 (11%) failures of BioFlo^® PICCs. This corresponded to incident rates of 12.6 and 7.3 per 1000 catheter days (incident rate ratio 0.58; 95% CI, 0.21-1.43; P = .172; Table 2). Failure was most commonly from thrombosis (n = 5; 7%) or occlusion (n = 5; 7%) in the standard care group, with lower incidences in the BioFlo^® group (n = 2 [3%] and n = 1 [1%], respectively). Figure 2 displays survival from PICC failure.

Considering the entire PICC dwell, of the 74 standard care patients, 49 (66%) had no complications, 9 (12%) had complications during the dwell but none at removal, 2 (3%) had no complications during the dwell but had a complication (ie, failure) at removal, and 14 (19%) had complications during the dwell and at removal. For the 72 BioFlo^® patients, 61 (85%) had no PICC complications, 3 (4%) had complications during the dwell but none at removal, 4 (5.5%) had no complications during the dwell but had a complication (ie, failure) at removal, and 4 (5.5%) had complications during the dwell and at removal.

More than twice as many standard care patients as BioFlo^® patients had a complication during the PICC dwell, and this difference was statistically significant (25 of 74, 34% vs 11 of 72, 15%; P = .009; Table 2). These results are consistent with the Kaplan-Meier curve, which shows longer complication-free survival with BioFlo^® (Figure 2A and 2B). The median BioFlo^® dwell was 1 day longer (13.8 vs 12.9 days), and the median time to first complication was one day later (4.0 BioFlo^® vs 3.0 standard care; Table 2).

As per supplementary Table 1, univariate Cox regression identified PICC failure as significantly associated with tip placement in the proximal superior vena cava (SVC) compared to the SVC–right atrium junction (HR 2.61; 95% CI, 1.17-5.82; P = .024). Reduced risk of PICC failure was significantly associated with any infusion during the dwell (continuous fluid infusion, P = .007; continuous antibiotic, P = .042; or intermittent infusion, P = .046) compared to no infusion. Other variables potentially influencing the risk of failure included PICC insertion by nurse specialist compared to consultant anesthetist (HR 2.61; 95% CI, 0.85-5.44) or registrar (HR 1.97; 95% CI, 0.57-6.77). These differences were not statistically significant; however, baseline imbalance between study groups for this variable and the feasibility design preclude absolute conclusions.

DISCUSSION

This is the first pilot feasibility trial of new PICC materials and valve design incorporated in the BioFlo^® PICC in the pediatric population. The trial incorporated best practice for randomized trials, including using a concurrent control group, centralized and concealed randomization, predetermined feasibility criteria, and a registered and published trial protocol.¹⁷ As in other studies,^15,24,34 PICC failure and complication prevalence was unacceptably high for this essential device. Standard care PICCs failed twice as often as the new BioFlo^® PICCs (22% vs 11%), which is a clinically important difference. As researchers in a pilot study, we did not expect to detect statistically significant differences; however, we found that overall complications during the dwell occurred significantly more with the standard care than BioFlo^® PICCs (P = .009).

 

BioFlo^® PICC material offers a major advancement in PICC material through the incorporation of AT technologies into catheter materials, such as PICCs. Endexo^® is a low molecular–weight, fluoro-oligomeric additive that self-locates to the top few nanometers of the material surface. When added to power-injectable polyurethane, the additive results in a strong but passive, nonstick, fluorinated surface in the base PICC material. This inhibits platelet adhesion, suppresses protein procoagulant conformation, and thereby reduces thrombus formation in medical devices. Additionally, Endexo^® is not a catheter coating; rather, it is incorporated within the polyurethane of the PICC, thereby ensuring these AT properties are present on the internal, external, and cut surfaces of the PICC. If this technology can reduce complication during treatment and reduce failure from infection, thrombosis, occlusion, fracture, and dislodgement, it will improve patient outcomes considerably and lower health system costs. Previous studies investigating valve technology in PICC design to reduce occlusion have been inconclusive.^12-14,35,36 Occlusion (both partial and complete) was less frequent in our study with the BioFlo^® group (n = 3; 4%) compared to the standard care group (n = 6; 8%). The results of this pilot study suggest that either the Endexo^® material or PASV^® technology has a positive association with occlusion reduction during PICC treatment.

Thrombosis was the primary failure type for the standard care PICCs, comprising one-third of failures. All but one patient with radiologically confirmed thrombosis required the removal of the PICC prior to completion of treatment. The decision to remove the PICC or retain and treat conservatively remained with the treating team. Raffini et al.⁷ found thrombosis to increase in patients with one or more coexisting chronic medical condition. Slightly more standard care than BioFlo^® patients were free of such comorbidities (25% vs 16%), yet standard care patients still had the higher number of thromboses (7% vs 3%). Morgenthaler and Rodriguez³⁷ reported vascular access-associated thrombosis in pediatrics to be less common than in adults but higher in medically complex children. Worryingly, Menendez et al.³⁸ reported pediatric thrombosis to be largely asymptomatic, so the true incidence in our study is likely higher because only radiologically confirmed thromboses were recorded.

Occlusion (partial or complete) was the predominant complication across the study, being associated with one-third of all failures. When occlusion complications during the dwell (some of which were resolved with treatment), in addition to those causing failure, were considered, this number was even greater. Occlusion complications are prevalent and costly. Smith et al.²⁴ reported that occlusion was the most common reason for PICC removal and the most likely complication to delay treatment. Both the BioFlo^® and standard care PICCs are pressure rated with good tensile strength; however, fracture occurred in 4% (n = 3) of standard care PICCs compared to no fractures in BioFlo^® PICCs. Although the numbers are small, it may suggest a superior tensile strength of the BioFlo^® material.

This study reinforces previously published results^24,38 that PICC tip position is important and can influence complications, such as occlusion and thrombosis. In addition, we found a significant association with failure when PICCs did not have a continuous infusion. These findings reinforce the need for optimal tip location at insertion and ongoing flushing and maintenance of PICCs not used for infusions.

Limitations of this study include the small sample size, which was not designed to detect statistical differences in the primary outcome between groups. Despite randomization, there were slight imbalances at baseline for inserter type and leukocyte count, although these were not significantly associated with PICC failure in the Cox regression (data not shown), and thus were unlikely to influence findings. Additionally, a difference of <10% was associated with PICC tip position, favoring the BioFlo^® group. PICC tip position outside the cavoatrial junction was positively associated with failure; therefore, the effect of tip positioning on outcomes is difficult to ascertain given the small sample size and feasibility nature of the study. Further study is warranted to further explore this effect. The population sampled was pediatric medical and surgical inpatients with a vessel size >2 mm attending the operating theater suite for PICC insertion, thereby limiting the study’s generalizability to adults and other populations, including neonates and those with PICCs inserted in the pediatric intensive care unit. The study could not be blinded because study products had to be visible to the clinical and research staff. However, it is unlikely that staff would intentionally sabotage PICCs to bias the study. Blinding was possible for the assessment of blood culture and ultrasound reports to diagnose infection and thrombosis. Strengths of this study included 100% protocol adherence, and no patients were lost to follow-up.

 

CONCLUSION

These results confirm that PICC failure is unacceptably high and suggest that the innovative BioFlo^® PICC material and design holds promise to improve PICC outcomes by reducing complications and overall PICC failure. Trials of this technology are feasible, safe, and acceptable to healthcare staff and parents. Further trials are required, including in other patient populations, to definitively identify clinical, cost-effective methods to prevent PICC failure and improve reliability during treatment.

Acknowledgments

The authors thank the children and parents of Lady Cilento Children’s Hospital for participating in this important research. A special thank you goes to the nurses within the Vascular Assessment and Management Service and to Karen Turner, Julieta Woosley, and Anna Dean for their efforts in data collecting and ensuring protocol adherence.

Disclosure

Griffith University has received unrestricted, investigator-initiated research or educational grants to support the research of T. K., A. J. U., and C. R. M. from product manufacturers 3M, Adhezion Inc, AngioDynamics, Bard Medical, Baxter, B. Braun Medical Inc, Becton Dickinson, CareFusion, Centurion Medical Products, Cook Medical, Entrotech, FloMedical, ICU Medical Inc, Medical Australia Limited, Medtronic, Smiths Medical, and Teleflex. Griffith University has received consultancy payments on behalf of C. R. M., A. J. U., and T. K. from manufacturers 3M, AngioDynamics, Bard Medical, B. Braun Medical Inc, Becton Dickinson, CareFusion, Mayo Healthcare Inc, ResQDevices, and Smiths Medical. AngioDynamics (the BioFlo^® PICC manufacturer) provided partial funds to undertake this research via an unrestricted donation to Griffith University (but not the study authors). Queensland Health provided in-kind support to fund the remainder of the trial. The funders had no role in the study design, collection, analysis, or interpretation of the data, writing of the report, or decision to submit the article for publication.

Peripherally inserted central catheters (PICCs) have evolved since their inception in the early 1970s and are used with increasing frequency for pediatric inpatients and outpatients.^1-3 Emerging literature, including a meta-analysis of international observational studies,⁴ reports PICC failure (complications necessitating premature removal) occurs in up to 30% of PICCs, most commonly due to infection, thrombosis, occlusion, and fracture.^4-7 Raffini et al.⁷ report the increasing incidence of pediatric PICC-related thrombosis increases morbidity and mortality⁸ and negatively impacts future vessel health and preservation.⁹

PICCs have progressed from relatively simple, silicone-based catheters with an external clamp to chemically engineered polyurethane with pressure-activated valves placed at the proximal or distal catheter hub with the intent to reduce occlusion.¹⁰ Further modernization of PICC material occurred with the incorporation of antithrombogenic (AT) material (Endexo^®). These PICCs are designed to contain a nonstick polymer, which is designed to reduce the adherence of blood components (platelets and clotting factors) and inhibit thrombus formation (and hence prevent deep vein thrombosis andocclusion, as well as inhibit microbial biofilm attachment [and subsequent infection]).¹¹

In addition to new materials, other aspects of this PICC design have been the addition of a pressure-activated safety valve (PASV^®) built into the proximal hub. Pressure-activated valve technology promises to prevent catheter occlusion by reducing blood reflux into the PICC; the valve opens with pressure during infusion and aspiration and remains closed with normal venous pressure, circumventing the need for clinicians to manually clamp the PICC and reducing human error and the potential for thrombosis, occlusion, and fracture development.¹² Hoffer et al.¹³ reported half as many occlusions of valved PICCs (3.3%) compared with nonvalved or clamped PICCs (7.1%); although not statistically significant (P = .10), perhaps due to the small sample, overall complications, including occlusion and infection, were significantly lessened with the valved PICC (35% vs 79%; P = .02). Comparatively, Pittiruti et al.¹⁴ conducted a trial of 2 types of valved PICCs with an open-ended, nonvalved PICC and found no reduction in PICC occlusion or catheter malfunction.

Today, PICC use is common for patients who require short-to-medium intravenous therapy. PICCs are increasingly recognized for their significant complications, including thrombosis and infection.¹⁵ Novel PICC technology, including the incorporation of AT material such as Endexo^® and PASV^®, may reduce complications; however, the clinical efficacy, cost-effectiveness, and acceptability of these innovations have not been tested through randomized trials in pediatric patients. In accordance with Medical Research Council guidelines¹⁶ for developing interventions, we pilot tested the feasibility of the BioFlo^® PICC, including intervention acceptability, compliance, recruitment, and initial estimates of effect, in anticipation of a subsequent full-scale efficacy randomized controlled trial. Our secondary aim was to compare the effectiveness of the BioFlo^® PICC with Endexo^® and PASV^® technology in reducing PICC complications and failure.

METHODS

Design

We undertook a pilot randomized controlled trial comparing the standard polyurethane PICC (with external clamp) with the BioFlo^® PICC (with internal valve) in preventing catheter failure in pediatric patients. The study was prospectively registered with the Australian Clinical Trials Registry (ACTRN12615001290583), and the research protocol was published.¹⁷

Study Setting

The study commenced in March 2016 at the Lady Cilento Children’s Hospital in South Brisbane, Australia, a tertiary-level, specialist, pediatric teaching hospital in Queensland, Australia, providing full-spectrum health services to children and young people from birth to 18 years of age. Recruitment, including data collection, was completed in November 2016.

Sample

The target sample size was 110 participants, 50 participants per group plus 10% for potential attrition, as determined by standard pilot trial sample size recommendations.¹⁸ With ethics approval, the sample size was later increased to 150 participants in order to adequately pilot a microbiological substudy method (published separately).¹⁷ Participants were consecutively recruited if they met the inclusion criteria: PICC insertion, age <18 years, predicted hospital stay >24 hours, single-lumen PICC, and written informed consent by an English-speaking, legal parent or guardian. Patients were excluded if they had a current (<48 hours) blood stream infection (BSI), vessel size <2 mm, could not speak English without an interpreter, required a multilumen PICC, or were previously enrolled in the study.

 

Interventions

Participants were randomized to receive either of the following PICCs: (1) standard care: Cook™ polyurethane, turbo-ject, power-injectable PICC (Cook Medical, Bloomington, IN) or (2) comparison: BioFlo^® polyurethane with Endexo^® technology (AngioDynamics Inc, Queensbury, NY).

Outcomes

The primary outcome was feasibility of a full-efficacy trial established by composite analysis of the elements of eligibility (>70% of patients will be eligible), recruitment (>70% of patients will agree to enroll), retention and attrition (<15% of participants are lost to follow-up or withdraw from the study), protocol adherence (>80% of participants receive their allocated, randomly assigned study product), missing data (<10% of data are missed during data collection), parent and healthcare staff satisfaction, and PICC failure effect size estimates to allow sample size calculations.^18,19 PICC failure was defined as the following complications associated with PICC removal: (1) catheter-associated BSI,^8,20-22 (2) local site infection,²² (3) venous thrombosis,²³ (4) occlusion,^24,25 (5) PICC fracture, or (6) PICC dislodgement.^25,26 Parents (or caregivers) and healthcare staff were asked to rate their level of confidence with the study product and ease of PICC removal by using a 0 to 100 numeric rating scale (NRS) of increasing confidence and/or ease. These data were collected at the time of PICC removal. Operators were also asked to rate their levels of satisfaction with the insertion equipment and ease of PICC insertion immediately upon completion of the insertion procedure (both 0-100 NRS of increasing satisfaction and/or ease). Secondary outcomes included individual PICC complications (eg, occlusion) occurring at any time point during the PICC dwell (including at removal), adverse events, pain, redness at the insertion site, and overall PICC dwell.

Study Procedures

The research nurse (ReN) screened operating theater lists for patients, obtained written informed consent, and initiated the randomization. Randomization was computer generated, and web based via Griffith University (https://www151.griffith.edu.au/random) to ensure allocation concealment until study entry. Patients were randomly assigned in a 1:1 ratio with computer-generated and randomly varied block sizes of 2 and 4. Data were collected by the ReN on the day of insertion, at day 1 postinsertion, then every 2 to 3 days thereafter so that PICCs were checked at least twice per week until study completion. Participants were included in the trial until 12 weeks post-PICC insertion, study withdrawal or PICC removal (whichever came first), with an additional 48 hours follow-up for infection outcomes. Patient review was face to face during the inpatient stay, with discharged patients’ follow-up occurring via outpatient clinics, hospital-in-the-home service, or telephone.

Data collection was via Research Electronic Data Capture (http://project-redcap.org/). The ReN collected data on primary and secondary outcomes by using the predefined criteria. Demographic and clinical data were collected to assess the success of randomization, describe the participant group, and display characteristics known to increase the risk of PICC complication and thrombosis. A blinded radiologist and infectious disease specialist reviewed and diagnosed thrombosis of deep veins and catheter-associated BSI outcomes, respectively.

PICC Procedures

Extensive prestudy education for 2 months prior to trial commencement was provided to all clinicians involved with the insertion and care of PICCs, including the study products. PICCs were inserted in an operating theater environment by a qualified consultant pediatric anesthetist, a senior anesthetic registrar or fellow in an approved anesthetic training program, or pediatric vascular access nurse practitioner. Ultrasound guidance was used to assess a patient’s vasculature and puncture the vessel. The operator chose the PICC size on the basis of clinical judgment of vessel size and patient needs and then inserted the allocated PICC.²⁷ Preferred PICC tip location was the cavoatrial junction. All PICC tip positions were confirmed with a chest x-ray before use.

Postinsertion, PICCs were managed by local interdisciplinary clinicians in accordance with local practice guidelines.^27-31 PICC care and management includes the use of 2% chlorhexidine gluconate in 70% alcohol for site antisepsis and neutral displacement needleless connectors (TUTA Pulse; Medical Australia Limited, Lidcombe, New South Wales, Australia); normal saline was used to flush after medication administration, and if the device was not in use for 6 hours or longer, heparin is instilled with securement via bordered polyurethane dressing (Tegaderm 1616; 3M, St Paul, Minnesota) and a sutureless securement device (Statlock VPPCSP; Bard, Georgia).

Statistical Analyses

Data were exported to Stata 15³² for cleaning and analysis. Data cleaning of outlying figures and missing and implausible data was undertaken prior to analysis. Missing data were not imputed. The PICC was the unit of measurement, and all randomly assigned patients were analyzed on an intention-to-treat basis.³³ Descriptive statistics (frequencies and percentages) were used to ascertain the primary outcome of feasibility for the larger trial. Incidence rates (per 1,000 catheter days) and rate ratios, including 95% confidence intervals (CIs), were calculated. The comparability of groups at baseline was described across demographic, clinical, and device characteristics. Kaplan-Meier survival curves (with log-rank tests) were used to compare PICC failure between study groups over time. Associations between baseline characteristics and failure were described by calculating hazard ratios (HRs). Univariable Cox regression was performed only due to the relatively low number of outcomes. P values of <.05 were considered statistically significant.

 

Ethics

The Children’s Health Service District, Queensland (Human Research Ethics Committee/15/QRCH/164), and Griffith University (2016/077) Human Research Ethics Committees provided ethics and governance approval. Informed consent was obtained from parents or legal guardians, with children providing youth assent if they were 7 years or older, dependent upon cognitive ability.

RESULTS

Participant and PICC Characteristics

Participant and PICC characteristics are described in Table 1. The majority of participant and PICC characteristics were balanced between intervention groups. The mean patient age was 7.3 years (standard deviation 5.0; range 0-18). PICC insertion was most commonly for a respiratory diagnosis (n = 98; 65%). Most PICCs were placed in the basilica vein (n = 115; 79%), with insertion being successful on the first attempt (n = 125; 86%). There was some imbalance (>10% absolute difference between groups) in nurse practitioner and registrar insertions (standard care 35% and 23% vs BioFlo^® 51% and 8%, respectively) and patients with leucocytes <1000 µl (standard care 10% vs BioFlo^® 22%). Optimal PICC tip location at the cavoatrial junction was higher with BioFlo^® than standard care, although this difference was <10%.

Feasibility Outcomes

As shown in Figure 1, the majority of feasibility criteria were met, with 94% of 188 screened patients being eligible to participate and 97% of eligible patients consenting to enroll. Of 150 patients randomly assigned, 4 (1 in standard care and 3 in BioFlo^®) were unable to have a PICC inserted or the procedure was cancelled. Demographic data only were collected for these 4 patients. No participants were lost to follow-up, and no primary outcome data were missing. Staff satisfaction with insertion kit and ease of insertion, ease of removal of the PICC, and parental confidence in the PICC product were similar across both groups (Table 2).

PICC Failure and Complications

In total, 24 of 146 participants (16%) experienced PICC failure. There were 16 (22%) failures of standard care PICCs and 8 (11%) failures of BioFlo^® PICCs. This corresponded to incident rates of 12.6 and 7.3 per 1000 catheter days (incident rate ratio 0.58; 95% CI, 0.21-1.43; P = .172; Table 2). Failure was most commonly from thrombosis (n = 5; 7%) or occlusion (n = 5; 7%) in the standard care group, with lower incidences in the BioFlo^® group (n = 2 [3%] and n = 1 [1%], respectively). Figure 2 displays survival from PICC failure.

Considering the entire PICC dwell, of the 74 standard care patients, 49 (66%) had no complications, 9 (12%) had complications during the dwell but none at removal, 2 (3%) had no complications during the dwell but had a complication (ie, failure) at removal, and 14 (19%) had complications during the dwell and at removal. For the 72 BioFlo^® patients, 61 (85%) had no PICC complications, 3 (4%) had complications during the dwell but none at removal, 4 (5.5%) had no complications during the dwell but had a complication (ie, failure) at removal, and 4 (5.5%) had complications during the dwell and at removal.

More than twice as many standard care patients as BioFlo^® patients had a complication during the PICC dwell, and this difference was statistically significant (25 of 74, 34% vs 11 of 72, 15%; P = .009; Table 2). These results are consistent with the Kaplan-Meier curve, which shows longer complication-free survival with BioFlo^® (Figure 2A and 2B). The median BioFlo^® dwell was 1 day longer (13.8 vs 12.9 days), and the median time to first complication was one day later (4.0 BioFlo^® vs 3.0 standard care; Table 2).

As per supplementary Table 1, univariate Cox regression identified PICC failure as significantly associated with tip placement in the proximal superior vena cava (SVC) compared to the SVC–right atrium junction (HR 2.61; 95% CI, 1.17-5.82; P = .024). Reduced risk of PICC failure was significantly associated with any infusion during the dwell (continuous fluid infusion, P = .007; continuous antibiotic, P = .042; or intermittent infusion, P = .046) compared to no infusion. Other variables potentially influencing the risk of failure included PICC insertion by nurse specialist compared to consultant anesthetist (HR 2.61; 95% CI, 0.85-5.44) or registrar (HR 1.97; 95% CI, 0.57-6.77). These differences were not statistically significant; however, baseline imbalance between study groups for this variable and the feasibility design preclude absolute conclusions.

DISCUSSION

This is the first pilot feasibility trial of new PICC materials and valve design incorporated in the BioFlo^® PICC in the pediatric population. The trial incorporated best practice for randomized trials, including using a concurrent control group, centralized and concealed randomization, predetermined feasibility criteria, and a registered and published trial protocol.¹⁷ As in other studies,^15,24,34 PICC failure and complication prevalence was unacceptably high for this essential device. Standard care PICCs failed twice as often as the new BioFlo^® PICCs (22% vs 11%), which is a clinically important difference. As researchers in a pilot study, we did not expect to detect statistically significant differences; however, we found that overall complications during the dwell occurred significantly more with the standard care than BioFlo^® PICCs (P = .009).

 

BioFlo^® PICC material offers a major advancement in PICC material through the incorporation of AT technologies into catheter materials, such as PICCs. Endexo^® is a low molecular–weight, fluoro-oligomeric additive that self-locates to the top few nanometers of the material surface. When added to power-injectable polyurethane, the additive results in a strong but passive, nonstick, fluorinated surface in the base PICC material. This inhibits platelet adhesion, suppresses protein procoagulant conformation, and thereby reduces thrombus formation in medical devices. Additionally, Endexo^® is not a catheter coating; rather, it is incorporated within the polyurethane of the PICC, thereby ensuring these AT properties are present on the internal, external, and cut surfaces of the PICC. If this technology can reduce complication during treatment and reduce failure from infection, thrombosis, occlusion, fracture, and dislodgement, it will improve patient outcomes considerably and lower health system costs. Previous studies investigating valve technology in PICC design to reduce occlusion have been inconclusive.^12-14,35,36 Occlusion (both partial and complete) was less frequent in our study with the BioFlo^® group (n = 3; 4%) compared to the standard care group (n = 6; 8%). The results of this pilot study suggest that either the Endexo^® material or PASV^® technology has a positive association with occlusion reduction during PICC treatment.

Thrombosis was the primary failure type for the standard care PICCs, comprising one-third of failures. All but one patient with radiologically confirmed thrombosis required the removal of the PICC prior to completion of treatment. The decision to remove the PICC or retain and treat conservatively remained with the treating team. Raffini et al.⁷ found thrombosis to increase in patients with one or more coexisting chronic medical condition. Slightly more standard care than BioFlo^® patients were free of such comorbidities (25% vs 16%), yet standard care patients still had the higher number of thromboses (7% vs 3%). Morgenthaler and Rodriguez³⁷ reported vascular access-associated thrombosis in pediatrics to be less common than in adults but higher in medically complex children. Worryingly, Menendez et al.³⁸ reported pediatric thrombosis to be largely asymptomatic, so the true incidence in our study is likely higher because only radiologically confirmed thromboses were recorded.

Occlusion (partial or complete) was the predominant complication across the study, being associated with one-third of all failures. When occlusion complications during the dwell (some of which were resolved with treatment), in addition to those causing failure, were considered, this number was even greater. Occlusion complications are prevalent and costly. Smith et al.²⁴ reported that occlusion was the most common reason for PICC removal and the most likely complication to delay treatment. Both the BioFlo^® and standard care PICCs are pressure rated with good tensile strength; however, fracture occurred in 4% (n = 3) of standard care PICCs compared to no fractures in BioFlo^® PICCs. Although the numbers are small, it may suggest a superior tensile strength of the BioFlo^® material.

This study reinforces previously published results^24,38 that PICC tip position is important and can influence complications, such as occlusion and thrombosis. In addition, we found a significant association with failure when PICCs did not have a continuous infusion. These findings reinforce the need for optimal tip location at insertion and ongoing flushing and maintenance of PICCs not used for infusions.

Limitations of this study include the small sample size, which was not designed to detect statistical differences in the primary outcome between groups. Despite randomization, there were slight imbalances at baseline for inserter type and leukocyte count, although these were not significantly associated with PICC failure in the Cox regression (data not shown), and thus were unlikely to influence findings. Additionally, a difference of <10% was associated with PICC tip position, favoring the BioFlo^® group. PICC tip position outside the cavoatrial junction was positively associated with failure; therefore, the effect of tip positioning on outcomes is difficult to ascertain given the small sample size and feasibility nature of the study. Further study is warranted to further explore this effect. The population sampled was pediatric medical and surgical inpatients with a vessel size >2 mm attending the operating theater suite for PICC insertion, thereby limiting the study’s generalizability to adults and other populations, including neonates and those with PICCs inserted in the pediatric intensive care unit. The study could not be blinded because study products had to be visible to the clinical and research staff. However, it is unlikely that staff would intentionally sabotage PICCs to bias the study. Blinding was possible for the assessment of blood culture and ultrasound reports to diagnose infection and thrombosis. Strengths of this study included 100% protocol adherence, and no patients were lost to follow-up.

 

CONCLUSION

These results confirm that PICC failure is unacceptably high and suggest that the innovative BioFlo^® PICC material and design holds promise to improve PICC outcomes by reducing complications and overall PICC failure. Trials of this technology are feasible, safe, and acceptable to healthcare staff and parents. Further trials are required, including in other patient populations, to definitively identify clinical, cost-effective methods to prevent PICC failure and improve reliability during treatment.

Acknowledgments

The authors thank the children and parents of Lady Cilento Children’s Hospital for participating in this important research. A special thank you goes to the nurses within the Vascular Assessment and Management Service and to Karen Turner, Julieta Woosley, and Anna Dean for their efforts in data collecting and ensuring protocol adherence.

Disclosure

Griffith University has received unrestricted, investigator-initiated research or educational grants to support the research of T. K., A. J. U., and C. R. M. from product manufacturers 3M, Adhezion Inc, AngioDynamics, Bard Medical, Baxter, B. Braun Medical Inc, Becton Dickinson, CareFusion, Centurion Medical Products, Cook Medical, Entrotech, FloMedical, ICU Medical Inc, Medical Australia Limited, Medtronic, Smiths Medical, and Teleflex. Griffith University has received consultancy payments on behalf of C. R. M., A. J. U., and T. K. from manufacturers 3M, AngioDynamics, Bard Medical, B. Braun Medical Inc, Becton Dickinson, CareFusion, Mayo Healthcare Inc, ResQDevices, and Smiths Medical. AngioDynamics (the BioFlo^® PICC manufacturer) provided partial funds to undertake this research via an unrestricted donation to Griffith University (but not the study authors). Queensland Health provided in-kind support to fund the remainder of the trial. The funders had no role in the study design, collection, analysis, or interpretation of the data, writing of the report, or decision to submit the article for publication.

Testing for adrenal insufficiency (AI) is common in the hospital setting. The gold standard remains the insulin tolerance test (ITT), in which cortisol concentration is measured after the induction of hypoglycemia to <35 mg/dL.¹ Alternatively, metyrapone testing works by blocking cortisol synthesis. If pretest adrenocorticotropic hormone (ACTH) concentrations are low and ACTH concentrations do not rise after the administration of metyrapone, the patient is given a diagnosis of AI. Both assays pose some risk to patients with AI and are typically only performed as confirmatory tests. Morning random cortisol concentrations can be used to suggest AI if concentrations are <3 mcg/dL, but they often provide indeterminate results if concentrations are between 3 and 15 mcg/dL.² Thus, morning cortisol concentrations in isolation are not diagnostic of AI. For these reasons, most experts recommend a dynamic, high-dose cosyntropin stimulation testing (CST) with 250 mcg of intravenous cosyntropin to screen for AI. The test can be done any time of day.³ Historically, an incremental response to cosyntropin, or “delta,” was also required to indicate a normal response to stimulation.⁴ However, the baseline cortisol concentration is dependent on circadian rhythm and level of stress. For this reason, a delta, whether large or small, has been abandoned as a requisite for the diagnosis of AI.^5-7 A normal CST is widely accepted to be identified by any cortisol concentration >18 mcg/dL during the test (basal or poststimulation).⁸

The seminal studies by Lindholm, Kehlet, and coauthors^9-11 validated the CST against the gold standard ITT and utilized only 0- and 30-minute cortisol concentrations. A later study in patients with pituitary disease demonstrated that 30-minute concentrations had a stronger correlation with the ITT than 60-minute concentrations (false-negative rate: 10% vs 27%).¹² However, in that study, a higher threshold was used for the 60-minute concentration than for what was obtained at 30 minutes (25.4 vs 21.8 mcg/dL, respectively). Multiple studies have shown that the 60-minute concentration is higher than the 30-minute concentration after cosyntropin stimulation.^4,5,13 Subsequent, small studies of patients who were known to have AI have shown that 60-minute concentrations are as useful as 30-minute concentrations.^5,14,15 Because 30-minute cortisol concentrations are often lower than 60-minute concentrations, a single 30-minute result may lead to a falsely abnormal test.^16,17 As such, the use of a single 60-minute test may be more appropriate. Indeed, some authors have suggested that measuring only 30-minute concentrations may lead to overdiagnosis of AI by missing an appropriate response, serum cortisol >18 mcg/dL, at 60 minutes.^17-19 Peak cortisol concentrations after low-dose cosyntropin stimulation (1 mcg) are seen at 60 minutes, and low-dose stimulation has been shown to be more variable than in the high-dose test (250 mg).^19,20

There is a lack of consensus to guide clinicians as to when cortisol concentrations should be measured after stimulation, and standard references lack uniformity. Commonly accessed medical resources—such as UpToDate and Jameson’s Endocrinology—recommend basal, 30-minute, and 60-minute cortisol concentrations, while Williams Textbook of Endocrinology recommends basal and 30-minute concentrations, and the Washington Manual recommends only a single 30-minute concentration.^7,21,22 Goldman-Cecil Medicine⁸ recommends checking a cortisol concentration between 30 and 60 minutes and recommends the same 18 mcg/dL cutoff for any test obtained in this time period. As a result of these variable recommendations, all 3 time points are often obtained. Prominent review articles continue to recommend checking all 3 concentrations while presenting evidence of peak cortisol response at 60 minutes poststimulation.¹³

In this study, we retrospectively examined CSTs in hospitalized, adult patients both in the intensive care unit (ICU) and hospital ward and/or floor settings to evaluate for significant differences in 30- and 60-minute cortisol concentrations and compare the concordance of screening at each time point alone with traditional CST at all 3 time points. By using these results, we discuss the utility of obtaining 3 cortisol samples.

METHODS

After receiving approval from the institutional review board, we retrospectively reviewed all standard, high-dose CSTs performed on adult inpatients at the Barnes-Jewish Hospital laboratory from January 1, 2012, to August 31, 2013. All patients received the same standard dose (250 mcg cosyntropin, a synthetic ACTH, at a concentration of 1 mcg/mL administered over 2 minutes) regardless of age or weight. We collected patient gender; age; time of baseline cortisol measurement; cortisol results at baseline, 30, and 60 minutes; and patient location (inpatient floor vs ICU status). Tests were included if results from all 3 time points (0, 30, 60 minute) were available.

 

Cortisol concentrations were assessed by the laboratory according to the manufacturer’s instructions by using the ADVIA Centaur Cortisol assay (Siemens Healthcare Diagnostics Inc, Tarrytown, NY), a competitive chemiluminescent immunoassay. For the traditional CST, a cortisol concentration ≥18 mcg/dL at any time point during the test was used to define normal (negative). Patients with a positive (no results >18 mcg/mL) CST were defined as “screen positives” for the purposes of this analysis. Patient location data were available that allowed for an ICU vs non-ICU comparison.

Statistical analyses were performed in SAS version 9.4 (SAS Institute Inc, Cary, North Carolina). Continuous variables were compared by using a 2-tailed Student t test. Percentiles and proportions were compared by using χ² tests or Fisher’s exact tests when appropriate. The concordance of screening at each time point compared with the traditional CST was calculated. Positive percent agreement (PPA) with the traditional CST in each subgroup (ICU and floor) and combined was also evaluated. A P value of .05 was used to determine significance.

RESULTS

A total of 702 complete cosyntropin tests on separate patients were included in the analysis. This included 198 ICU patients and 504 non-ICU (floor) patients. Fifty-one percent of patients were male in both the floor and ICU subgroups. The average age of ICU patients was 60.2 ± 13.2 years compared to 57.3 ± 17.3 years for patients on a general medicine floor (P = .02).

Cortisol concentrations obtained at 30 minutes were significantly higher than baseline cortisol concentrations (baseline: 12.8 mcg/dL; 30 minutes: 23.9 mcg/dL; P < .001) for all patients. The average cortisol concentrations obtained at 60 minutes (27.4 mcg/dL) were significantly higher than those at baseline and 30 minutes (P < .001). This trend was seen in each subgroup of patients in the ICU and on the floor (Figure). The average baseline cortisol concentration was higher for ICU patients compared to floor patients (17.6 mcg/dL vs 10.9 mcg/dL, respectively).

By using the traditional CST, there were 26 (13.1%) positive tests for AI in ICU patients and 84 (16.7%) positive tests in floor patients (Table).

The Table shows the number of patients who screened positive at each time point and compares the concordance of these results with the results of the overall CST in each subgroup (ICU and floor). The 60-minute concentration demonstrated higher concordance with the traditional CST than the 30-minute concentration overall (99.7% vs 88.0%, respectively), in ICU patients (100% vs 91.9%, respectively), and in floor patients (99.6% vs 86.9%, respectively). In the ICU subgroup, 60-minute concentrations were 100% concordant with the traditional CSTs. The PPA of a 60-minute–only screening compared to a traditional CST was better than a 30-minute–only screening overall (98% vs 57%, respectively), in ICU patients (100% vs 62%, respectively), and in floor patients (98% vs 56%, respectively). A 60-minute concentration was required to prevent false-positive screening in 11.7% of all screening tests, but the 30-minute concentration only prevented false-positive screening in 0.3% of screening tests. Of all 30-minute concentrations screening positive for AI alone, 42.7% were negative for AI at 60 minutes. Conversely, only 1.8% of all 60-minute concentrations screening positive for AI alone were negative for AI at 30 minutes. The likelihood of a false-positive screening test at 30 minutes was higher in floor patients (13.1%) than in ICU patients (8.1%). The difference between the false-positive screening rate of a single 30-minute cortisol concentration and a single 60-minute concentration was significant (P < .0001) for both floor and ICU patients. There were no instances of basal cortisol concentrations >18 mcg/dL that were subsequently <18 mcg/dL at 30 and 60 minutes after cosyntropin stimulation.

Only 13% of CSTs were started in the recommended 3-hour window from 6:00 am to 8:59 am. The remaining tests were begun outside this window.

DISCUSSION

Our investigation of 702 CSTs, the largest retrospective analysis to date, finds that the 60-minute cortisol concentration is significantly higher than the 30-minute concentration in a standard, high-dose CST. Sixty-minute cortisol concentrations are more concordant with traditional CST results than the 30-minute concentrations in both critically ill ICU and noncritically ill floor patients. This suggests that a single 60-minute measurement is sufficient for AI screening. The use of only 30-minute concentrations would lead to a significant increase in false-positive screening tests and significantly lower PPA (98% vs 57%). With peak cortisol concentrations occurring at 60-minutes poststimulation, measuring both 30- and 60-minute poststimulation concentrations does not appear to be of significant clinical benefit. The cost-saving from reduced phlebotomy and laboratory expenses would be significant, especially in locations with limited staff or financial resources. Our findings are similar to other recent results by Chitale et al.,¹⁷ Mansoor et al.,¹⁶ and Zueger et al.¹⁸

 

Zueger et al.¹⁸ evaluated the results of high-dose CST in 73 patients and found 13.7% of patients with inadequate cortisol response (<18 mcg/dL) at 30 minutes had normal concentrations at 60 minutes (>18 mcg/dL). Their study did not identify a single case of normal cortisol concentration at 30 minutes that would have inappropriately screened positive for AI if cortisol concentrations were only checked at 60 minutes. Similarly, they suggested that the 30-minute test did not add any additional diagnostic value; however, no confirmatory testing was performed.

Higher cortisol concentrations at 60 minutes poststimulation may result in improved specificity for AI without reducing sensitivity, but it may also indicate that the cutoff value may need to be raised from 18 mcg/dL at 60 minutes to maintain an appropriate clinical sensitivity. Continued research should resolve this clinical question with gold-standard confirmatory testing. Furthermore, there is debate about an appropriate screening cortisol concentration threshold for critically ill patients. Researchers have compared concentrations of 25 mcg/dL to the traditional 18 mcg/dL to improve sensitivity for AI, but these studies do not involve comparisons to confirmatory testing and often result in reduced specificity.^23,24

In our study, only a small fraction of testing was performed in the early-morning hours, when basal cortisol results are of value. There may be indications to perform traditional CSTs with a basal concentration, such as for suspected secondary AI, but testing must be performed in the early morning for interpretable results per current recommendations. However, poststimulation cortisol concentrations may be interpreted regardless of the time of day at which the test was initiated.³

Our study is limited by its scope because it is a retrospective analysis. It is also limited by a lack of gold-standard, clinical confirmatory testing or analysis of other clinical data. Our method of testing and interpretation is considered the screening standard and is often used to plan treatment for AI without confirmatory testing, as ITT is not routinely available for hospitalized patients. The validation of the traditional CST to the ITT has been performed extensively, but a randomized trial comparing a single 60-minute concentration to the ITT may be useful. The exact timing of blood draws may have introduced error in the concentration measurements, and this is critical to screening accuracy. Total serum cortisol is 10% bound to albumin,²⁵ and medications such as steroids or opioids and medical conditions such as obesity or liver disease can affect cortisol concentrations.²⁶ Albumin and free cortisol concentrations that may be used to adjust for these variables were not available.

CONCLUSION

We recommend changes to the standard CST to exclude a basal cortisol concentration unless it is indicated for the evaluation of secondary AI or obtained at the appropriate early-morning hour. A single 60-minute poststimulation cortisol concentration may be an appropriate screening test for AI and demonstrates high concordance with the traditional CST. The use of a 30-minute poststimulation concentration alone may lead to a significantly higher number of false-positive results. Alternatively, the stimulated cortisol threshold used to define a normal test may need to be higher at 60 minutes to maintain the appropriate sensitivity. Further study and comparison with confirmatory testing are needed.

Disclosure

The authors have no relevant conflicts of interest to disclose.

Testing for adrenal insufficiency (AI) is common in the hospital setting. The gold standard remains the insulin tolerance test (ITT), in which cortisol concentration is measured after the induction of hypoglycemia to <35 mg/dL.¹ Alternatively, metyrapone testing works by blocking cortisol synthesis. If pretest adrenocorticotropic hormone (ACTH) concentrations are low and ACTH concentrations do not rise after the administration of metyrapone, the patient is given a diagnosis of AI. Both assays pose some risk to patients with AI and are typically only performed as confirmatory tests. Morning random cortisol concentrations can be used to suggest AI if concentrations are <3 mcg/dL, but they often provide indeterminate results if concentrations are between 3 and 15 mcg/dL.² Thus, morning cortisol concentrations in isolation are not diagnostic of AI. For these reasons, most experts recommend a dynamic, high-dose cosyntropin stimulation testing (CST) with 250 mcg of intravenous cosyntropin to screen for AI. The test can be done any time of day.³ Historically, an incremental response to cosyntropin, or “delta,” was also required to indicate a normal response to stimulation.⁴ However, the baseline cortisol concentration is dependent on circadian rhythm and level of stress. For this reason, a delta, whether large or small, has been abandoned as a requisite for the diagnosis of AI.^5-7 A normal CST is widely accepted to be identified by any cortisol concentration >18 mcg/dL during the test (basal or poststimulation).⁸

The seminal studies by Lindholm, Kehlet, and coauthors^9-11 validated the CST against the gold standard ITT and utilized only 0- and 30-minute cortisol concentrations. A later study in patients with pituitary disease demonstrated that 30-minute concentrations had a stronger correlation with the ITT than 60-minute concentrations (false-negative rate: 10% vs 27%).¹² However, in that study, a higher threshold was used for the 60-minute concentration than for what was obtained at 30 minutes (25.4 vs 21.8 mcg/dL, respectively). Multiple studies have shown that the 60-minute concentration is higher than the 30-minute concentration after cosyntropin stimulation.^4,5,13 Subsequent, small studies of patients who were known to have AI have shown that 60-minute concentrations are as useful as 30-minute concentrations.^5,14,15 Because 30-minute cortisol concentrations are often lower than 60-minute concentrations, a single 30-minute result may lead to a falsely abnormal test.^16,17 As such, the use of a single 60-minute test may be more appropriate. Indeed, some authors have suggested that measuring only 30-minute concentrations may lead to overdiagnosis of AI by missing an appropriate response, serum cortisol >18 mcg/dL, at 60 minutes.^17-19 Peak cortisol concentrations after low-dose cosyntropin stimulation (1 mcg) are seen at 60 minutes, and low-dose stimulation has been shown to be more variable than in the high-dose test (250 mg).^19,20

There is a lack of consensus to guide clinicians as to when cortisol concentrations should be measured after stimulation, and standard references lack uniformity. Commonly accessed medical resources—such as UpToDate and Jameson’s Endocrinology—recommend basal, 30-minute, and 60-minute cortisol concentrations, while Williams Textbook of Endocrinology recommends basal and 30-minute concentrations, and the Washington Manual recommends only a single 30-minute concentration.^7,21,22 Goldman-Cecil Medicine⁸ recommends checking a cortisol concentration between 30 and 60 minutes and recommends the same 18 mcg/dL cutoff for any test obtained in this time period. As a result of these variable recommendations, all 3 time points are often obtained. Prominent review articles continue to recommend checking all 3 concentrations while presenting evidence of peak cortisol response at 60 minutes poststimulation.¹³

In this study, we retrospectively examined CSTs in hospitalized, adult patients both in the intensive care unit (ICU) and hospital ward and/or floor settings to evaluate for significant differences in 30- and 60-minute cortisol concentrations and compare the concordance of screening at each time point alone with traditional CST at all 3 time points. By using these results, we discuss the utility of obtaining 3 cortisol samples.

METHODS

After receiving approval from the institutional review board, we retrospectively reviewed all standard, high-dose CSTs performed on adult inpatients at the Barnes-Jewish Hospital laboratory from January 1, 2012, to August 31, 2013. All patients received the same standard dose (250 mcg cosyntropin, a synthetic ACTH, at a concentration of 1 mcg/mL administered over 2 minutes) regardless of age or weight. We collected patient gender; age; time of baseline cortisol measurement; cortisol results at baseline, 30, and 60 minutes; and patient location (inpatient floor vs ICU status). Tests were included if results from all 3 time points (0, 30, 60 minute) were available.

 

Cortisol concentrations were assessed by the laboratory according to the manufacturer’s instructions by using the ADVIA Centaur Cortisol assay (Siemens Healthcare Diagnostics Inc, Tarrytown, NY), a competitive chemiluminescent immunoassay. For the traditional CST, a cortisol concentration ≥18 mcg/dL at any time point during the test was used to define normal (negative). Patients with a positive (no results >18 mcg/mL) CST were defined as “screen positives” for the purposes of this analysis. Patient location data were available that allowed for an ICU vs non-ICU comparison.

Statistical analyses were performed in SAS version 9.4 (SAS Institute Inc, Cary, North Carolina). Continuous variables were compared by using a 2-tailed Student t test. Percentiles and proportions were compared by using χ² tests or Fisher’s exact tests when appropriate. The concordance of screening at each time point compared with the traditional CST was calculated. Positive percent agreement (PPA) with the traditional CST in each subgroup (ICU and floor) and combined was also evaluated. A P value of .05 was used to determine significance.

RESULTS

A total of 702 complete cosyntropin tests on separate patients were included in the analysis. This included 198 ICU patients and 504 non-ICU (floor) patients. Fifty-one percent of patients were male in both the floor and ICU subgroups. The average age of ICU patients was 60.2 ± 13.2 years compared to 57.3 ± 17.3 years for patients on a general medicine floor (P = .02).

Cortisol concentrations obtained at 30 minutes were significantly higher than baseline cortisol concentrations (baseline: 12.8 mcg/dL; 30 minutes: 23.9 mcg/dL; P < .001) for all patients. The average cortisol concentrations obtained at 60 minutes (27.4 mcg/dL) were significantly higher than those at baseline and 30 minutes (P < .001). This trend was seen in each subgroup of patients in the ICU and on the floor (Figure). The average baseline cortisol concentration was higher for ICU patients compared to floor patients (17.6 mcg/dL vs 10.9 mcg/dL, respectively).

By using the traditional CST, there were 26 (13.1%) positive tests for AI in ICU patients and 84 (16.7%) positive tests in floor patients (Table).

The Table shows the number of patients who screened positive at each time point and compares the concordance of these results with the results of the overall CST in each subgroup (ICU and floor). The 60-minute concentration demonstrated higher concordance with the traditional CST than the 30-minute concentration overall (99.7% vs 88.0%, respectively), in ICU patients (100% vs 91.9%, respectively), and in floor patients (99.6% vs 86.9%, respectively). In the ICU subgroup, 60-minute concentrations were 100% concordant with the traditional CSTs. The PPA of a 60-minute–only screening compared to a traditional CST was better than a 30-minute–only screening overall (98% vs 57%, respectively), in ICU patients (100% vs 62%, respectively), and in floor patients (98% vs 56%, respectively). A 60-minute concentration was required to prevent false-positive screening in 11.7% of all screening tests, but the 30-minute concentration only prevented false-positive screening in 0.3% of screening tests. Of all 30-minute concentrations screening positive for AI alone, 42.7% were negative for AI at 60 minutes. Conversely, only 1.8% of all 60-minute concentrations screening positive for AI alone were negative for AI at 30 minutes. The likelihood of a false-positive screening test at 30 minutes was higher in floor patients (13.1%) than in ICU patients (8.1%). The difference between the false-positive screening rate of a single 30-minute cortisol concentration and a single 60-minute concentration was significant (P < .0001) for both floor and ICU patients. There were no instances of basal cortisol concentrations >18 mcg/dL that were subsequently <18 mcg/dL at 30 and 60 minutes after cosyntropin stimulation.

Only 13% of CSTs were started in the recommended 3-hour window from 6:00 am to 8:59 am. The remaining tests were begun outside this window.

DISCUSSION

Our investigation of 702 CSTs, the largest retrospective analysis to date, finds that the 60-minute cortisol concentration is significantly higher than the 30-minute concentration in a standard, high-dose CST. Sixty-minute cortisol concentrations are more concordant with traditional CST results than the 30-minute concentrations in both critically ill ICU and noncritically ill floor patients. This suggests that a single 60-minute measurement is sufficient for AI screening. The use of only 30-minute concentrations would lead to a significant increase in false-positive screening tests and significantly lower PPA (98% vs 57%). With peak cortisol concentrations occurring at 60-minutes poststimulation, measuring both 30- and 60-minute poststimulation concentrations does not appear to be of significant clinical benefit. The cost-saving from reduced phlebotomy and laboratory expenses would be significant, especially in locations with limited staff or financial resources. Our findings are similar to other recent results by Chitale et al.,¹⁷ Mansoor et al.,¹⁶ and Zueger et al.¹⁸

 

Zueger et al.¹⁸ evaluated the results of high-dose CST in 73 patients and found 13.7% of patients with inadequate cortisol response (<18 mcg/dL) at 30 minutes had normal concentrations at 60 minutes (>18 mcg/dL). Their study did not identify a single case of normal cortisol concentration at 30 minutes that would have inappropriately screened positive for AI if cortisol concentrations were only checked at 60 minutes. Similarly, they suggested that the 30-minute test did not add any additional diagnostic value; however, no confirmatory testing was performed.

Higher cortisol concentrations at 60 minutes poststimulation may result in improved specificity for AI without reducing sensitivity, but it may also indicate that the cutoff value may need to be raised from 18 mcg/dL at 60 minutes to maintain an appropriate clinical sensitivity. Continued research should resolve this clinical question with gold-standard confirmatory testing. Furthermore, there is debate about an appropriate screening cortisol concentration threshold for critically ill patients. Researchers have compared concentrations of 25 mcg/dL to the traditional 18 mcg/dL to improve sensitivity for AI, but these studies do not involve comparisons to confirmatory testing and often result in reduced specificity.^23,24

In our study, only a small fraction of testing was performed in the early-morning hours, when basal cortisol results are of value. There may be indications to perform traditional CSTs with a basal concentration, such as for suspected secondary AI, but testing must be performed in the early morning for interpretable results per current recommendations. However, poststimulation cortisol concentrations may be interpreted regardless of the time of day at which the test was initiated.³

Our study is limited by its scope because it is a retrospective analysis. It is also limited by a lack of gold-standard, clinical confirmatory testing or analysis of other clinical data. Our method of testing and interpretation is considered the screening standard and is often used to plan treatment for AI without confirmatory testing, as ITT is not routinely available for hospitalized patients. The validation of the traditional CST to the ITT has been performed extensively, but a randomized trial comparing a single 60-minute concentration to the ITT may be useful. The exact timing of blood draws may have introduced error in the concentration measurements, and this is critical to screening accuracy. Total serum cortisol is 10% bound to albumin,²⁵ and medications such as steroids or opioids and medical conditions such as obesity or liver disease can affect cortisol concentrations.²⁶ Albumin and free cortisol concentrations that may be used to adjust for these variables were not available.

CONCLUSION

We recommend changes to the standard CST to exclude a basal cortisol concentration unless it is indicated for the evaluation of secondary AI or obtained at the appropriate early-morning hour. A single 60-minute poststimulation cortisol concentration may be an appropriate screening test for AI and demonstrates high concordance with the traditional CST. The use of a 30-minute poststimulation concentration alone may lead to a significantly higher number of false-positive results. Alternatively, the stimulated cortisol threshold used to define a normal test may need to be higher at 60 minutes to maintain the appropriate sensitivity. Further study and comparison with confirmatory testing are needed.

Disclosure

The authors have no relevant conflicts of interest to disclose.

Testing for adrenal insufficiency (AI) is common in the hospital setting. The gold standard remains the insulin tolerance test (ITT), in which cortisol concentration is measured after the induction of hypoglycemia to <35 mg/dL.¹ Alternatively, metyrapone testing works by blocking cortisol synthesis. If pretest adrenocorticotropic hormone (ACTH) concentrations are low and ACTH concentrations do not rise after the administration of metyrapone, the patient is given a diagnosis of AI. Both assays pose some risk to patients with AI and are typically only performed as confirmatory tests. Morning random cortisol concentrations can be used to suggest AI if concentrations are <3 mcg/dL, but they often provide indeterminate results if concentrations are between 3 and 15 mcg/dL.² Thus, morning cortisol concentrations in isolation are not diagnostic of AI. For these reasons, most experts recommend a dynamic, high-dose cosyntropin stimulation testing (CST) with 250 mcg of intravenous cosyntropin to screen for AI. The test can be done any time of day.³ Historically, an incremental response to cosyntropin, or “delta,” was also required to indicate a normal response to stimulation.⁴ However, the baseline cortisol concentration is dependent on circadian rhythm and level of stress. For this reason, a delta, whether large or small, has been abandoned as a requisite for the diagnosis of AI.^5-7 A normal CST is widely accepted to be identified by any cortisol concentration >18 mcg/dL during the test (basal or poststimulation).⁸

The seminal studies by Lindholm, Kehlet, and coauthors^9-11 validated the CST against the gold standard ITT and utilized only 0- and 30-minute cortisol concentrations. A later study in patients with pituitary disease demonstrated that 30-minute concentrations had a stronger correlation with the ITT than 60-minute concentrations (false-negative rate: 10% vs 27%).¹² However, in that study, a higher threshold was used for the 60-minute concentration than for what was obtained at 30 minutes (25.4 vs 21.8 mcg/dL, respectively). Multiple studies have shown that the 60-minute concentration is higher than the 30-minute concentration after cosyntropin stimulation.^4,5,13 Subsequent, small studies of patients who were known to have AI have shown that 60-minute concentrations are as useful as 30-minute concentrations.^5,14,15 Because 30-minute cortisol concentrations are often lower than 60-minute concentrations, a single 30-minute result may lead to a falsely abnormal test.^16,17 As such, the use of a single 60-minute test may be more appropriate. Indeed, some authors have suggested that measuring only 30-minute concentrations may lead to overdiagnosis of AI by missing an appropriate response, serum cortisol >18 mcg/dL, at 60 minutes.^17-19 Peak cortisol concentrations after low-dose cosyntropin stimulation (1 mcg) are seen at 60 minutes, and low-dose stimulation has been shown to be more variable than in the high-dose test (250 mg).^19,20

There is a lack of consensus to guide clinicians as to when cortisol concentrations should be measured after stimulation, and standard references lack uniformity. Commonly accessed medical resources—such as UpToDate and Jameson’s Endocrinology—recommend basal, 30-minute, and 60-minute cortisol concentrations, while Williams Textbook of Endocrinology recommends basal and 30-minute concentrations, and the Washington Manual recommends only a single 30-minute concentration.^7,21,22 Goldman-Cecil Medicine⁸ recommends checking a cortisol concentration between 30 and 60 minutes and recommends the same 18 mcg/dL cutoff for any test obtained in this time period. As a result of these variable recommendations, all 3 time points are often obtained. Prominent review articles continue to recommend checking all 3 concentrations while presenting evidence of peak cortisol response at 60 minutes poststimulation.¹³

In this study, we retrospectively examined CSTs in hospitalized, adult patients both in the intensive care unit (ICU) and hospital ward and/or floor settings to evaluate for significant differences in 30- and 60-minute cortisol concentrations and compare the concordance of screening at each time point alone with traditional CST at all 3 time points. By using these results, we discuss the utility of obtaining 3 cortisol samples.

METHODS

After receiving approval from the institutional review board, we retrospectively reviewed all standard, high-dose CSTs performed on adult inpatients at the Barnes-Jewish Hospital laboratory from January 1, 2012, to August 31, 2013. All patients received the same standard dose (250 mcg cosyntropin, a synthetic ACTH, at a concentration of 1 mcg/mL administered over 2 minutes) regardless of age or weight. We collected patient gender; age; time of baseline cortisol measurement; cortisol results at baseline, 30, and 60 minutes; and patient location (inpatient floor vs ICU status). Tests were included if results from all 3 time points (0, 30, 60 minute) were available.

 

Cortisol concentrations were assessed by the laboratory according to the manufacturer’s instructions by using the ADVIA Centaur Cortisol assay (Siemens Healthcare Diagnostics Inc, Tarrytown, NY), a competitive chemiluminescent immunoassay. For the traditional CST, a cortisol concentration ≥18 mcg/dL at any time point during the test was used to define normal (negative). Patients with a positive (no results >18 mcg/mL) CST were defined as “screen positives” for the purposes of this analysis. Patient location data were available that allowed for an ICU vs non-ICU comparison.

Statistical analyses were performed in SAS version 9.4 (SAS Institute Inc, Cary, North Carolina). Continuous variables were compared by using a 2-tailed Student t test. Percentiles and proportions were compared by using χ² tests or Fisher’s exact tests when appropriate. The concordance of screening at each time point compared with the traditional CST was calculated. Positive percent agreement (PPA) with the traditional CST in each subgroup (ICU and floor) and combined was also evaluated. A P value of .05 was used to determine significance.

RESULTS

A total of 702 complete cosyntropin tests on separate patients were included in the analysis. This included 198 ICU patients and 504 non-ICU (floor) patients. Fifty-one percent of patients were male in both the floor and ICU subgroups. The average age of ICU patients was 60.2 ± 13.2 years compared to 57.3 ± 17.3 years for patients on a general medicine floor (P = .02).

Cortisol concentrations obtained at 30 minutes were significantly higher than baseline cortisol concentrations (baseline: 12.8 mcg/dL; 30 minutes: 23.9 mcg/dL; P < .001) for all patients. The average cortisol concentrations obtained at 60 minutes (27.4 mcg/dL) were significantly higher than those at baseline and 30 minutes (P < .001). This trend was seen in each subgroup of patients in the ICU and on the floor (Figure). The average baseline cortisol concentration was higher for ICU patients compared to floor patients (17.6 mcg/dL vs 10.9 mcg/dL, respectively).

By using the traditional CST, there were 26 (13.1%) positive tests for AI in ICU patients and 84 (16.7%) positive tests in floor patients (Table).

The Table shows the number of patients who screened positive at each time point and compares the concordance of these results with the results of the overall CST in each subgroup (ICU and floor). The 60-minute concentration demonstrated higher concordance with the traditional CST than the 30-minute concentration overall (99.7% vs 88.0%, respectively), in ICU patients (100% vs 91.9%, respectively), and in floor patients (99.6% vs 86.9%, respectively). In the ICU subgroup, 60-minute concentrations were 100% concordant with the traditional CSTs. The PPA of a 60-minute–only screening compared to a traditional CST was better than a 30-minute–only screening overall (98% vs 57%, respectively), in ICU patients (100% vs 62%, respectively), and in floor patients (98% vs 56%, respectively). A 60-minute concentration was required to prevent false-positive screening in 11.7% of all screening tests, but the 30-minute concentration only prevented false-positive screening in 0.3% of screening tests. Of all 30-minute concentrations screening positive for AI alone, 42.7% were negative for AI at 60 minutes. Conversely, only 1.8% of all 60-minute concentrations screening positive for AI alone were negative for AI at 30 minutes. The likelihood of a false-positive screening test at 30 minutes was higher in floor patients (13.1%) than in ICU patients (8.1%). The difference between the false-positive screening rate of a single 30-minute cortisol concentration and a single 60-minute concentration was significant (P < .0001) for both floor and ICU patients. There were no instances of basal cortisol concentrations >18 mcg/dL that were subsequently <18 mcg/dL at 30 and 60 minutes after cosyntropin stimulation.

Only 13% of CSTs were started in the recommended 3-hour window from 6:00 am to 8:59 am. The remaining tests were begun outside this window.

DISCUSSION

Our investigation of 702 CSTs, the largest retrospective analysis to date, finds that the 60-minute cortisol concentration is significantly higher than the 30-minute concentration in a standard, high-dose CST. Sixty-minute cortisol concentrations are more concordant with traditional CST results than the 30-minute concentrations in both critically ill ICU and noncritically ill floor patients. This suggests that a single 60-minute measurement is sufficient for AI screening. The use of only 30-minute concentrations would lead to a significant increase in false-positive screening tests and significantly lower PPA (98% vs 57%). With peak cortisol concentrations occurring at 60-minutes poststimulation, measuring both 30- and 60-minute poststimulation concentrations does not appear to be of significant clinical benefit. The cost-saving from reduced phlebotomy and laboratory expenses would be significant, especially in locations with limited staff or financial resources. Our findings are similar to other recent results by Chitale et al.,¹⁷ Mansoor et al.,¹⁶ and Zueger et al.¹⁸

 

Zueger et al.¹⁸ evaluated the results of high-dose CST in 73 patients and found 13.7% of patients with inadequate cortisol response (<18 mcg/dL) at 30 minutes had normal concentrations at 60 minutes (>18 mcg/dL). Their study did not identify a single case of normal cortisol concentration at 30 minutes that would have inappropriately screened positive for AI if cortisol concentrations were only checked at 60 minutes. Similarly, they suggested that the 30-minute test did not add any additional diagnostic value; however, no confirmatory testing was performed.

Higher cortisol concentrations at 60 minutes poststimulation may result in improved specificity for AI without reducing sensitivity, but it may also indicate that the cutoff value may need to be raised from 18 mcg/dL at 60 minutes to maintain an appropriate clinical sensitivity. Continued research should resolve this clinical question with gold-standard confirmatory testing. Furthermore, there is debate about an appropriate screening cortisol concentration threshold for critically ill patients. Researchers have compared concentrations of 25 mcg/dL to the traditional 18 mcg/dL to improve sensitivity for AI, but these studies do not involve comparisons to confirmatory testing and often result in reduced specificity.^23,24

In our study, only a small fraction of testing was performed in the early-morning hours, when basal cortisol results are of value. There may be indications to perform traditional CSTs with a basal concentration, such as for suspected secondary AI, but testing must be performed in the early morning for interpretable results per current recommendations. However, poststimulation cortisol concentrations may be interpreted regardless of the time of day at which the test was initiated.³

Our study is limited by its scope because it is a retrospective analysis. It is also limited by a lack of gold-standard, clinical confirmatory testing or analysis of other clinical data. Our method of testing and interpretation is considered the screening standard and is often used to plan treatment for AI without confirmatory testing, as ITT is not routinely available for hospitalized patients. The validation of the traditional CST to the ITT has been performed extensively, but a randomized trial comparing a single 60-minute concentration to the ITT may be useful. The exact timing of blood draws may have introduced error in the concentration measurements, and this is critical to screening accuracy. Total serum cortisol is 10% bound to albumin,²⁵ and medications such as steroids or opioids and medical conditions such as obesity or liver disease can affect cortisol concentrations.²⁶ Albumin and free cortisol concentrations that may be used to adjust for these variables were not available.

CONCLUSION

We recommend changes to the standard CST to exclude a basal cortisol concentration unless it is indicated for the evaluation of secondary AI or obtained at the appropriate early-morning hour. A single 60-minute poststimulation cortisol concentration may be an appropriate screening test for AI and demonstrates high concordance with the traditional CST. The use of a 30-minute poststimulation concentration alone may lead to a significantly higher number of false-positive results. Alternatively, the stimulated cortisol threshold used to define a normal test may need to be higher at 60 minutes to maintain the appropriate sensitivity. Further study and comparison with confirmatory testing are needed.

Disclosure

The authors have no relevant conflicts of interest to disclose.

Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

METHODS

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

 

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

The period between January 1, 2014, and December 31, 2014, was considered preintervention, and August 1, 2015, to July 31, 2016, was considered postintervention. January 1, 2015, to July 31, 2015, was considered a “run-in” period because it was the interval during which the interventions on the hospitalist service were being rolled out. Data from this period were not included in the pre- or postintervention analyses but are shown in Figure 1.

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

 

RESULTS

There were 46,215 visits originally included in the study. Ninety-two visits (0.2%) were excluded due to missing or invalid data. A total of 10,344 visits occurred during the “run-in” period between January 1, 2015, and July 31, 2015, leaving 35,871 patient visits during the pre- and postintervention periods. In the hospitalist group, there were 3442 visits before the intervention and 3700 after. There were 13,470 visits in the nonhospitalist group before the intervention and 15,259 after.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

Estimates from adjusted and unadjusted linear models are shown in Table 2. In adjusted models, telemetry utilization in the postintervention period was reduced by 69% (95% confidence interval [CI], −72% to −64%; P < .001) in the hospitalist group and by 22% (95% CI, −27% to −16%; P <.001) in the nonhospitalist group. Compared with nonhospitalists, hospitalists had a 60% greater reduction in telemetry rates (95% CI, −65% to −54%; P < .001).

In the randomly selected sample of patients pre- and postintervention who received telemetry monitoring, there was an increase in telemetry appropriateness on the hospitalist service (46% to 72%, P = .025; Table 1). In the nonhospitalist group, appropriate telemetry utilization did not change significantly. Of the 100 randomly selected patients in the hospitalist group after the intervention who did not receive telemetry, no patient had an AHA Class I indication, and only 4 patients had a Class II indication.^3,17

DISCUSSION

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

 

CONCLUSION

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

METHODS

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

 

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

The period between January 1, 2014, and December 31, 2014, was considered preintervention, and August 1, 2015, to July 31, 2016, was considered postintervention. January 1, 2015, to July 31, 2015, was considered a “run-in” period because it was the interval during which the interventions on the hospitalist service were being rolled out. Data from this period were not included in the pre- or postintervention analyses but are shown in Figure 1.

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

 

RESULTS

There were 46,215 visits originally included in the study. Ninety-two visits (0.2%) were excluded due to missing or invalid data. A total of 10,344 visits occurred during the “run-in” period between January 1, 2015, and July 31, 2015, leaving 35,871 patient visits during the pre- and postintervention periods. In the hospitalist group, there were 3442 visits before the intervention and 3700 after. There were 13,470 visits in the nonhospitalist group before the intervention and 15,259 after.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

Estimates from adjusted and unadjusted linear models are shown in Table 2. In adjusted models, telemetry utilization in the postintervention period was reduced by 69% (95% confidence interval [CI], −72% to −64%; P < .001) in the hospitalist group and by 22% (95% CI, −27% to −16%; P <.001) in the nonhospitalist group. Compared with nonhospitalists, hospitalists had a 60% greater reduction in telemetry rates (95% CI, −65% to −54%; P < .001).

In the randomly selected sample of patients pre- and postintervention who received telemetry monitoring, there was an increase in telemetry appropriateness on the hospitalist service (46% to 72%, P = .025; Table 1). In the nonhospitalist group, appropriate telemetry utilization did not change significantly. Of the 100 randomly selected patients in the hospitalist group after the intervention who did not receive telemetry, no patient had an AHA Class I indication, and only 4 patients had a Class II indication.^3,17

DISCUSSION

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

 

CONCLUSION

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

METHODS

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

 

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

The period between January 1, 2014, and December 31, 2014, was considered preintervention, and August 1, 2015, to July 31, 2016, was considered postintervention. January 1, 2015, to July 31, 2015, was considered a “run-in” period because it was the interval during which the interventions on the hospitalist service were being rolled out. Data from this period were not included in the pre- or postintervention analyses but are shown in Figure 1.

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

 

RESULTS

There were 46,215 visits originally included in the study. Ninety-two visits (0.2%) were excluded due to missing or invalid data. A total of 10,344 visits occurred during the “run-in” period between January 1, 2015, and July 31, 2015, leaving 35,871 patient visits during the pre- and postintervention periods. In the hospitalist group, there were 3442 visits before the intervention and 3700 after. There were 13,470 visits in the nonhospitalist group before the intervention and 15,259 after.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

Estimates from adjusted and unadjusted linear models are shown in Table 2. In adjusted models, telemetry utilization in the postintervention period was reduced by 69% (95% confidence interval [CI], −72% to −64%; P < .001) in the hospitalist group and by 22% (95% CI, −27% to −16%; P <.001) in the nonhospitalist group. Compared with nonhospitalists, hospitalists had a 60% greater reduction in telemetry rates (95% CI, −65% to −54%; P < .001).

In the randomly selected sample of patients pre- and postintervention who received telemetry monitoring, there was an increase in telemetry appropriateness on the hospitalist service (46% to 72%, P = .025; Table 1). In the nonhospitalist group, appropriate telemetry utilization did not change significantly. Of the 100 randomly selected patients in the hospitalist group after the intervention who did not receive telemetry, no patient had an AHA Class I indication, and only 4 patients had a Class II indication.^3,17

DISCUSSION

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

 

CONCLUSION

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

METHODS

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

 

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

RESULTS

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

We observed an increase in the mean predicted readmission risk for hospitalizations in the surgery/gynecology and cardiovascular hospitalizations in early 2011 (Figure 1), a period between the introduction of ACA in March 2010 and the introduction of HRRP in October 2012. In time series models, the surgery/gynecology, cardiovascular, and neurology cohorts had increased predictive risks of readmission of 0.24%, 0.32%, and 0.13% per year, respectively, although this difference did not reach statistical significance for the cardiovascular cohort (Table 1). We found no association between introduction of ACA or HRRP and predicted risk for these cohorts (Table 1). There were no trends or differences in predicted readmission risk for hospitalizations in the medicine cohort. We observed a seasonal variation in predicted readmission risk for the cardiorespiratory cohort but no notable change in predicted risk over time (Figure 1); in the time series model, there was a slight decrease in risk following introduction of HRRP (Table 1).

After categorizing hospitalizations by predicted readmission risk, trends in the percent of hospitalizations in low, middle, and high risk strata differed by cohort. In the surgery/gynecology cohort, the proportion of hospitalizations in the lowest risk stratum increased only slightly, from 20.1% in 2009 to 21.1% of all surgery/gynecology hospitalizations in 2015 (Appendix Table 2). The proportion of surgery/gynecology hospitalizations in the high risk stratum (top quintile of risk) increased from 16.1% to 21.6% between 2009 and 2011 and remained at 21.8% in 2015, and the proportion of surgery/gynecology hospitalizations in the middle risk stratum (middle three quintiles of risk) decreased from 63.7% in 2009 to 59.4% in 2011 to 57.1% in 2015. Low-risk hospitalizations in the medicine cohort decreased from 21.7% in 2009 to 19.0% in 2015, while high-risk hospitalizations increased from 18.2% to 20.7% during the period. Hospitalizations in the lowest stratum of risk steadily declined in both the cardiovascular and neurology cohorts, from 24.9% to 14.8% and 22.6% to 17.3% of hospitalizations during the period, respectively; this was accompanied by an increase in the proportion of high-risk hospitalizations in each of these cohorts from 16.0% to 23.4% and 17.8% to 21.6%, respectively. The proportion of hospitalizations in each of the 3 risk strata remained relatively stable in the cardiorespiratory cohort (Appendix Table 2).

In each of the 5 cohorts, O/E readmissions steadily declined from 2009 to 2015 for hospitalizations with the lowest, middle, and highest predicted readmission risk (Figure 2). Each risk stratum had similar rates of decline during the study period for all cohorts (Table 2). Among surgery/gynecology hospitalizations, the monthly O/E readmission declined by 0.030 per year from an initial ratio of 0.936 for the lowest risk hospitalizations, by 0.037 per year for the middle risk hospitalizations, and by 0.036 per year for the highest risk hospitalizations (Table 2). Similarly, for hospitalizations in the lowest versus highest risk of readmission, annual decreases in O/E readmission rates were 0.018 versus 0.015, 0.034 versus 0.033, 0.020 versus 0.015, and 0.038 versus 0.029 for the medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively. For all cohorts and in all risk strata, we found no significant change in O/E readmission risk with introduction of ACA or HRRP (Table 2).

 

DISCUSSION

In this 6-year, national study of Medicare hospitalizations, we found that readmission risk increased over time for surgical and neurological patients but did not increase in medicine or cardiorespiratory hospitalizations, even though those cohorts are known to have had substantial decreases in admissions and readmissions over the same time period.^7,8 Moreover, we found that O/E readmissions decreased similarly for all hospitalized Medicare patients, whether of low, moderate, or high risk of readmission. These findings suggest that hospital efforts have resulted in improved outcomes across the risk spectrum.

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

 

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

METHODS

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

 

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

RESULTS

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

We observed an increase in the mean predicted readmission risk for hospitalizations in the surgery/gynecology and cardiovascular hospitalizations in early 2011 (Figure 1), a period between the introduction of ACA in March 2010 and the introduction of HRRP in October 2012. In time series models, the surgery/gynecology, cardiovascular, and neurology cohorts had increased predictive risks of readmission of 0.24%, 0.32%, and 0.13% per year, respectively, although this difference did not reach statistical significance for the cardiovascular cohort (Table 1). We found no association between introduction of ACA or HRRP and predicted risk for these cohorts (Table 1). There were no trends or differences in predicted readmission risk for hospitalizations in the medicine cohort. We observed a seasonal variation in predicted readmission risk for the cardiorespiratory cohort but no notable change in predicted risk over time (Figure 1); in the time series model, there was a slight decrease in risk following introduction of HRRP (Table 1).

After categorizing hospitalizations by predicted readmission risk, trends in the percent of hospitalizations in low, middle, and high risk strata differed by cohort. In the surgery/gynecology cohort, the proportion of hospitalizations in the lowest risk stratum increased only slightly, from 20.1% in 2009 to 21.1% of all surgery/gynecology hospitalizations in 2015 (Appendix Table 2). The proportion of surgery/gynecology hospitalizations in the high risk stratum (top quintile of risk) increased from 16.1% to 21.6% between 2009 and 2011 and remained at 21.8% in 2015, and the proportion of surgery/gynecology hospitalizations in the middle risk stratum (middle three quintiles of risk) decreased from 63.7% in 2009 to 59.4% in 2011 to 57.1% in 2015. Low-risk hospitalizations in the medicine cohort decreased from 21.7% in 2009 to 19.0% in 2015, while high-risk hospitalizations increased from 18.2% to 20.7% during the period. Hospitalizations in the lowest stratum of risk steadily declined in both the cardiovascular and neurology cohorts, from 24.9% to 14.8% and 22.6% to 17.3% of hospitalizations during the period, respectively; this was accompanied by an increase in the proportion of high-risk hospitalizations in each of these cohorts from 16.0% to 23.4% and 17.8% to 21.6%, respectively. The proportion of hospitalizations in each of the 3 risk strata remained relatively stable in the cardiorespiratory cohort (Appendix Table 2).

In each of the 5 cohorts, O/E readmissions steadily declined from 2009 to 2015 for hospitalizations with the lowest, middle, and highest predicted readmission risk (Figure 2). Each risk stratum had similar rates of decline during the study period for all cohorts (Table 2). Among surgery/gynecology hospitalizations, the monthly O/E readmission declined by 0.030 per year from an initial ratio of 0.936 for the lowest risk hospitalizations, by 0.037 per year for the middle risk hospitalizations, and by 0.036 per year for the highest risk hospitalizations (Table 2). Similarly, for hospitalizations in the lowest versus highest risk of readmission, annual decreases in O/E readmission rates were 0.018 versus 0.015, 0.034 versus 0.033, 0.020 versus 0.015, and 0.038 versus 0.029 for the medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively. For all cohorts and in all risk strata, we found no significant change in O/E readmission risk with introduction of ACA or HRRP (Table 2).

 

DISCUSSION

In this 6-year, national study of Medicare hospitalizations, we found that readmission risk increased over time for surgical and neurological patients but did not increase in medicine or cardiorespiratory hospitalizations, even though those cohorts are known to have had substantial decreases in admissions and readmissions over the same time period.^7,8 Moreover, we found that O/E readmissions decreased similarly for all hospitalized Medicare patients, whether of low, moderate, or high risk of readmission. These findings suggest that hospital efforts have resulted in improved outcomes across the risk spectrum.

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

 

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

METHODS

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

 

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

RESULTS

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

We observed an increase in the mean predicted readmission risk for hospitalizations in the surgery/gynecology and cardiovascular hospitalizations in early 2011 (Figure 1), a period between the introduction of ACA in March 2010 and the introduction of HRRP in October 2012. In time series models, the surgery/gynecology, cardiovascular, and neurology cohorts had increased predictive risks of readmission of 0.24%, 0.32%, and 0.13% per year, respectively, although this difference did not reach statistical significance for the cardiovascular cohort (Table 1). We found no association between introduction of ACA or HRRP and predicted risk for these cohorts (Table 1). There were no trends or differences in predicted readmission risk for hospitalizations in the medicine cohort. We observed a seasonal variation in predicted readmission risk for the cardiorespiratory cohort but no notable change in predicted risk over time (Figure 1); in the time series model, there was a slight decrease in risk following introduction of HRRP (Table 1).

After categorizing hospitalizations by predicted readmission risk, trends in the percent of hospitalizations in low, middle, and high risk strata differed by cohort. In the surgery/gynecology cohort, the proportion of hospitalizations in the lowest risk stratum increased only slightly, from 20.1% in 2009 to 21.1% of all surgery/gynecology hospitalizations in 2015 (Appendix Table 2). The proportion of surgery/gynecology hospitalizations in the high risk stratum (top quintile of risk) increased from 16.1% to 21.6% between 2009 and 2011 and remained at 21.8% in 2015, and the proportion of surgery/gynecology hospitalizations in the middle risk stratum (middle three quintiles of risk) decreased from 63.7% in 2009 to 59.4% in 2011 to 57.1% in 2015. Low-risk hospitalizations in the medicine cohort decreased from 21.7% in 2009 to 19.0% in 2015, while high-risk hospitalizations increased from 18.2% to 20.7% during the period. Hospitalizations in the lowest stratum of risk steadily declined in both the cardiovascular and neurology cohorts, from 24.9% to 14.8% and 22.6% to 17.3% of hospitalizations during the period, respectively; this was accompanied by an increase in the proportion of high-risk hospitalizations in each of these cohorts from 16.0% to 23.4% and 17.8% to 21.6%, respectively. The proportion of hospitalizations in each of the 3 risk strata remained relatively stable in the cardiorespiratory cohort (Appendix Table 2).

In each of the 5 cohorts, O/E readmissions steadily declined from 2009 to 2015 for hospitalizations with the lowest, middle, and highest predicted readmission risk (Figure 2). Each risk stratum had similar rates of decline during the study period for all cohorts (Table 2). Among surgery/gynecology hospitalizations, the monthly O/E readmission declined by 0.030 per year from an initial ratio of 0.936 for the lowest risk hospitalizations, by 0.037 per year for the middle risk hospitalizations, and by 0.036 per year for the highest risk hospitalizations (Table 2). Similarly, for hospitalizations in the lowest versus highest risk of readmission, annual decreases in O/E readmission rates were 0.018 versus 0.015, 0.034 versus 0.033, 0.020 versus 0.015, and 0.038 versus 0.029 for the medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively. For all cohorts and in all risk strata, we found no significant change in O/E readmission risk with introduction of ACA or HRRP (Table 2).

 

DISCUSSION

In this 6-year, national study of Medicare hospitalizations, we found that readmission risk increased over time for surgical and neurological patients but did not increase in medicine or cardiorespiratory hospitalizations, even though those cohorts are known to have had substantial decreases in admissions and readmissions over the same time period.^7,8 Moreover, we found that O/E readmissions decreased similarly for all hospitalized Medicare patients, whether of low, moderate, or high risk of readmission. These findings suggest that hospital efforts have resulted in improved outcomes across the risk spectrum.

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

 

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

MATERIALS AND METHODS

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

All components of the training program are required to receive the certificate of completion with the exception of the refresher training. Learner feedback after each 3-day course and refresher training was incorporated into subsequent iterations of the training program. During initial phases, additional hands-on faculty were recruited from emergency medicine and critical care who had extensive experience with bedside ultrasound. Subsequently, faculty consisted of former course participants. All faculty followed a standard set of ultrasound and educational principles to guide the hands-on training of participants (supplementary Appendix 2).

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

 

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Portfolio development was a key aspect in overall POCUS competency for each participant. The hospital medicine department’s required portfolio files are presented in the Figure, with standards coinciding with the quality assurance grading rubric as developed by the POCUS committee at Regions Hospital and described by Mathews and Zwank.²⁸ Images taken with real patients were submitted without patient identifiers to a shared online portal. Faculty provided regular cycling feedback by entering the status of submission (accepted or declined) and specific comments on images and interpretations. Learners worked off of the feedback, practiced their skills, and resubmitted files. An image was considered acceptable if it met criteria of depth, axis, and gain and showed the required organ. Participants could use the same patient for different views but could not use the same patient for multiple images of the same view.

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

 

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For hospitalists who completed both the 3-day and 1-day courses, the change in course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, was assessed by summarizing the change from post 3-day metrics to pre 1-day metrics (Table 2). The differences between these 2 assessments were evaluated by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level. Changes in skills score from post 3-day assessment to pre 1-day assessment were also compared for hospitalists completing any of the portfolio and those completing none, and for hospitalists attending any monthly scanning sessions and those who did not attend any, by using analysis of variance and Scheffe tests.

Multiple linear regression was performed with the change in skills assessment score from postcompletion of the 3-day course to precompletion of the 1-day course as the dependent variable. Hospitalists were split into 2 age groups (30-39 and 40-49) for the purpose of this analysis. The percent of monthly scanning sessions attended, age category, timing of 1-day course, and percent portfolio were assessed as possible predictors of the skills score by using simple linear regression with a P = .05 cutoff. A final model was chosen based on predictors significant in simple linear regression and included the percent of the portfolio completed and attendance of monthly scanning sessions.

RESULTS

Demographics

Of the 56 3-day course participants, 53 had complete data (Table 3). Three participants with incomplete data completed most of the course but left prior to postcourse assessments and were excluded from the analysis. Twenty-three hospitalists also completed the 1-day in-person course. Seven hospitalists completed the 1-day course 3 months after the initial course, 8 completed it at 6 months, and 8 completed it at 12 months. Completed portfolios required 164 approved video images. Fifteen of the 23 hospitalists at the 1-day course have started and are working towards completion of the online portfolio, while 9 of the 23 participated in the monthly scanning sessions.

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

 

DISCUSSION

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

CONCLUSION

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

MATERIALS AND METHODS

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

All components of the training program are required to receive the certificate of completion with the exception of the refresher training. Learner feedback after each 3-day course and refresher training was incorporated into subsequent iterations of the training program. During initial phases, additional hands-on faculty were recruited from emergency medicine and critical care who had extensive experience with bedside ultrasound. Subsequently, faculty consisted of former course participants. All faculty followed a standard set of ultrasound and educational principles to guide the hands-on training of participants (supplementary Appendix 2).

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

 

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Portfolio development was a key aspect in overall POCUS competency for each participant. The hospital medicine department’s required portfolio files are presented in the Figure, with standards coinciding with the quality assurance grading rubric as developed by the POCUS committee at Regions Hospital and described by Mathews and Zwank.²⁸ Images taken with real patients were submitted without patient identifiers to a shared online portal. Faculty provided regular cycling feedback by entering the status of submission (accepted or declined) and specific comments on images and interpretations. Learners worked off of the feedback, practiced their skills, and resubmitted files. An image was considered acceptable if it met criteria of depth, axis, and gain and showed the required organ. Participants could use the same patient for different views but could not use the same patient for multiple images of the same view.

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

 

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For hospitalists who completed both the 3-day and 1-day courses, the change in course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, was assessed by summarizing the change from post 3-day metrics to pre 1-day metrics (Table 2). The differences between these 2 assessments were evaluated by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level. Changes in skills score from post 3-day assessment to pre 1-day assessment were also compared for hospitalists completing any of the portfolio and those completing none, and for hospitalists attending any monthly scanning sessions and those who did not attend any, by using analysis of variance and Scheffe tests.

Multiple linear regression was performed with the change in skills assessment score from postcompletion of the 3-day course to precompletion of the 1-day course as the dependent variable. Hospitalists were split into 2 age groups (30-39 and 40-49) for the purpose of this analysis. The percent of monthly scanning sessions attended, age category, timing of 1-day course, and percent portfolio were assessed as possible predictors of the skills score by using simple linear regression with a P = .05 cutoff. A final model was chosen based on predictors significant in simple linear regression and included the percent of the portfolio completed and attendance of monthly scanning sessions.

RESULTS

Demographics

Of the 56 3-day course participants, 53 had complete data (Table 3). Three participants with incomplete data completed most of the course but left prior to postcourse assessments and were excluded from the analysis. Twenty-three hospitalists also completed the 1-day in-person course. Seven hospitalists completed the 1-day course 3 months after the initial course, 8 completed it at 6 months, and 8 completed it at 12 months. Completed portfolios required 164 approved video images. Fifteen of the 23 hospitalists at the 1-day course have started and are working towards completion of the online portfolio, while 9 of the 23 participated in the monthly scanning sessions.

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

 

DISCUSSION

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

CONCLUSION

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

MATERIALS AND METHODS

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

All components of the training program are required to receive the certificate of completion with the exception of the refresher training. Learner feedback after each 3-day course and refresher training was incorporated into subsequent iterations of the training program. During initial phases, additional hands-on faculty were recruited from emergency medicine and critical care who had extensive experience with bedside ultrasound. Subsequently, faculty consisted of former course participants. All faculty followed a standard set of ultrasound and educational principles to guide the hands-on training of participants (supplementary Appendix 2).

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

 

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Portfolio development was a key aspect in overall POCUS competency for each participant. The hospital medicine department’s required portfolio files are presented in the Figure, with standards coinciding with the quality assurance grading rubric as developed by the POCUS committee at Regions Hospital and described by Mathews and Zwank.²⁸ Images taken with real patients were submitted without patient identifiers to a shared online portal. Faculty provided regular cycling feedback by entering the status of submission (accepted or declined) and specific comments on images and interpretations. Learners worked off of the feedback, practiced their skills, and resubmitted files. An image was considered acceptable if it met criteria of depth, axis, and gain and showed the required organ. Participants could use the same patient for different views but could not use the same patient for multiple images of the same view.

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

 

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For hospitalists who completed both the 3-day and 1-day courses, the change in course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, was assessed by summarizing the change from post 3-day metrics to pre 1-day metrics (Table 2). The differences between these 2 assessments were evaluated by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level. Changes in skills score from post 3-day assessment to pre 1-day assessment were also compared for hospitalists completing any of the portfolio and those completing none, and for hospitalists attending any monthly scanning sessions and those who did not attend any, by using analysis of variance and Scheffe tests.

Multiple linear regression was performed with the change in skills assessment score from postcompletion of the 3-day course to precompletion of the 1-day course as the dependent variable. Hospitalists were split into 2 age groups (30-39 and 40-49) for the purpose of this analysis. The percent of monthly scanning sessions attended, age category, timing of 1-day course, and percent portfolio were assessed as possible predictors of the skills score by using simple linear regression with a P = .05 cutoff. A final model was chosen based on predictors significant in simple linear regression and included the percent of the portfolio completed and attendance of monthly scanning sessions.

RESULTS

Demographics

Of the 56 3-day course participants, 53 had complete data (Table 3). Three participants with incomplete data completed most of the course but left prior to postcourse assessments and were excluded from the analysis. Twenty-three hospitalists also completed the 1-day in-person course. Seven hospitalists completed the 1-day course 3 months after the initial course, 8 completed it at 6 months, and 8 completed it at 12 months. Completed portfolios required 164 approved video images. Fifteen of the 23 hospitalists at the 1-day course have started and are working towards completion of the online portfolio, while 9 of the 23 participated in the monthly scanning sessions.

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

 

DISCUSSION

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

CONCLUSION

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

METHODS

From 2014 to 2016, our research team recruited inpatients who were being treated in the Pennsylvania State University Hershey Cancer Institute in Hershey, Pennsylvania. Patients were eligible if they were English speaking, over the age of 19, not cognitively impaired, and had been admitted for cancer-related treatments that required at least a 3-day inpatient stay. During recruitment, patients were told that the study was on patient care and room décor, and thus those who were not being given artwork did not know about the artwork option. By using a permuted block design with mixed block size, we randomly assigned consenting patients to 1 of the following 3 groups: (1) those who chose the painting displayed in their rooms, (2) those whose painting was randomly selected, and (3) those with no painting in their rooms, only white boards in their line of vision (see Figure 1). All paintings were created by artists in central Pennsylvania and reproduced as high-quality digital prints for the study, costing approximately $90 apiece. Members of the research team visited patients in the designated rooms 3 times during their stay—within 24 hours of being admitted, within 24 to 48 hours of the first visit, and within 24 to 48 hours of the second visit—with each visit lasting from 5 to 10 minutes. Patients who were given the opportunity to select art for their rooms were shown a catalogue of approximately 20 available paintings from which to choose a desired print; as with the group whose paintings were randomly selected for them, patients who made a choice had a print immediately hung in their room by members of the research team for the entirety of their inpatient stay.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

 

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

RESULTS

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

As shown in Figure 2, participants who had art in their rooms generally had more positive impressions of the hospital environment than those who did not. For 6 of the 17 paired attributes, participants with artwork were significantly more likely to choose the positive attribute—specifically, such patients indicated their rooms were more interesting, colorful, pleasant, attractive, ornate, and tasteful. With regard to the other attributes, though not reaching levels of significance, the overall pattern clearly reflected a more positive impression of rooms with art than without it.

DISCUSSION

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

 

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

METHODS

From 2014 to 2016, our research team recruited inpatients who were being treated in the Pennsylvania State University Hershey Cancer Institute in Hershey, Pennsylvania. Patients were eligible if they were English speaking, over the age of 19, not cognitively impaired, and had been admitted for cancer-related treatments that required at least a 3-day inpatient stay. During recruitment, patients were told that the study was on patient care and room décor, and thus those who were not being given artwork did not know about the artwork option. By using a permuted block design with mixed block size, we randomly assigned consenting patients to 1 of the following 3 groups: (1) those who chose the painting displayed in their rooms, (2) those whose painting was randomly selected, and (3) those with no painting in their rooms, only white boards in their line of vision (see Figure 1). All paintings were created by artists in central Pennsylvania and reproduced as high-quality digital prints for the study, costing approximately $90 apiece. Members of the research team visited patients in the designated rooms 3 times during their stay—within 24 hours of being admitted, within 24 to 48 hours of the first visit, and within 24 to 48 hours of the second visit—with each visit lasting from 5 to 10 minutes. Patients who were given the opportunity to select art for their rooms were shown a catalogue of approximately 20 available paintings from which to choose a desired print; as with the group whose paintings were randomly selected for them, patients who made a choice had a print immediately hung in their room by members of the research team for the entirety of their inpatient stay.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

 

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

RESULTS

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

As shown in Figure 2, participants who had art in their rooms generally had more positive impressions of the hospital environment than those who did not. For 6 of the 17 paired attributes, participants with artwork were significantly more likely to choose the positive attribute—specifically, such patients indicated their rooms were more interesting, colorful, pleasant, attractive, ornate, and tasteful. With regard to the other attributes, though not reaching levels of significance, the overall pattern clearly reflected a more positive impression of rooms with art than without it.

DISCUSSION

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

 

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

METHODS

From 2014 to 2016, our research team recruited inpatients who were being treated in the Pennsylvania State University Hershey Cancer Institute in Hershey, Pennsylvania. Patients were eligible if they were English speaking, over the age of 19, not cognitively impaired, and had been admitted for cancer-related treatments that required at least a 3-day inpatient stay. During recruitment, patients were told that the study was on patient care and room décor, and thus those who were not being given artwork did not know about the artwork option. By using a permuted block design with mixed block size, we randomly assigned consenting patients to 1 of the following 3 groups: (1) those who chose the painting displayed in their rooms, (2) those whose painting was randomly selected, and (3) those with no painting in their rooms, only white boards in their line of vision (see Figure 1). All paintings were created by artists in central Pennsylvania and reproduced as high-quality digital prints for the study, costing approximately $90 apiece. Members of the research team visited patients in the designated rooms 3 times during their stay—within 24 hours of being admitted, within 24 to 48 hours of the first visit, and within 24 to 48 hours of the second visit—with each visit lasting from 5 to 10 minutes. Patients who were given the opportunity to select art for their rooms were shown a catalogue of approximately 20 available paintings from which to choose a desired print; as with the group whose paintings were randomly selected for them, patients who made a choice had a print immediately hung in their room by members of the research team for the entirety of their inpatient stay.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

 

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

RESULTS

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

As shown in Figure 2, participants who had art in their rooms generally had more positive impressions of the hospital environment than those who did not. For 6 of the 17 paired attributes, participants with artwork were significantly more likely to choose the positive attribute—specifically, such patients indicated their rooms were more interesting, colorful, pleasant, attractive, ornate, and tasteful. With regard to the other attributes, though not reaching levels of significance, the overall pattern clearly reflected a more positive impression of rooms with art than without it.

DISCUSSION

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

 

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

METHODS

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

 

RESULTS

In 2015, 2334 general hospitals completed all questions of interest in both surveys. Among respondents, 65.8% used individual provider performance reports. Individual provider performance use increased by 3.3% each year from 2013 to 2015 (P = .006; Figure).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

DISCUSSION

We found that a large and increasing proportion of US hospitals reported using electronic data to measure individual provider performance. Hospitals that reported IPP use tended to be larger and have higher expenditures than hospitals that did not use IPPs. Adjusting for other hospital characteristics, participation in a bundled payment program was associated with greater odds of using IPPs. To our knowledge, our study is the first population-level analysis of IPP use by US hospitals.

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

 

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

METHODS

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

 

RESULTS

In 2015, 2334 general hospitals completed all questions of interest in both surveys. Among respondents, 65.8% used individual provider performance reports. Individual provider performance use increased by 3.3% each year from 2013 to 2015 (P = .006; Figure).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

DISCUSSION

We found that a large and increasing proportion of US hospitals reported using electronic data to measure individual provider performance. Hospitals that reported IPP use tended to be larger and have higher expenditures than hospitals that did not use IPPs. Adjusting for other hospital characteristics, participation in a bundled payment program was associated with greater odds of using IPPs. To our knowledge, our study is the first population-level analysis of IPP use by US hospitals.

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

 

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

METHODS

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

 

RESULTS

In 2015, 2334 general hospitals completed all questions of interest in both surveys. Among respondents, 65.8% used individual provider performance reports. Individual provider performance use increased by 3.3% each year from 2013 to 2015 (P = .006; Figure).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

DISCUSSION

We found that a large and increasing proportion of US hospitals reported using electronic data to measure individual provider performance. Hospitals that reported IPP use tended to be larger and have higher expenditures than hospitals that did not use IPPs. Adjusting for other hospital characteristics, participation in a bundled payment program was associated with greater odds of using IPPs. To our knowledge, our study is the first population-level analysis of IPP use by US hospitals.

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

 

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

METHODS

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

RESULTS

Respondents and Programs

Of the estimated 606 PS in the AAP SoSu, 291 (49%) US-based surgeons (PS) responded with 251 (41%) sufficiently completed surveys (Table). The initial and completed survey response rate for pediatric OS through the POSNA listserv was 58% and 48% (340/706), respectively. These respondents represented 185 unique PS programs and 212/340 (62%) unique OS programs in the US (supplemental Appendix B).

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

 

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Both PS and OS programs with hospitalist involvement reported that surgical teams were either primarily responsible for, or shared with the hospitalist, most aspects of patient care, including medication dosing, nutrition, and fluids (Figure). PH management of antibiotic and nonsurgical comorbidities was higher for OS programs than PS programs.

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

DISCUSSION

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

 

CONCLUSION

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

METHODS

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

RESULTS

Respondents and Programs

Of the estimated 606 PS in the AAP SoSu, 291 (49%) US-based surgeons (PS) responded with 251 (41%) sufficiently completed surveys (Table). The initial and completed survey response rate for pediatric OS through the POSNA listserv was 58% and 48% (340/706), respectively. These respondents represented 185 unique PS programs and 212/340 (62%) unique OS programs in the US (supplemental Appendix B).

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

 

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Both PS and OS programs with hospitalist involvement reported that surgical teams were either primarily responsible for, or shared with the hospitalist, most aspects of patient care, including medication dosing, nutrition, and fluids (Figure). PH management of antibiotic and nonsurgical comorbidities was higher for OS programs than PS programs.

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

DISCUSSION

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

 

CONCLUSION

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

METHODS

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

RESULTS

Respondents and Programs

Of the estimated 606 PS in the AAP SoSu, 291 (49%) US-based surgeons (PS) responded with 251 (41%) sufficiently completed surveys (Table). The initial and completed survey response rate for pediatric OS through the POSNA listserv was 58% and 48% (340/706), respectively. These respondents represented 185 unique PS programs and 212/340 (62%) unique OS programs in the US (supplemental Appendix B).

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

 

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Both PS and OS programs with hospitalist involvement reported that surgical teams were either primarily responsible for, or shared with the hospitalist, most aspects of patient care, including medication dosing, nutrition, and fluids (Figure). PH management of antibiotic and nonsurgical comorbidities was higher for OS programs than PS programs.

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

DISCUSSION

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

 

CONCLUSION

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

METHODS

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

We evaluated multiple patient characteristics for each admission that were accessible in the EHR at the time of hospital admission including patient comorbidities, medication count, and number of notes and discharge summaries. The Charlson Comorbidity Index (CCI) Deyo version was used to score all patients based on the EHR’s active problem list at the time of admission.^8,9 The CCI is a validated score created by categorizing comorbidities using International Classification of Diseases, Ninth and Tenth Revisions.⁸ Higher CCI scores predict increased mortality and resource usage. For each admission, we also counted the number of active medications, the number of prior discharge summaries, and the total number of notes available in the EHR at the time of patient admission. Patient admissions were grouped by calendar year, the mean numbers of active medications, prior discharge summaries, and total available notes per patient during each year were calculated (Table). Data comparisons were completed between 2003 and 2011 as well as between 2011 and 2015; median data are also provided for these years (Table). These years were chosen based on the years of the duty hour changes as well as comparing a not brand new, but still immature EHR (2003), a mature EHR (2011), and the most recent available data (2015).

RESULTS

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

 

DISCUSSION

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

CONCLUSION

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

 

Acknowledgments

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

METHODS

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

We evaluated multiple patient characteristics for each admission that were accessible in the EHR at the time of hospital admission including patient comorbidities, medication count, and number of notes and discharge summaries. The Charlson Comorbidity Index (CCI) Deyo version was used to score all patients based on the EHR’s active problem list at the time of admission.^8,9 The CCI is a validated score created by categorizing comorbidities using International Classification of Diseases, Ninth and Tenth Revisions.⁸ Higher CCI scores predict increased mortality and resource usage. For each admission, we also counted the number of active medications, the number of prior discharge summaries, and the total number of notes available in the EHR at the time of patient admission. Patient admissions were grouped by calendar year, the mean numbers of active medications, prior discharge summaries, and total available notes per patient during each year were calculated (Table). Data comparisons were completed between 2003 and 2011 as well as between 2011 and 2015; median data are also provided for these years (Table). These years were chosen based on the years of the duty hour changes as well as comparing a not brand new, but still immature EHR (2003), a mature EHR (2011), and the most recent available data (2015).

RESULTS

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

 

DISCUSSION

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

CONCLUSION

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

 

Acknowledgments

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

METHODS

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

We evaluated multiple patient characteristics for each admission that were accessible in the EHR at the time of hospital admission including patient comorbidities, medication count, and number of notes and discharge summaries. The Charlson Comorbidity Index (CCI) Deyo version was used to score all patients based on the EHR’s active problem list at the time of admission.^8,9 The CCI is a validated score created by categorizing comorbidities using International Classification of Diseases, Ninth and Tenth Revisions.⁸ Higher CCI scores predict increased mortality and resource usage. For each admission, we also counted the number of active medications, the number of prior discharge summaries, and the total number of notes available in the EHR at the time of patient admission. Patient admissions were grouped by calendar year, the mean numbers of active medications, prior discharge summaries, and total available notes per patient during each year were calculated (Table). Data comparisons were completed between 2003 and 2011 as well as between 2011 and 2015; median data are also provided for these years (Table). These years were chosen based on the years of the duty hour changes as well as comparing a not brand new, but still immature EHR (2003), a mature EHR (2011), and the most recent available data (2015).

RESULTS

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

 

DISCUSSION

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

CONCLUSION

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

 

Acknowledgments

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.