Lisa Shieh, MD, PhD

Setting

This study took place at Stanford Hospital and Clinics, a teaching academic center in Stanford, California. Stanford Hospital is a 444‐bed, urban medical center with 114 telemetry intermediate ICU beds, and 66 ICU beds. The 264 medicalsurgical beds lack telemetry monitoring, which can only be completed in the intermediate and full ICU. All patients on telemetry units receive both cardiac monitoring and increased nursing ratios. Transfer orders are placed in the electronic medical record to shift patients between care levels. Bed control attempts to transfer patients as soon as an open bed in the appropriate care level exists.

The study included all 5 housestaff inpatient general internal medicine wards teams (which excludes cardiology, pulmonary hypertension, hematology, oncology, and post‐transplant patients). Hospitalists and nonhospitalists attend on the wards for 1‐ to 2‐week blocks. Teaching teams are staffed by 1 to 2 medical students, 2 interns, 1 resident, and 1 attending. The university institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Participants

Ten full‐ and part‐time hospitalist physicians participated in the standardized telemetry teaching. Fifty‐six of the approximately 80 medical students and housestaff on hospitalists' teams completed the educational evaluation. Both hospitalist and nonhospitalist teams participated in daily multidisciplinary rounds, focusing on barriers to discharge including telemetry use. Twelve nonhospitalists served on the wards during the intervention period. Hospitalists covered 72% of the internal medicine wards during the intervention period.

Study Design

We investigated the impact of a multipronged approach to guide telemetry usage from January 2013 to August 2013 (intervention period).

Statistical Analysis of Clinical Outcome Measures

Continuous outcomes were tested using 2‐tailed t tests. Comparison of continuous outcome included differences in telemetry and nontelemetry LOS and CMI. Pairwise comparisons were made for various time periods. A P value of <0.05 was considered statistically significant. Statistical analyses were performed using Stata 12.0 software (StataCorp, College Station, TX).

Clinical and Value Outcomes

Education Outcomes

Out of the 56 participants completing the education module and survey, 28.6% were medical students, 53.6% were interns, 12.5% were second‐year residents, and 5.4% were third‐year residents. Several findings were seen at baseline via pretest. In evaluating patterns of current telemetry use, 32.2% of participants reported evaluating the necessity of telemetry for patients on admission only, 26.3% during transitions of care, 5.1% after discharge plans were cemented, 33.1% on a daily basis, and 3.4% rarely. When asked which member of the care team was most likely to encourage use of appropriate telemetry, 20.8% identified another resident, 13.9% nursing, 37.5% attending physician, 20.8% self, 4.2% the team as a whole, and 2.8% as not any.

Figure 1 shows premodule results regarding the trainees perceived percentage of patient encounters during which a participant's team discussed their patient's need for telemetry.

Figure 1

In assessing perception of current telemetry utilization, 1.8% of participants thought 0% to 10% of patients were currently on telemetry, 19.6% thought 11% to 20%, 42.9% thought 21% to 31%, 30.4% thought 31% to 40%, and 3.6% thought 41% to 50%.

Two areas were assessed at both baseline and after the intervention: knowledge of indications of telemetry use and cost related to telemetry use. We saw increased awareness of cost‐saving actions. To assess current knowledge of the indications of proper telemetry use according to American Heart Association guidelines, participants were presented with a list of 5 patients with different clinical indications for telemetry use and asked which patient required telemetry the most. Of the participants, 54.5% identified the correct answer in the pretest and 61.8% identified the correct answer in the post‐test. To assess knowledge of the costs of telemetry relative to other patient care, participants were presented with a patient case and asked to identify the most and least cost‐saving actions to safely care for the patient. When asked to identify the most cost‐saving action, 20.3% identified the correct answer in the pretest and 61.0% identified the correct answer in the post‐test. Of those who answered incorrectly in the pretest, 51.1% answered correctly in the post‐test (P=0.002). When asked to identify the least cost‐saving action, 23.7% identified the correct answer in the pretest and 50.9% identified the correct answer in the posttest. Of those who answered incorrectly in the pretest, 60.0% answered correctly in the post‐test (P=0.003).

In the post‐test, when asked about the importance of appropriate telemetry usage in providing cost‐conscious care and assuring appropriate hospital resource management, 76.8% of participants found the need very important, 21.4% somewhat important, and 1.8% as not applicable. The most commonly perceived barriers impeding discontinuation of telemetry, as reported by participants via post‐test, were nursing desires and time. Figure 2 shows all perceived barriers.

Figure 2

Setting

This study took place at Stanford Hospital and Clinics, a teaching academic center in Stanford, California. Stanford Hospital is a 444‐bed, urban medical center with 114 telemetry intermediate ICU beds, and 66 ICU beds. The 264 medicalsurgical beds lack telemetry monitoring, which can only be completed in the intermediate and full ICU. All patients on telemetry units receive both cardiac monitoring and increased nursing ratios. Transfer orders are placed in the electronic medical record to shift patients between care levels. Bed control attempts to transfer patients as soon as an open bed in the appropriate care level exists.

The study included all 5 housestaff inpatient general internal medicine wards teams (which excludes cardiology, pulmonary hypertension, hematology, oncology, and post‐transplant patients). Hospitalists and nonhospitalists attend on the wards for 1‐ to 2‐week blocks. Teaching teams are staffed by 1 to 2 medical students, 2 interns, 1 resident, and 1 attending. The university institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Participants

Ten full‐ and part‐time hospitalist physicians participated in the standardized telemetry teaching. Fifty‐six of the approximately 80 medical students and housestaff on hospitalists' teams completed the educational evaluation. Both hospitalist and nonhospitalist teams participated in daily multidisciplinary rounds, focusing on barriers to discharge including telemetry use. Twelve nonhospitalists served on the wards during the intervention period. Hospitalists covered 72% of the internal medicine wards during the intervention period.

Study Design

We investigated the impact of a multipronged approach to guide telemetry usage from January 2013 to August 2013 (intervention period).

Statistical Analysis of Clinical Outcome Measures

Continuous outcomes were tested using 2‐tailed t tests. Comparison of continuous outcome included differences in telemetry and nontelemetry LOS and CMI. Pairwise comparisons were made for various time periods. A P value of <0.05 was considered statistically significant. Statistical analyses were performed using Stata 12.0 software (StataCorp, College Station, TX).

Clinical and Value Outcomes

Education Outcomes

Out of the 56 participants completing the education module and survey, 28.6% were medical students, 53.6% were interns, 12.5% were second‐year residents, and 5.4% were third‐year residents. Several findings were seen at baseline via pretest. In evaluating patterns of current telemetry use, 32.2% of participants reported evaluating the necessity of telemetry for patients on admission only, 26.3% during transitions of care, 5.1% after discharge plans were cemented, 33.1% on a daily basis, and 3.4% rarely. When asked which member of the care team was most likely to encourage use of appropriate telemetry, 20.8% identified another resident, 13.9% nursing, 37.5% attending physician, 20.8% self, 4.2% the team as a whole, and 2.8% as not any.

Figure 1 shows premodule results regarding the trainees perceived percentage of patient encounters during which a participant's team discussed their patient's need for telemetry.

Figure 1

In assessing perception of current telemetry utilization, 1.8% of participants thought 0% to 10% of patients were currently on telemetry, 19.6% thought 11% to 20%, 42.9% thought 21% to 31%, 30.4% thought 31% to 40%, and 3.6% thought 41% to 50%.

Two areas were assessed at both baseline and after the intervention: knowledge of indications of telemetry use and cost related to telemetry use. We saw increased awareness of cost‐saving actions. To assess current knowledge of the indications of proper telemetry use according to American Heart Association guidelines, participants were presented with a list of 5 patients with different clinical indications for telemetry use and asked which patient required telemetry the most. Of the participants, 54.5% identified the correct answer in the pretest and 61.8% identified the correct answer in the post‐test. To assess knowledge of the costs of telemetry relative to other patient care, participants were presented with a patient case and asked to identify the most and least cost‐saving actions to safely care for the patient. When asked to identify the most cost‐saving action, 20.3% identified the correct answer in the pretest and 61.0% identified the correct answer in the post‐test. Of those who answered incorrectly in the pretest, 51.1% answered correctly in the post‐test (P=0.002). When asked to identify the least cost‐saving action, 23.7% identified the correct answer in the pretest and 50.9% identified the correct answer in the posttest. Of those who answered incorrectly in the pretest, 60.0% answered correctly in the post‐test (P=0.003).

In the post‐test, when asked about the importance of appropriate telemetry usage in providing cost‐conscious care and assuring appropriate hospital resource management, 76.8% of participants found the need very important, 21.4% somewhat important, and 1.8% as not applicable. The most commonly perceived barriers impeding discontinuation of telemetry, as reported by participants via post‐test, were nursing desires and time. Figure 2 shows all perceived barriers.

Figure 2

Setting

This study took place at Stanford Hospital and Clinics, a teaching academic center in Stanford, California. Stanford Hospital is a 444‐bed, urban medical center with 114 telemetry intermediate ICU beds, and 66 ICU beds. The 264 medicalsurgical beds lack telemetry monitoring, which can only be completed in the intermediate and full ICU. All patients on telemetry units receive both cardiac monitoring and increased nursing ratios. Transfer orders are placed in the electronic medical record to shift patients between care levels. Bed control attempts to transfer patients as soon as an open bed in the appropriate care level exists.

The study included all 5 housestaff inpatient general internal medicine wards teams (which excludes cardiology, pulmonary hypertension, hematology, oncology, and post‐transplant patients). Hospitalists and nonhospitalists attend on the wards for 1‐ to 2‐week blocks. Teaching teams are staffed by 1 to 2 medical students, 2 interns, 1 resident, and 1 attending. The university institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Participants

Ten full‐ and part‐time hospitalist physicians participated in the standardized telemetry teaching. Fifty‐six of the approximately 80 medical students and housestaff on hospitalists' teams completed the educational evaluation. Both hospitalist and nonhospitalist teams participated in daily multidisciplinary rounds, focusing on barriers to discharge including telemetry use. Twelve nonhospitalists served on the wards during the intervention period. Hospitalists covered 72% of the internal medicine wards during the intervention period.

Study Design

We investigated the impact of a multipronged approach to guide telemetry usage from January 2013 to August 2013 (intervention period).

Statistical Analysis of Clinical Outcome Measures

Continuous outcomes were tested using 2‐tailed t tests. Comparison of continuous outcome included differences in telemetry and nontelemetry LOS and CMI. Pairwise comparisons were made for various time periods. A P value of <0.05 was considered statistically significant. Statistical analyses were performed using Stata 12.0 software (StataCorp, College Station, TX).

Clinical and Value Outcomes

Education Outcomes

Out of the 56 participants completing the education module and survey, 28.6% were medical students, 53.6% were interns, 12.5% were second‐year residents, and 5.4% were third‐year residents. Several findings were seen at baseline via pretest. In evaluating patterns of current telemetry use, 32.2% of participants reported evaluating the necessity of telemetry for patients on admission only, 26.3% during transitions of care, 5.1% after discharge plans were cemented, 33.1% on a daily basis, and 3.4% rarely. When asked which member of the care team was most likely to encourage use of appropriate telemetry, 20.8% identified another resident, 13.9% nursing, 37.5% attending physician, 20.8% self, 4.2% the team as a whole, and 2.8% as not any.

Figure 1 shows premodule results regarding the trainees perceived percentage of patient encounters during which a participant's team discussed their patient's need for telemetry.

Figure 1

In assessing perception of current telemetry utilization, 1.8% of participants thought 0% to 10% of patients were currently on telemetry, 19.6% thought 11% to 20%, 42.9% thought 21% to 31%, 30.4% thought 31% to 40%, and 3.6% thought 41% to 50%.

Two areas were assessed at both baseline and after the intervention: knowledge of indications of telemetry use and cost related to telemetry use. We saw increased awareness of cost‐saving actions. To assess current knowledge of the indications of proper telemetry use according to American Heart Association guidelines, participants were presented with a list of 5 patients with different clinical indications for telemetry use and asked which patient required telemetry the most. Of the participants, 54.5% identified the correct answer in the pretest and 61.8% identified the correct answer in the post‐test. To assess knowledge of the costs of telemetry relative to other patient care, participants were presented with a patient case and asked to identify the most and least cost‐saving actions to safely care for the patient. When asked to identify the most cost‐saving action, 20.3% identified the correct answer in the pretest and 61.0% identified the correct answer in the post‐test. Of those who answered incorrectly in the pretest, 51.1% answered correctly in the post‐test (P=0.002). When asked to identify the least cost‐saving action, 23.7% identified the correct answer in the pretest and 50.9% identified the correct answer in the posttest. Of those who answered incorrectly in the pretest, 60.0% answered correctly in the post‐test (P=0.003).

In the post‐test, when asked about the importance of appropriate telemetry usage in providing cost‐conscious care and assuring appropriate hospital resource management, 76.8% of participants found the need very important, 21.4% somewhat important, and 1.8% as not applicable. The most commonly perceived barriers impeding discontinuation of telemetry, as reported by participants via post‐test, were nursing desires and time. Figure 2 shows all perceived barriers.

Figure 2

Intended Improvement/Study Question

Our team set out to plan and implement a set of multifaceted interventions within 90 days. The short‐term goal was a 50% reduction in the CVC IAP and all‐cause IAP rate within 18 months, and the long‐term goal was sustained reduction of CVC IAP and all‐cause IAP rate.

Setting

Stanford University Medical Center is an academic medical center with 465 beds and over 25,000 inpatient admissions per year, providing both general acute care services and tertiary medical care. Residents perform CVC bedside procedures when central venous access is needed, in the intensive care unit (ICU), operating room (OR), and inpatient units. Prior to this project, ultrasound equipment was only available in the emergency department (ED) and ICUs. There was no formal CVC procedure supervision policy, CVC training curriculum, and procedure note templates for documentation of CVC insertion.

Planning the Interventions

A multidisciplinary quality‐improvement team met weekly during the 90‐day design period from January 2007 to March 2007. Our team included representatives from the departments of medicine, anesthesia and critical care, surgery, nursing, and emergency medicine. We also partnered with our institution's clinical and administrative leaders, experts in simulation, and the hospital quality department.

We hypothesized that a standardized set of education and training interventions promoting ultrasound‐guided IJ CVC insertion as the method of choice at our hospital would significantly reduce our rate of CVC‐associated IAP. Our multifaceted intervention included: (1) clinical and documentation standards based on evidence, (2) cognitive aids, (3) simulation training, (4) purchase and deployment of ultrasound equipment, and (5) feedback to clinical services.

Our team followed the define, measure, analyze, improve, control (DMAIC) framework.[15] We set interval goals with target completion dates throughout the 90‐day period, identified owners of each goal, and tracked progress with a shared spreadsheet.

In the 90‐day intervention, we accomplished the following: (1) conducted root‐cause analysis of IAP cases for fiscal year 2006, (2) created clinical and documentation standards around CVC placement, (3) created cognitive aids and procedure note templates, (4) developed simulation training courses, and (5) requested purchase of additional ultrasound equipment.

Data Collection

To evaluate our progress in reducing the rates of IAP, we tracked the incidence of IAP using UHC and AHRQ PSI methodology. In collaboration with our hospital's quality department, we manually reviewed every PSI‐identified case of IAP. This review has focused on identifying whether or not pneumothorax actually occurred, and whether it was associated with CVC insertion. For those associated with CVC, data were collected for patient location and service, the procedure site, whether ultrasound was used, whether a chest tube was required, and the final disposition of the patient.

Demographic data (age, gender, case mix index [CMI]) shown in Table 1 were obtained through MIDAS+ Solutions (Tucson, Arizona), a proprietary database that contains healthcare management coded data. Total hospital CVC insertion rates were calculated using International Classification of Diseases, Ninth Revision (ICD‐9) coding for 38.93 and 38.97. ICU central lineassociated blood stream infections (CLABSI) data were obtained from internal collection by our infection control team. Number and location of CVCs placed in the ICU data were obtained from nursing flow sheets in our electronic medical record (EMR). Cost information was provided by our finance department using internal accounting.

The project granted a Notice of Determination of Approval from the Stanford Administrative Panels for the Protection of Human Subjects (institutional review board).

Methods of Evaluation/Analysis

For the purpose of this analysis, the preintervention period was defined as January 1, 2006 through December 31, 2006, our first year of IAP case review. We defined the intervention period as January 1, 2007 through December 31, 2007, during which we planned and implemented hospital‐wide standardization of CVC insertion practices and incorporated CVC insertion training simulation into resident orientation in July 2007. The postintervention period was defined as January 1, 2008 through December 31, 2014.

All statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX). [2] tests were used to determine statistical differences in pre‐ versus postintervention patient demographic data (age, gender, CMI), CVC insertion rates, and CLABSI rates. Because IAP is a rare event, a statistical process control g‐chart was created using QI Macros (KnowWare International, Inc., Denver, CO) to show the number of CVC procedures between IAP. [2] and Fisher exact tests were used to determine statistical differences in CVC anatomic location and use of ultrasound pre‐ and postintervention. A 2‐sided Z test to show a difference in proportions was used to determine statistical differences in CVC‐related IAP rate and all‐cause IAP rate pre‐ and postintervention.

Measuring Adherence to Intervention

Outcomes

The rate of CVC‐associated IAP was calculated as the total number of IAPs attributed to CVCs divided by the total number of CVCs inserted determined by ICD‐9 coding for 38.93 and 38.97. The total IAP rate was calculated as the total number of IAP/1000 discharges.

Interventions

Our team began the intervention in early 2007 with promotion of ultrasound‐guided IJ catheterization. Clinical exceptions included: (1) trauma or code situations where access to the neck is limited, (2) suspected or confirmed neck injuries, (3) presence of a tracheostomy, and (4) bilateral internal jugular sites unsuitable for catheterization.

Our hospital adopted new formal CVC insertion policies consistent with the above training and education efforts. All physicians were required to document CVC insertions using the template available in the EMR. To be certified to perform CVC insertion independently, trainee physicians were required to complete the simulation training and successfully place a minimum of 5 CVCs directly supervised by an already‐certified physician. This was consistent with the Accreditation Council for Graduate Medical Education suggested minimum requirement in 2007. In our critical care units, all CVC insertions must be supervised by an ICU fellow or attending.

To reinforce the on‐the‐ground work by our physician leaders, we created 2 education tools to embed best practices into our CVC insertion workflow. A checklist with best practices for CVC insertion that was distributed throughout the hospital via central line kits and educational flyers, and a CVC insertion procedure note template consistent with California Department of Public Health documentation requirements was made available in our EMR.

In June 2007, we integrated CVC insertion simulation training into procedure workshops required for all medicine, surgery, anesthesia, and emergency medicine trainees during their intern year. These workshops promoted ultrasound‐guided IJ catheterization and supporting evidence for the new IJ site preference. Training sessions were 2 to 3 hours, and included a demonstration of best‐practice CVC insertion, as well as training with simulation models supervised by an instructor using a standardized CVC checklist. These trainings used both the Blue Phantom human torso model (www.bluephantom.com; CAE Healthcare, Sarasota, FL), and the MOVAT Chicken Model (http://www.movat.com; MOVAT, Los Angeles, CA), a biological ultrasound‐compatible training device that simulates all of the steps of CVC insertion. Each medicine resident is required to complete a CVC insertion on the MOVAT model. In addition, the surgery residents have follow‐up testing with a checklist and by an observer within 1 to 2 months after their workshop.

Hospital administration provided funds to purchase ultrasound machines for patient units such as medicine, cardiology, ED, and ICU). A total of 4 Site‐Rite (Bard Access Systems, Inc., Salt Lake City, UT) ultrasounds were purchased in 2007. The hospital has continued to purchase ultrasound units yearly, and had 53 ultrasound units in 2014

Cases of IAP were continuously reviewed throughout the intervention period. Based on their higher CVC‐associated IAP rates, the ORs and catheterization lab were identified as having opportunities for improvement. In 2008, Hospital quality‐improvement leadership met with physician leaders in these areas to review their CVC‐related IAP data and to discuss strategies to reduce their IAP rates. These strategies included lessons learned from other services that had successfully decreased their IAP rates.

To sustain our gains, we continue to review all IAP through our coding quality, clinical documentation, quality reporting departments, and peer review. We have implemented other strategies to decrease IAP, such as the use of ultrasound guidance for bedside thoracentesis procedures, which became possible after the availability of more ultrasound devices. Training for ultrasound‐guided thoracentesis was done by our procedure‐team attending during supervision of residents.

Outcomes

Preintervention (January 1, 2006 to December 31, 2006)

There were a total of 26 cases of IAP in 2006. Of these, 15 (58%) were associated with CVC insertion (Figure 1). The single procedure associated with the largest proportion of IAP was SC CVC insertion (11 cases, 42% of all IAP cases). Eleven CVC‐associated IAPs were significant enough to require chest tube placement. Our hospital recorded a total of 1593 CVC insertions (ICD‐9 codes 38.93 and 38.97) in 2006.

Figure 1

Postintervention (January 1, 2008 to December 31, 2014)

There were a total of 80 cases of IAP over 7 years, of which 24 (30%) were associated with CVC insertion. Of these, 16 required chest tube placement. In the last 4 years of the postintervention period (20112014), there were only 5 cases of CVC‐associated IAP requiring chest tube placement (Figure 1). There were a total of 12,000 CVC insertions recorded over the same period.

We successfully met both our short‐ and long‐term goals. Our preintervention CVC‐associated IAP rate was 0.94%, and our post‐intervention rate during 2008 was 0.44%, a short‐term reduction of 53% (P=0.088). Our average postintervention CVC‐associated IAP rate for the years 2008 through 2014 was 0.13%, a significant long‐term reduction of 86% (P<0.0001) (Table 2). The decrease in CVC‐associated IAP rates occurred despite an older patient population (P<0.001) and a higher CMI (P<0.001) in postintervention patients who received a CVC (Table 1). Special cause variation corresponding to a change in our process is demonstrated in Figure 2. The preintervention average number of procedures between IAP was 114.8 and increased to 460.7 in the postintervention period.

Table 2.Summary of Outcomes

	Total CVC (n=95)		Subclavian (n=29)			Internal Jugular (n=58)					Femoral (n=8)
NOTE: Chart review of CVC placement in ICU during 2014 showing high compliance to intervention: US guided IJ placement, standard procedure note completion, and supervision, and significant long‐term reduction in CVC‐associated IAP rate and all‐cause IAP rate. A total of 100 patient charts were randomly selected and reviewed. Five patients receiving a PICC were not included in the analysis. All PICC lines were placed by trained registered nurses without supervision under US guidance. Of note, proportions of CVC types are not accurate representation; additional subclavian and internal jugular lines were randomly selected and reviewed. Abbreviations: CVC, central venous catheterization; IAP, iatrogenic pneumothorax; ICU intensive are unit; IJ, internal jugular; MICU, medical intensive care unit; PICC, peripherally inserted central catheter; SICU, surgical intensive care unit; US, ultrasound. *Surgical ICU includes all surgery primary teams including trauma, general surgery, cardiothoracic surgery, and neurosurgery.
Compliance to intervention
US guided	68.1%		20.7%			86.2%					100.0%
Procedure note completion	90.4%		93.1%			86.2%					100.0%
Supervision	70.2%		77.8%			73.1%					87.5%
Level of training
Resident	61.1%		58.6%			60.3%					75.0%
Fellow	25.3%		27.6%			24.1%					25.0%
Attending	4.2%		6.9%			3.4%					0.0%
Advance practitioner	3.2%		3.4%			3.4%					0.0%
Unknown	6.3%		3.4%			8.6%					0.0%
Supervisor type
Resident	3.0%		4.8%			2.6%					0.0%
Fellow	54.5%		33.3%			57.9%					100.0%
Attending	42.4%		61.9%			39.5%					0.0%
Location of CVC Placement	Internal Jugular (n=25)					Subclavian (n=25)
MICU	32.0%					32.0%
SICU*	40.0%					52.0%
Operating room	28.0%					16.0%
Average no. of attempts/procedure	1.4					1.5
Indications for subclavian insertion (n=25)
Trauma/surgical site	60.0%
Need for additional access	16.0%
Unsuccessful IJ placement	4.0%
Unclear	20.0%
Iatrogenic Pneumothorax Rate (20062014)	Year
	2006	2007		2008	2009		2010	2011	2012			2013	2014
% of CVC insertions associated with IAP	0.94	1.49		0.44	0.13		0.20	0.07	0.04			0.11	0.07
All‐cause IAP per 1,000 discharges	1.32	1.29		0.98	0.71		0.83	0.49	0.13			0.35	0.23
	Preintervention				Postintervention					P Value
CVC‐ associated IAP short term (2006 vs 2008)	0.94%				0.44%					0.088
CVC‐associated IAP long term (2006 vs 20082014)	0.94%				0.13%					<0.0001
All‐cause IAP per 1,000 discharges short term (2006 vs 2008)	1.32				0.98					<0.0001
All‐cause IAP per 1,000 discharges long term (2006 vs 2008‐14)	1.32				0.52					<0.0001

Figure 2

With the decrease in CVC‐associated IAP, we also saw a decrease in our all‐cause IAP rate per 1000 discharges from 1.32 in 2006 to 0.98 in 2008. This represents a 26% short‐term reduction (P<0.0001). We also saw a decrease in our all‐cause IAP rate per 1000 discharges to 0.52 from 2008 to 2014, representing a 61% long‐term reduction (P<0.0001). This decrease in all‐cause IAP postintervention occurred despite an older patient population (P<0.01) for all discharges. Our hospital is now in the highest performance UHC quartile for all‐cause IAP in 2012 to 2014.

After our multifaceted intervention in 2007, there was substantially more and consistent documentation of CVC procedure notes from less than 500 in 2006 to greater than 2000 in 2009. The distribution of CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed a decrease in the proportion of femoral lines from 15% to 11%, increase in IJ lines from 31% to 49%, and a decrease in SC from 54% to 40% (P=0.001). The distribution of IJ CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed an increase in the proportion of procedures with ultrasound documentation from 13% to 93% (P<0.001) (Figure 3).

Figure 3

In our ICU 2014 audit, the majority of CVC lines were placed by residents under supervision (>70%), and most used the standard CVC note template to document the procedure (90%). Of the total CVC approach, 66% were IJ and 4% were SC. Eighty‐six percent used ultrasound during IJ placement. The majority of SC insertions were placed in the surgical ICU and had clear indications (80%) for placement. Of those, 75% were due to trauma (limited access to neck) or surgery (interfering with surgical site) (Table 2).

Summary

This quality‐improvement intervention demonstrates that a multidisciplinary team can successfully implement a multifaceted intervention that sustainably reduces the rate of IAP complications from CVC placement and improves patient safety over 7 years. We found high compliance with our intervention, which included an increase in CVC notes and documentation of ultrasound guidance. There was also an increase in the IJ approach in our postintervention period. We showed statistically significant long‐term reductions in both CVC‐associated and all‐cause IAP rates. From 2011 to 2014, there were only 5 cases of CVC‐associated IAP requiring chest tube placement. Post hoc analysis showed a statistically significant decrease in CLABSI rates (P<0.0001) from a preintervention rate of 1.6 infections per 1000 central line days to postintervention average rate of 0.68 infections per 1000 central line days. This decrease may be related to the incorporation of wide sterile barrier techniques in our CVC training workshops, checklists, and template procedure notes.

A strength of this study is the sustained significant long‐term reduction in IAP. There are few data that exist to describe sustained interventions in this area. Sustainability was achieved by integrating our interventions into ongoing programs that already existed in the hospital; we incorporated our simulation training into the existing new resident orientation, increased the availability of existing ultrasound equipment, and continued our IAP chart review through coding quality with feedback to involved services. The procedure note template continues to be easily available in our EMR, and the SC approach to CVC placement is limited to select cases.

Based on a post hoc cost‐benefit analysis, the financial benefits of decreasing the rate of IAP outweigh the costs associated with implementation of this initiative. The purchase cost for a Site‐Rite (Bard Access Systems) ultrasound machine was $18,000. The cost of materials for 1 workshop is $5000 annually. Cases from the Nationwide Inpatient Sample that were flagged by this PSI had 7.0% excess mortality, 4.4 days of excess hospitalization, and approximately $18,000 in excess hospital charges.[17, 18] Based on these data, if we had continued at our preintervention rate of CVC‐associated IAP requiring chest tube placement, we would estimate 9 additional CVC‐associated IAPs requiring chest tube insertion per year. This would result in over $180,000 of additional costs annually. Based on an initial cost of $100,000 for 4 workshops and the necessary equipment, we would have realized our cost savings in less than 1 year postintervention. These are all approximate costs, and further detailed analysis is needed.

One challenge with this intervention is the culture change away from using the SC approach, and the concern from trainees of how they would learn to perform SC CVC if needed. We would suggest dedicated SC CVC ultrasound training for those services who may need to use this approach (eg, neuroanesthesia and trauma).

Interpretation/Relation to Other Evidence

The field of implementation science can help explain why some projects are successful and others fail. We can further dissect the success of this project using an implementation science model similar to that described by French et al.[19] French et al. describe 4 behavior‐change techniques. These steps include (1) who needs to do what differently, (2) which barriers and enablers need to be addressed, (3) which intervention component could overcome the barriers and enhance enablers, and (4) how can behavior change be measured and understood. Barriers included suboptimal skills of residents, low awareness of evidence‐based guidelines, and entrenched practices inconsistent with best evidence. There was also a belief that IJ lines were more likely to become infected. Targeted behaviors needing to be done differently were the choice of CVC placement site and insertion technique. Barriers to change were assessed by asking members of the project team to explore with members of their service what led them to do CVC lines without ultrasound guidance. Enhancements focused on information provision, simulation practice, and persuasive communication. Behavior change was measured by tracking the number of IAPs, site of CVC, and documentation of technique. Continuation of these interventions based on this theoretical framework drove maintenance of gains.

We completed our main intervention planning in 90 days, and met our short‐term goal on schedule. The Institute for Healthcare Improvement (IHI) advocates that such short timelines are efficient mechanisms for developing and acting on projects. Other institutions have reported on similar rapid‐cycle planning and short‐term goal setting[20]

Limitations

Our study captures the experience of a quality‐improvement team at a single academic center, and our results may not be generalizable to other institutions. Our chart review process only occurred once a case had been identified through AHRQ PSI methodology. It is possible that the PSI does not capture all cases of IAP, although we believe our coding department has a very rigorous process to look for all IAP evidence in the patient's record. We used administrative data to determine the number of hospital‐wide CVC procedures.

Our compliance data with interventions from STRIDE are based on looking for key words in procedure note documentation (so undocumented notes are not captured). To validate this, we performed a manual audit of our adherence to our intervention in 2014, and those data are consistent with the results from our STRIDE data.

Our study's observational design also cannot control for exogenous effects on physician practice relating to CVC insertion or the overall risk of IAP. Some of our decrease in complications may be from the increase in PICC line use. Nevertheless, our CVC‐associated IAP rate has decreased despite >6000 non‐PICC CVCs in our ICU over the past 5 years, and a rising CMI (18% increase in postintervention period) and older population of patients with CVC insertion (P<0.0001)

Intended Improvement/Study Question

Our team set out to plan and implement a set of multifaceted interventions within 90 days. The short‐term goal was a 50% reduction in the CVC IAP and all‐cause IAP rate within 18 months, and the long‐term goal was sustained reduction of CVC IAP and all‐cause IAP rate.

Setting

Stanford University Medical Center is an academic medical center with 465 beds and over 25,000 inpatient admissions per year, providing both general acute care services and tertiary medical care. Residents perform CVC bedside procedures when central venous access is needed, in the intensive care unit (ICU), operating room (OR), and inpatient units. Prior to this project, ultrasound equipment was only available in the emergency department (ED) and ICUs. There was no formal CVC procedure supervision policy, CVC training curriculum, and procedure note templates for documentation of CVC insertion.

Planning the Interventions

A multidisciplinary quality‐improvement team met weekly during the 90‐day design period from January 2007 to March 2007. Our team included representatives from the departments of medicine, anesthesia and critical care, surgery, nursing, and emergency medicine. We also partnered with our institution's clinical and administrative leaders, experts in simulation, and the hospital quality department.

We hypothesized that a standardized set of education and training interventions promoting ultrasound‐guided IJ CVC insertion as the method of choice at our hospital would significantly reduce our rate of CVC‐associated IAP. Our multifaceted intervention included: (1) clinical and documentation standards based on evidence, (2) cognitive aids, (3) simulation training, (4) purchase and deployment of ultrasound equipment, and (5) feedback to clinical services.

Our team followed the define, measure, analyze, improve, control (DMAIC) framework.[15] We set interval goals with target completion dates throughout the 90‐day period, identified owners of each goal, and tracked progress with a shared spreadsheet.

In the 90‐day intervention, we accomplished the following: (1) conducted root‐cause analysis of IAP cases for fiscal year 2006, (2) created clinical and documentation standards around CVC placement, (3) created cognitive aids and procedure note templates, (4) developed simulation training courses, and (5) requested purchase of additional ultrasound equipment.

Data Collection

To evaluate our progress in reducing the rates of IAP, we tracked the incidence of IAP using UHC and AHRQ PSI methodology. In collaboration with our hospital's quality department, we manually reviewed every PSI‐identified case of IAP. This review has focused on identifying whether or not pneumothorax actually occurred, and whether it was associated with CVC insertion. For those associated with CVC, data were collected for patient location and service, the procedure site, whether ultrasound was used, whether a chest tube was required, and the final disposition of the patient.

Demographic data (age, gender, case mix index [CMI]) shown in Table 1 were obtained through MIDAS+ Solutions (Tucson, Arizona), a proprietary database that contains healthcare management coded data. Total hospital CVC insertion rates were calculated using International Classification of Diseases, Ninth Revision (ICD‐9) coding for 38.93 and 38.97. ICU central lineassociated blood stream infections (CLABSI) data were obtained from internal collection by our infection control team. Number and location of CVCs placed in the ICU data were obtained from nursing flow sheets in our electronic medical record (EMR). Cost information was provided by our finance department using internal accounting.

The project granted a Notice of Determination of Approval from the Stanford Administrative Panels for the Protection of Human Subjects (institutional review board).

Methods of Evaluation/Analysis

For the purpose of this analysis, the preintervention period was defined as January 1, 2006 through December 31, 2006, our first year of IAP case review. We defined the intervention period as January 1, 2007 through December 31, 2007, during which we planned and implemented hospital‐wide standardization of CVC insertion practices and incorporated CVC insertion training simulation into resident orientation in July 2007. The postintervention period was defined as January 1, 2008 through December 31, 2014.

All statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX). [2] tests were used to determine statistical differences in pre‐ versus postintervention patient demographic data (age, gender, CMI), CVC insertion rates, and CLABSI rates. Because IAP is a rare event, a statistical process control g‐chart was created using QI Macros (KnowWare International, Inc., Denver, CO) to show the number of CVC procedures between IAP. [2] and Fisher exact tests were used to determine statistical differences in CVC anatomic location and use of ultrasound pre‐ and postintervention. A 2‐sided Z test to show a difference in proportions was used to determine statistical differences in CVC‐related IAP rate and all‐cause IAP rate pre‐ and postintervention.

Measuring Adherence to Intervention

Outcomes

The rate of CVC‐associated IAP was calculated as the total number of IAPs attributed to CVCs divided by the total number of CVCs inserted determined by ICD‐9 coding for 38.93 and 38.97. The total IAP rate was calculated as the total number of IAP/1000 discharges.

Interventions

Our team began the intervention in early 2007 with promotion of ultrasound‐guided IJ catheterization. Clinical exceptions included: (1) trauma or code situations where access to the neck is limited, (2) suspected or confirmed neck injuries, (3) presence of a tracheostomy, and (4) bilateral internal jugular sites unsuitable for catheterization.

Our hospital adopted new formal CVC insertion policies consistent with the above training and education efforts. All physicians were required to document CVC insertions using the template available in the EMR. To be certified to perform CVC insertion independently, trainee physicians were required to complete the simulation training and successfully place a minimum of 5 CVCs directly supervised by an already‐certified physician. This was consistent with the Accreditation Council for Graduate Medical Education suggested minimum requirement in 2007. In our critical care units, all CVC insertions must be supervised by an ICU fellow or attending.

To reinforce the on‐the‐ground work by our physician leaders, we created 2 education tools to embed best practices into our CVC insertion workflow. A checklist with best practices for CVC insertion that was distributed throughout the hospital via central line kits and educational flyers, and a CVC insertion procedure note template consistent with California Department of Public Health documentation requirements was made available in our EMR.

In June 2007, we integrated CVC insertion simulation training into procedure workshops required for all medicine, surgery, anesthesia, and emergency medicine trainees during their intern year. These workshops promoted ultrasound‐guided IJ catheterization and supporting evidence for the new IJ site preference. Training sessions were 2 to 3 hours, and included a demonstration of best‐practice CVC insertion, as well as training with simulation models supervised by an instructor using a standardized CVC checklist. These trainings used both the Blue Phantom human torso model (www.bluephantom.com; CAE Healthcare, Sarasota, FL), and the MOVAT Chicken Model (http://www.movat.com; MOVAT, Los Angeles, CA), a biological ultrasound‐compatible training device that simulates all of the steps of CVC insertion. Each medicine resident is required to complete a CVC insertion on the MOVAT model. In addition, the surgery residents have follow‐up testing with a checklist and by an observer within 1 to 2 months after their workshop.

Hospital administration provided funds to purchase ultrasound machines for patient units such as medicine, cardiology, ED, and ICU). A total of 4 Site‐Rite (Bard Access Systems, Inc., Salt Lake City, UT) ultrasounds were purchased in 2007. The hospital has continued to purchase ultrasound units yearly, and had 53 ultrasound units in 2014

Cases of IAP were continuously reviewed throughout the intervention period. Based on their higher CVC‐associated IAP rates, the ORs and catheterization lab were identified as having opportunities for improvement. In 2008, Hospital quality‐improvement leadership met with physician leaders in these areas to review their CVC‐related IAP data and to discuss strategies to reduce their IAP rates. These strategies included lessons learned from other services that had successfully decreased their IAP rates.

To sustain our gains, we continue to review all IAP through our coding quality, clinical documentation, quality reporting departments, and peer review. We have implemented other strategies to decrease IAP, such as the use of ultrasound guidance for bedside thoracentesis procedures, which became possible after the availability of more ultrasound devices. Training for ultrasound‐guided thoracentesis was done by our procedure‐team attending during supervision of residents.

Outcomes

Preintervention (January 1, 2006 to December 31, 2006)

There were a total of 26 cases of IAP in 2006. Of these, 15 (58%) were associated with CVC insertion (Figure 1). The single procedure associated with the largest proportion of IAP was SC CVC insertion (11 cases, 42% of all IAP cases). Eleven CVC‐associated IAPs were significant enough to require chest tube placement. Our hospital recorded a total of 1593 CVC insertions (ICD‐9 codes 38.93 and 38.97) in 2006.

Figure 1

Postintervention (January 1, 2008 to December 31, 2014)

There were a total of 80 cases of IAP over 7 years, of which 24 (30%) were associated with CVC insertion. Of these, 16 required chest tube placement. In the last 4 years of the postintervention period (20112014), there were only 5 cases of CVC‐associated IAP requiring chest tube placement (Figure 1). There were a total of 12,000 CVC insertions recorded over the same period.

We successfully met both our short‐ and long‐term goals. Our preintervention CVC‐associated IAP rate was 0.94%, and our post‐intervention rate during 2008 was 0.44%, a short‐term reduction of 53% (P=0.088). Our average postintervention CVC‐associated IAP rate for the years 2008 through 2014 was 0.13%, a significant long‐term reduction of 86% (P<0.0001) (Table 2). The decrease in CVC‐associated IAP rates occurred despite an older patient population (P<0.001) and a higher CMI (P<0.001) in postintervention patients who received a CVC (Table 1). Special cause variation corresponding to a change in our process is demonstrated in Figure 2. The preintervention average number of procedures between IAP was 114.8 and increased to 460.7 in the postintervention period.

Table 2.Summary of Outcomes

	Total CVC (n=95)		Subclavian (n=29)			Internal Jugular (n=58)					Femoral (n=8)
NOTE: Chart review of CVC placement in ICU during 2014 showing high compliance to intervention: US guided IJ placement, standard procedure note completion, and supervision, and significant long‐term reduction in CVC‐associated IAP rate and all‐cause IAP rate. A total of 100 patient charts were randomly selected and reviewed. Five patients receiving a PICC were not included in the analysis. All PICC lines were placed by trained registered nurses without supervision under US guidance. Of note, proportions of CVC types are not accurate representation; additional subclavian and internal jugular lines were randomly selected and reviewed. Abbreviations: CVC, central venous catheterization; IAP, iatrogenic pneumothorax; ICU intensive are unit; IJ, internal jugular; MICU, medical intensive care unit; PICC, peripherally inserted central catheter; SICU, surgical intensive care unit; US, ultrasound. *Surgical ICU includes all surgery primary teams including trauma, general surgery, cardiothoracic surgery, and neurosurgery.
Compliance to intervention
US guided	68.1%		20.7%			86.2%					100.0%
Procedure note completion	90.4%		93.1%			86.2%					100.0%
Supervision	70.2%		77.8%			73.1%					87.5%
Level of training
Resident	61.1%		58.6%			60.3%					75.0%
Fellow	25.3%		27.6%			24.1%					25.0%
Attending	4.2%		6.9%			3.4%					0.0%
Advance practitioner	3.2%		3.4%			3.4%					0.0%
Unknown	6.3%		3.4%			8.6%					0.0%
Supervisor type
Resident	3.0%		4.8%			2.6%					0.0%
Fellow	54.5%		33.3%			57.9%					100.0%
Attending	42.4%		61.9%			39.5%					0.0%
Location of CVC Placement	Internal Jugular (n=25)					Subclavian (n=25)
MICU	32.0%					32.0%
SICU*	40.0%					52.0%
Operating room	28.0%					16.0%
Average no. of attempts/procedure	1.4					1.5
Indications for subclavian insertion (n=25)
Trauma/surgical site	60.0%
Need for additional access	16.0%
Unsuccessful IJ placement	4.0%
Unclear	20.0%
Iatrogenic Pneumothorax Rate (20062014)	Year
	2006	2007		2008	2009		2010	2011	2012			2013	2014
% of CVC insertions associated with IAP	0.94	1.49		0.44	0.13		0.20	0.07	0.04			0.11	0.07
All‐cause IAP per 1,000 discharges	1.32	1.29		0.98	0.71		0.83	0.49	0.13			0.35	0.23
	Preintervention				Postintervention					P Value
CVC‐ associated IAP short term (2006 vs 2008)	0.94%				0.44%					0.088
CVC‐associated IAP long term (2006 vs 20082014)	0.94%				0.13%					<0.0001
All‐cause IAP per 1,000 discharges short term (2006 vs 2008)	1.32				0.98					<0.0001
All‐cause IAP per 1,000 discharges long term (2006 vs 2008‐14)	1.32				0.52					<0.0001

Figure 2

With the decrease in CVC‐associated IAP, we also saw a decrease in our all‐cause IAP rate per 1000 discharges from 1.32 in 2006 to 0.98 in 2008. This represents a 26% short‐term reduction (P<0.0001). We also saw a decrease in our all‐cause IAP rate per 1000 discharges to 0.52 from 2008 to 2014, representing a 61% long‐term reduction (P<0.0001). This decrease in all‐cause IAP postintervention occurred despite an older patient population (P<0.01) for all discharges. Our hospital is now in the highest performance UHC quartile for all‐cause IAP in 2012 to 2014.

After our multifaceted intervention in 2007, there was substantially more and consistent documentation of CVC procedure notes from less than 500 in 2006 to greater than 2000 in 2009. The distribution of CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed a decrease in the proportion of femoral lines from 15% to 11%, increase in IJ lines from 31% to 49%, and a decrease in SC from 54% to 40% (P=0.001). The distribution of IJ CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed an increase in the proportion of procedures with ultrasound documentation from 13% to 93% (P<0.001) (Figure 3).

Figure 3

In our ICU 2014 audit, the majority of CVC lines were placed by residents under supervision (>70%), and most used the standard CVC note template to document the procedure (90%). Of the total CVC approach, 66% were IJ and 4% were SC. Eighty‐six percent used ultrasound during IJ placement. The majority of SC insertions were placed in the surgical ICU and had clear indications (80%) for placement. Of those, 75% were due to trauma (limited access to neck) or surgery (interfering with surgical site) (Table 2).

Summary

This quality‐improvement intervention demonstrates that a multidisciplinary team can successfully implement a multifaceted intervention that sustainably reduces the rate of IAP complications from CVC placement and improves patient safety over 7 years. We found high compliance with our intervention, which included an increase in CVC notes and documentation of ultrasound guidance. There was also an increase in the IJ approach in our postintervention period. We showed statistically significant long‐term reductions in both CVC‐associated and all‐cause IAP rates. From 2011 to 2014, there were only 5 cases of CVC‐associated IAP requiring chest tube placement. Post hoc analysis showed a statistically significant decrease in CLABSI rates (P<0.0001) from a preintervention rate of 1.6 infections per 1000 central line days to postintervention average rate of 0.68 infections per 1000 central line days. This decrease may be related to the incorporation of wide sterile barrier techniques in our CVC training workshops, checklists, and template procedure notes.

A strength of this study is the sustained significant long‐term reduction in IAP. There are few data that exist to describe sustained interventions in this area. Sustainability was achieved by integrating our interventions into ongoing programs that already existed in the hospital; we incorporated our simulation training into the existing new resident orientation, increased the availability of existing ultrasound equipment, and continued our IAP chart review through coding quality with feedback to involved services. The procedure note template continues to be easily available in our EMR, and the SC approach to CVC placement is limited to select cases.

Based on a post hoc cost‐benefit analysis, the financial benefits of decreasing the rate of IAP outweigh the costs associated with implementation of this initiative. The purchase cost for a Site‐Rite (Bard Access Systems) ultrasound machine was $18,000. The cost of materials for 1 workshop is $5000 annually. Cases from the Nationwide Inpatient Sample that were flagged by this PSI had 7.0% excess mortality, 4.4 days of excess hospitalization, and approximately $18,000 in excess hospital charges.[17, 18] Based on these data, if we had continued at our preintervention rate of CVC‐associated IAP requiring chest tube placement, we would estimate 9 additional CVC‐associated IAPs requiring chest tube insertion per year. This would result in over $180,000 of additional costs annually. Based on an initial cost of $100,000 for 4 workshops and the necessary equipment, we would have realized our cost savings in less than 1 year postintervention. These are all approximate costs, and further detailed analysis is needed.

One challenge with this intervention is the culture change away from using the SC approach, and the concern from trainees of how they would learn to perform SC CVC if needed. We would suggest dedicated SC CVC ultrasound training for those services who may need to use this approach (eg, neuroanesthesia and trauma).

Interpretation/Relation to Other Evidence

The field of implementation science can help explain why some projects are successful and others fail. We can further dissect the success of this project using an implementation science model similar to that described by French et al.[19] French et al. describe 4 behavior‐change techniques. These steps include (1) who needs to do what differently, (2) which barriers and enablers need to be addressed, (3) which intervention component could overcome the barriers and enhance enablers, and (4) how can behavior change be measured and understood. Barriers included suboptimal skills of residents, low awareness of evidence‐based guidelines, and entrenched practices inconsistent with best evidence. There was also a belief that IJ lines were more likely to become infected. Targeted behaviors needing to be done differently were the choice of CVC placement site and insertion technique. Barriers to change were assessed by asking members of the project team to explore with members of their service what led them to do CVC lines without ultrasound guidance. Enhancements focused on information provision, simulation practice, and persuasive communication. Behavior change was measured by tracking the number of IAPs, site of CVC, and documentation of technique. Continuation of these interventions based on this theoretical framework drove maintenance of gains.

We completed our main intervention planning in 90 days, and met our short‐term goal on schedule. The Institute for Healthcare Improvement (IHI) advocates that such short timelines are efficient mechanisms for developing and acting on projects. Other institutions have reported on similar rapid‐cycle planning and short‐term goal setting[20]

Limitations

Our study captures the experience of a quality‐improvement team at a single academic center, and our results may not be generalizable to other institutions. Our chart review process only occurred once a case had been identified through AHRQ PSI methodology. It is possible that the PSI does not capture all cases of IAP, although we believe our coding department has a very rigorous process to look for all IAP evidence in the patient's record. We used administrative data to determine the number of hospital‐wide CVC procedures.

Our compliance data with interventions from STRIDE are based on looking for key words in procedure note documentation (so undocumented notes are not captured). To validate this, we performed a manual audit of our adherence to our intervention in 2014, and those data are consistent with the results from our STRIDE data.

Our study's observational design also cannot control for exogenous effects on physician practice relating to CVC insertion or the overall risk of IAP. Some of our decrease in complications may be from the increase in PICC line use. Nevertheless, our CVC‐associated IAP rate has decreased despite >6000 non‐PICC CVCs in our ICU over the past 5 years, and a rising CMI (18% increase in postintervention period) and older population of patients with CVC insertion (P<0.0001)

Intended Improvement/Study Question

Our team set out to plan and implement a set of multifaceted interventions within 90 days. The short‐term goal was a 50% reduction in the CVC IAP and all‐cause IAP rate within 18 months, and the long‐term goal was sustained reduction of CVC IAP and all‐cause IAP rate.

Setting

Stanford University Medical Center is an academic medical center with 465 beds and over 25,000 inpatient admissions per year, providing both general acute care services and tertiary medical care. Residents perform CVC bedside procedures when central venous access is needed, in the intensive care unit (ICU), operating room (OR), and inpatient units. Prior to this project, ultrasound equipment was only available in the emergency department (ED) and ICUs. There was no formal CVC procedure supervision policy, CVC training curriculum, and procedure note templates for documentation of CVC insertion.

Planning the Interventions

A multidisciplinary quality‐improvement team met weekly during the 90‐day design period from January 2007 to March 2007. Our team included representatives from the departments of medicine, anesthesia and critical care, surgery, nursing, and emergency medicine. We also partnered with our institution's clinical and administrative leaders, experts in simulation, and the hospital quality department.

We hypothesized that a standardized set of education and training interventions promoting ultrasound‐guided IJ CVC insertion as the method of choice at our hospital would significantly reduce our rate of CVC‐associated IAP. Our multifaceted intervention included: (1) clinical and documentation standards based on evidence, (2) cognitive aids, (3) simulation training, (4) purchase and deployment of ultrasound equipment, and (5) feedback to clinical services.

Our team followed the define, measure, analyze, improve, control (DMAIC) framework.[15] We set interval goals with target completion dates throughout the 90‐day period, identified owners of each goal, and tracked progress with a shared spreadsheet.

In the 90‐day intervention, we accomplished the following: (1) conducted root‐cause analysis of IAP cases for fiscal year 2006, (2) created clinical and documentation standards around CVC placement, (3) created cognitive aids and procedure note templates, (4) developed simulation training courses, and (5) requested purchase of additional ultrasound equipment.

Data Collection

To evaluate our progress in reducing the rates of IAP, we tracked the incidence of IAP using UHC and AHRQ PSI methodology. In collaboration with our hospital's quality department, we manually reviewed every PSI‐identified case of IAP. This review has focused on identifying whether or not pneumothorax actually occurred, and whether it was associated with CVC insertion. For those associated with CVC, data were collected for patient location and service, the procedure site, whether ultrasound was used, whether a chest tube was required, and the final disposition of the patient.

Demographic data (age, gender, case mix index [CMI]) shown in Table 1 were obtained through MIDAS+ Solutions (Tucson, Arizona), a proprietary database that contains healthcare management coded data. Total hospital CVC insertion rates were calculated using International Classification of Diseases, Ninth Revision (ICD‐9) coding for 38.93 and 38.97. ICU central lineassociated blood stream infections (CLABSI) data were obtained from internal collection by our infection control team. Number and location of CVCs placed in the ICU data were obtained from nursing flow sheets in our electronic medical record (EMR). Cost information was provided by our finance department using internal accounting.

The project granted a Notice of Determination of Approval from the Stanford Administrative Panels for the Protection of Human Subjects (institutional review board).

Methods of Evaluation/Analysis

For the purpose of this analysis, the preintervention period was defined as January 1, 2006 through December 31, 2006, our first year of IAP case review. We defined the intervention period as January 1, 2007 through December 31, 2007, during which we planned and implemented hospital‐wide standardization of CVC insertion practices and incorporated CVC insertion training simulation into resident orientation in July 2007. The postintervention period was defined as January 1, 2008 through December 31, 2014.

All statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX). [2] tests were used to determine statistical differences in pre‐ versus postintervention patient demographic data (age, gender, CMI), CVC insertion rates, and CLABSI rates. Because IAP is a rare event, a statistical process control g‐chart was created using QI Macros (KnowWare International, Inc., Denver, CO) to show the number of CVC procedures between IAP. [2] and Fisher exact tests were used to determine statistical differences in CVC anatomic location and use of ultrasound pre‐ and postintervention. A 2‐sided Z test to show a difference in proportions was used to determine statistical differences in CVC‐related IAP rate and all‐cause IAP rate pre‐ and postintervention.

Measuring Adherence to Intervention

Outcomes

The rate of CVC‐associated IAP was calculated as the total number of IAPs attributed to CVCs divided by the total number of CVCs inserted determined by ICD‐9 coding for 38.93 and 38.97. The total IAP rate was calculated as the total number of IAP/1000 discharges.

Interventions

Our team began the intervention in early 2007 with promotion of ultrasound‐guided IJ catheterization. Clinical exceptions included: (1) trauma or code situations where access to the neck is limited, (2) suspected or confirmed neck injuries, (3) presence of a tracheostomy, and (4) bilateral internal jugular sites unsuitable for catheterization.

Our hospital adopted new formal CVC insertion policies consistent with the above training and education efforts. All physicians were required to document CVC insertions using the template available in the EMR. To be certified to perform CVC insertion independently, trainee physicians were required to complete the simulation training and successfully place a minimum of 5 CVCs directly supervised by an already‐certified physician. This was consistent with the Accreditation Council for Graduate Medical Education suggested minimum requirement in 2007. In our critical care units, all CVC insertions must be supervised by an ICU fellow or attending.

To reinforce the on‐the‐ground work by our physician leaders, we created 2 education tools to embed best practices into our CVC insertion workflow. A checklist with best practices for CVC insertion that was distributed throughout the hospital via central line kits and educational flyers, and a CVC insertion procedure note template consistent with California Department of Public Health documentation requirements was made available in our EMR.

In June 2007, we integrated CVC insertion simulation training into procedure workshops required for all medicine, surgery, anesthesia, and emergency medicine trainees during their intern year. These workshops promoted ultrasound‐guided IJ catheterization and supporting evidence for the new IJ site preference. Training sessions were 2 to 3 hours, and included a demonstration of best‐practice CVC insertion, as well as training with simulation models supervised by an instructor using a standardized CVC checklist. These trainings used both the Blue Phantom human torso model (www.bluephantom.com; CAE Healthcare, Sarasota, FL), and the MOVAT Chicken Model (http://www.movat.com; MOVAT, Los Angeles, CA), a biological ultrasound‐compatible training device that simulates all of the steps of CVC insertion. Each medicine resident is required to complete a CVC insertion on the MOVAT model. In addition, the surgery residents have follow‐up testing with a checklist and by an observer within 1 to 2 months after their workshop.

Hospital administration provided funds to purchase ultrasound machines for patient units such as medicine, cardiology, ED, and ICU). A total of 4 Site‐Rite (Bard Access Systems, Inc., Salt Lake City, UT) ultrasounds were purchased in 2007. The hospital has continued to purchase ultrasound units yearly, and had 53 ultrasound units in 2014

Cases of IAP were continuously reviewed throughout the intervention period. Based on their higher CVC‐associated IAP rates, the ORs and catheterization lab were identified as having opportunities for improvement. In 2008, Hospital quality‐improvement leadership met with physician leaders in these areas to review their CVC‐related IAP data and to discuss strategies to reduce their IAP rates. These strategies included lessons learned from other services that had successfully decreased their IAP rates.

To sustain our gains, we continue to review all IAP through our coding quality, clinical documentation, quality reporting departments, and peer review. We have implemented other strategies to decrease IAP, such as the use of ultrasound guidance for bedside thoracentesis procedures, which became possible after the availability of more ultrasound devices. Training for ultrasound‐guided thoracentesis was done by our procedure‐team attending during supervision of residents.

Outcomes

Preintervention (January 1, 2006 to December 31, 2006)

There were a total of 26 cases of IAP in 2006. Of these, 15 (58%) were associated with CVC insertion (Figure 1). The single procedure associated with the largest proportion of IAP was SC CVC insertion (11 cases, 42% of all IAP cases). Eleven CVC‐associated IAPs were significant enough to require chest tube placement. Our hospital recorded a total of 1593 CVC insertions (ICD‐9 codes 38.93 and 38.97) in 2006.

Figure 1

Postintervention (January 1, 2008 to December 31, 2014)

There were a total of 80 cases of IAP over 7 years, of which 24 (30%) were associated with CVC insertion. Of these, 16 required chest tube placement. In the last 4 years of the postintervention period (20112014), there were only 5 cases of CVC‐associated IAP requiring chest tube placement (Figure 1). There were a total of 12,000 CVC insertions recorded over the same period.

We successfully met both our short‐ and long‐term goals. Our preintervention CVC‐associated IAP rate was 0.94%, and our post‐intervention rate during 2008 was 0.44%, a short‐term reduction of 53% (P=0.088). Our average postintervention CVC‐associated IAP rate for the years 2008 through 2014 was 0.13%, a significant long‐term reduction of 86% (P<0.0001) (Table 2). The decrease in CVC‐associated IAP rates occurred despite an older patient population (P<0.001) and a higher CMI (P<0.001) in postintervention patients who received a CVC (Table 1). Special cause variation corresponding to a change in our process is demonstrated in Figure 2. The preintervention average number of procedures between IAP was 114.8 and increased to 460.7 in the postintervention period.

Table 2.Summary of Outcomes

	Total CVC (n=95)		Subclavian (n=29)			Internal Jugular (n=58)					Femoral (n=8)
NOTE: Chart review of CVC placement in ICU during 2014 showing high compliance to intervention: US guided IJ placement, standard procedure note completion, and supervision, and significant long‐term reduction in CVC‐associated IAP rate and all‐cause IAP rate. A total of 100 patient charts were randomly selected and reviewed. Five patients receiving a PICC were not included in the analysis. All PICC lines were placed by trained registered nurses without supervision under US guidance. Of note, proportions of CVC types are not accurate representation; additional subclavian and internal jugular lines were randomly selected and reviewed. Abbreviations: CVC, central venous catheterization; IAP, iatrogenic pneumothorax; ICU intensive are unit; IJ, internal jugular; MICU, medical intensive care unit; PICC, peripherally inserted central catheter; SICU, surgical intensive care unit; US, ultrasound. *Surgical ICU includes all surgery primary teams including trauma, general surgery, cardiothoracic surgery, and neurosurgery.
Compliance to intervention
US guided	68.1%		20.7%			86.2%					100.0%
Procedure note completion	90.4%		93.1%			86.2%					100.0%
Supervision	70.2%		77.8%			73.1%					87.5%
Level of training
Resident	61.1%		58.6%			60.3%					75.0%
Fellow	25.3%		27.6%			24.1%					25.0%
Attending	4.2%		6.9%			3.4%					0.0%
Advance practitioner	3.2%		3.4%			3.4%					0.0%
Unknown	6.3%		3.4%			8.6%					0.0%
Supervisor type
Resident	3.0%		4.8%			2.6%					0.0%
Fellow	54.5%		33.3%			57.9%					100.0%
Attending	42.4%		61.9%			39.5%					0.0%
Location of CVC Placement	Internal Jugular (n=25)					Subclavian (n=25)
MICU	32.0%					32.0%
SICU*	40.0%					52.0%
Operating room	28.0%					16.0%
Average no. of attempts/procedure	1.4					1.5
Indications for subclavian insertion (n=25)
Trauma/surgical site	60.0%
Need for additional access	16.0%
Unsuccessful IJ placement	4.0%
Unclear	20.0%
Iatrogenic Pneumothorax Rate (20062014)	Year
	2006	2007		2008	2009		2010	2011	2012			2013	2014
% of CVC insertions associated with IAP	0.94	1.49		0.44	0.13		0.20	0.07	0.04			0.11	0.07
All‐cause IAP per 1,000 discharges	1.32	1.29		0.98	0.71		0.83	0.49	0.13			0.35	0.23
	Preintervention				Postintervention					P Value
CVC‐ associated IAP short term (2006 vs 2008)	0.94%				0.44%					0.088
CVC‐associated IAP long term (2006 vs 20082014)	0.94%				0.13%					<0.0001
All‐cause IAP per 1,000 discharges short term (2006 vs 2008)	1.32				0.98					<0.0001
All‐cause IAP per 1,000 discharges long term (2006 vs 2008‐14)	1.32				0.52					<0.0001

Figure 2

With the decrease in CVC‐associated IAP, we also saw a decrease in our all‐cause IAP rate per 1000 discharges from 1.32 in 2006 to 0.98 in 2008. This represents a 26% short‐term reduction (P<0.0001). We also saw a decrease in our all‐cause IAP rate per 1000 discharges to 0.52 from 2008 to 2014, representing a 61% long‐term reduction (P<0.0001). This decrease in all‐cause IAP postintervention occurred despite an older patient population (P<0.01) for all discharges. Our hospital is now in the highest performance UHC quartile for all‐cause IAP in 2012 to 2014.

After our multifaceted intervention in 2007, there was substantially more and consistent documentation of CVC procedure notes from less than 500 in 2006 to greater than 2000 in 2009. The distribution of CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed a decrease in the proportion of femoral lines from 15% to 11%, increase in IJ lines from 31% to 49%, and a decrease in SC from 54% to 40% (P=0.001). The distribution of IJ CVC procedure notes in the pre‐ (2006) versus postintervention (20082014) period showed an increase in the proportion of procedures with ultrasound documentation from 13% to 93% (P<0.001) (Figure 3).

Figure 3

In our ICU 2014 audit, the majority of CVC lines were placed by residents under supervision (>70%), and most used the standard CVC note template to document the procedure (90%). Of the total CVC approach, 66% were IJ and 4% were SC. Eighty‐six percent used ultrasound during IJ placement. The majority of SC insertions were placed in the surgical ICU and had clear indications (80%) for placement. Of those, 75% were due to trauma (limited access to neck) or surgery (interfering with surgical site) (Table 2).

Summary

This quality‐improvement intervention demonstrates that a multidisciplinary team can successfully implement a multifaceted intervention that sustainably reduces the rate of IAP complications from CVC placement and improves patient safety over 7 years. We found high compliance with our intervention, which included an increase in CVC notes and documentation of ultrasound guidance. There was also an increase in the IJ approach in our postintervention period. We showed statistically significant long‐term reductions in both CVC‐associated and all‐cause IAP rates. From 2011 to 2014, there were only 5 cases of CVC‐associated IAP requiring chest tube placement. Post hoc analysis showed a statistically significant decrease in CLABSI rates (P<0.0001) from a preintervention rate of 1.6 infections per 1000 central line days to postintervention average rate of 0.68 infections per 1000 central line days. This decrease may be related to the incorporation of wide sterile barrier techniques in our CVC training workshops, checklists, and template procedure notes.

A strength of this study is the sustained significant long‐term reduction in IAP. There are few data that exist to describe sustained interventions in this area. Sustainability was achieved by integrating our interventions into ongoing programs that already existed in the hospital; we incorporated our simulation training into the existing new resident orientation, increased the availability of existing ultrasound equipment, and continued our IAP chart review through coding quality with feedback to involved services. The procedure note template continues to be easily available in our EMR, and the SC approach to CVC placement is limited to select cases.

Based on a post hoc cost‐benefit analysis, the financial benefits of decreasing the rate of IAP outweigh the costs associated with implementation of this initiative. The purchase cost for a Site‐Rite (Bard Access Systems) ultrasound machine was $18,000. The cost of materials for 1 workshop is $5000 annually. Cases from the Nationwide Inpatient Sample that were flagged by this PSI had 7.0% excess mortality, 4.4 days of excess hospitalization, and approximately $18,000 in excess hospital charges.[17, 18] Based on these data, if we had continued at our preintervention rate of CVC‐associated IAP requiring chest tube placement, we would estimate 9 additional CVC‐associated IAPs requiring chest tube insertion per year. This would result in over $180,000 of additional costs annually. Based on an initial cost of $100,000 for 4 workshops and the necessary equipment, we would have realized our cost savings in less than 1 year postintervention. These are all approximate costs, and further detailed analysis is needed.

One challenge with this intervention is the culture change away from using the SC approach, and the concern from trainees of how they would learn to perform SC CVC if needed. We would suggest dedicated SC CVC ultrasound training for those services who may need to use this approach (eg, neuroanesthesia and trauma).

Interpretation/Relation to Other Evidence

The field of implementation science can help explain why some projects are successful and others fail. We can further dissect the success of this project using an implementation science model similar to that described by French et al.[19] French et al. describe 4 behavior‐change techniques. These steps include (1) who needs to do what differently, (2) which barriers and enablers need to be addressed, (3) which intervention component could overcome the barriers and enhance enablers, and (4) how can behavior change be measured and understood. Barriers included suboptimal skills of residents, low awareness of evidence‐based guidelines, and entrenched practices inconsistent with best evidence. There was also a belief that IJ lines were more likely to become infected. Targeted behaviors needing to be done differently were the choice of CVC placement site and insertion technique. Barriers to change were assessed by asking members of the project team to explore with members of their service what led them to do CVC lines without ultrasound guidance. Enhancements focused on information provision, simulation practice, and persuasive communication. Behavior change was measured by tracking the number of IAPs, site of CVC, and documentation of technique. Continuation of these interventions based on this theoretical framework drove maintenance of gains.

We completed our main intervention planning in 90 days, and met our short‐term goal on schedule. The Institute for Healthcare Improvement (IHI) advocates that such short timelines are efficient mechanisms for developing and acting on projects. Other institutions have reported on similar rapid‐cycle planning and short‐term goal setting[20]

Limitations

Our study captures the experience of a quality‐improvement team at a single academic center, and our results may not be generalizable to other institutions. Our chart review process only occurred once a case had been identified through AHRQ PSI methodology. It is possible that the PSI does not capture all cases of IAP, although we believe our coding department has a very rigorous process to look for all IAP evidence in the patient's record. We used administrative data to determine the number of hospital‐wide CVC procedures.

Our compliance data with interventions from STRIDE are based on looking for key words in procedure note documentation (so undocumented notes are not captured). To validate this, we performed a manual audit of our adherence to our intervention in 2014, and those data are consistent with the results from our STRIDE data.

Our study's observational design also cannot control for exogenous effects on physician practice relating to CVC insertion or the overall risk of IAP. Some of our decrease in complications may be from the increase in PICC line use. Nevertheless, our CVC‐associated IAP rate has decreased despite >6000 non‐PICC CVCs in our ICU over the past 5 years, and a rising CMI (18% increase in postintervention period) and older population of patients with CVC insertion (P<0.0001)

Setting

This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.

Nurse Education

Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.

In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.

Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.

The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Screening Tool

A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.

Figure 1

The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO₂) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.

If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.

If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.

Data Collection

Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.

Performance Measurement

The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.

Clinical Activities

To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.

Statistical Analysis

To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.

Clinical Activities

Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.

Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.

Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.

Figure 2

We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).

Setting

This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.

Nurse Education

Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.

In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.

Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.

The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Screening Tool

A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.

Figure 1

The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO₂) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.

If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.

If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.

Data Collection

Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.

Performance Measurement

The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.

Clinical Activities

To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.

Statistical Analysis

To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.

Clinical Activities

Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.

Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.

Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.

Figure 2

We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).

Setting

This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.

Nurse Education

Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.

In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.

Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.

The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.

Screening Tool

A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.

Figure 1

The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO₂) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.

If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.

If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.

Data Collection

Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.

Performance Measurement

The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.

Clinical Activities

To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.

Statistical Analysis

To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.

Clinical Activities

Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.

Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.

Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.

Figure 2

We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).

Intervention

This study utilized Medigram (Medigram, Inc., https://medigram.com), a free HCGM application for smartphones (available on iOS and Android) that allows users to send and receive encrypted, password‐protected text messages via the hospital wireless fidelity (Wi‐Fi) network, using commercial cellular networks as backup.

Study Design

In an eight‐week prospective, cluster‐randomized, controlled trial conducted at Stanford Hospital (June 25, 2012August 17, 2012), three of five inpatient medicine teams were randomized to use Medigram in addition to the existing hospital paging system (HCGM teams); the remaining two teams were assigned to use hospital paging only (control teams). Each team included one attending physician, one resident, two interns, two medical students, and a case manager. According to prescheduled rotations, attendings rotated every two weeks, and residents, interns, and medical students rotated every four weeks. All rotations were either off‐service or off‐site, with the exception of two attendings who rotated between study teams but within their experimental designations. Case managers remained with the same team. Additionally, the satellite pharmacy was provided with an HCGM‐equipped smartphone to communicate with experimental teams.

Participation was voluntary, with a 96% participation rate (n=75). HCGM teams downloaded the free application onto their smartphones. Participants without smartphones were provided with one for the duration of the study. Proper application use was demonstrated by one researcher in a 10‐minute standardized presentation. HCGM teams were encouraged to use the application in lieu of paging, except when patient care could be compromised.

All participants completed linked baseline and post‐study surveys. Gift cards valued at $10 were provided on completion of each survey. Though participants were assigned to either HCGM or control groups based on the randomized assignment of their preset cluster (hospital team) to an HCGM or control group, analysis was performed on the individual level due to the hospital's set rotation schedule, which resulted in dynamic, frequently changing clusters. We also compared average length of stay and time of discharge for patients treated by control versus HCGM teams. Clinical outcome data were obtained from the hospital's database using Midas+ Statit Solutions (Midas+ Statit Solutions Group, Tucson, AZ). Survey and clinical outcome data were analyzed in Stata (StataCorp, College Station, TX) and R (R Foundation for Statistical Computing, Vienna, Austria).

Survey Design and Analysis

Identical, anonymous baseline surveys were administered to control and HCGM teams. These surveys assessed attitudes toward the hospital paging system using a 5‐point Likert scale (1=low, 5=high) to evaluate perceived measures of effectiveness, workflow integration, and overall satisfaction. Wilcoxon rank sum tests were used to compare control and HCGM group responses to these questions. Free response questions asked participants to list the most effective and ineffective aspects of the paging system.

Post‐study surveys included all baseline survey questions, as well as questions about personal texting behavior. Post‐study HCGM surveys also included a parallel set of questions rating the HCGM application on the same measures of perceived effectiveness, workflow integration, and overall satisfaction. Wilcoxon signed rank tests were used to compare HCGM participants' baseline evaluations of paging to their post‐study evaluations of the HCGM application. Baseline and post‐study surveys were linked by the last four digits of respondent cell phone numbers. To compare control and HCGM group perceptions of the hospital paging system at study completion, post‐study survey responses were evaluated using Wilcoxon rank sum tests. The family‐wise error rate was left unadjusted due to concerns around inflated type II errors, given the high degree of correlation between survey questions.

All free response questions were analyzed using thematic analysis and grounded theory. After reviewing responses to each question, a list of overarching themes was constructed. Two researchers then independently reviewed each free‐response entry to assign it to one or more of these themes (some responses included several ideas with distinct themes). Entries with concordant theme assignments (90%) were coded as such; nonconcordant entries required an additional round of review to reach concordance. Finally, objective outcome measures including length of stay and time of discharge were analyzed by two‐sample t test.

Information Security

The HCGM application in this study features 256‐bit encryption technology and requires a six‐digit password to access texts. For added security, a study‐dedicated server (HP ProLiant DL 180 G6; Hewlett‐Packard Co., Palo Alto, CA) with 4‐TB hard drive capacity (4 Seagate Barracuda ST1000DM003 1 TB 7200 RPM internal hard drives; Seagate Technology PLC, Cupertino, CA) was installed in the Stanford School of Medicine Data Center to store encrypted text messages. Data stored on the phones/server were accessible only to study participants, not researchers. These security measures were approved by Stanford Hospital and Stanford School of Medicine's security and privacy review process.

Hospital Paging System

Stanford Hospital and Clinics is a quaternary care academic medical center with 613 beds, 49 operating rooms, and over 25,000 inpatient admissions per year.[8] The institution uses one‐way alphanumeric pagers (primary model: Daviscomm BR802 Flex Pager from USA Mobility, secondary model: Sun Telecom Titan 3 Plus from USA Mobility; USA Mobility Inc., Springfield, VA). USA Mobility operates the largest one‐ and two‐way paging networks in the United States.[9]

Control and HCGM Group Characteristics

Control and HCGM groups were well matched demographically (Table 1). The average ages of control and HCGM group members were 30.10 and 30.95, respectively. Both groups were 59% male and 41% female.

A similar distribution of team member roles was observed in both groups, with two exceptions. First, the proportion of attending respondents in the HCGM group was lower than in the control group. This was due to the fact that several HCGM attendings entered discrepant ID codes on their surveys, thus making it impossible to link baseline and post‐study responses; these data were excluded. Additionally, two HCGM attendings were on service for four, rather than the standard two weeks, meaning two additional data points from unique attendings could not be obtained. Second, the experimental group included four pharmacists, whereas the control group did not. As a sensitivity test, we analyzed the data excluding the pharmacists, and this did not change our results.

Baseline Evaluations of the Hospital Paging System

At baseline, there were no significant differences between control and HCGM participants' perceptions of paging effectiveness (see Supporting Table 1, in the online version of this article). On a 5‐point rating scale (1=low, 5=high), 63 subjects rated their overall satisfaction with the paging system an average of 2.79 (95% confidence interval: 2.55‐3.03).

In free response questions, components of the paging system most frequently cited as effective included: reliability of message transmission, alphanumeric text paging, and ease of use (30.4%, 25.0%, and 14.3% of 56 respondents, respectively) (Table 2). Ineffective aspects included: time wasted waiting for responses to pages, the unidirectional nature of pagers, and needing to find a computer to send a text page (29.3%, 24.1%, and 20.7% of 58 respondents, respectively) (Table 2).

Baseline Utilization of Text Messaging

The majority of participants were familiar with text messaging and regularly used it personally and professionally prior to the start of the study. 90.5% of participants (n = 63) reported sending an average of 1 personal text messages per day, with the largest proportion (39.7%) sending 1‐5 texts per day (see Supporting Figure 1A in the online version of this article). 58.1% of respondents (n=62) reported sending an average of 1 text messages per day related to patient care (see Supporting Figure 1B in the online version of this article), with the largest fraction (58.3%) sending 1‐5 texts per day.

HCGM Adoption and Usage Patterns

Active use of HCGM was defined as using the application to send or receive an average of 1 text messages per day. Of HCGM participants, 67% self‐reported 1 week of active use of the application, indicating a strong compliance rate. Among non‐attendings, 70% reported sending 1 or more texts to other team members per day; this percentage increased to 86% among those whose attendings texted them at least once per day (47% of non‐attendings). Respondents who text frequently in their personal lives (>5 texts/day) were more likely to use the application; 90% of these respondents sent 1 or more HCGM texts per day.

Among 12 subjects who did not report sending or receiving 1 HCGM text/day, the top three reasons were: other team members were not using it (67%), no need to use it given the close proximity of other team members (67%), and other (33%). A Wilcoxon rank sum test was used to compare the ages of active versus nonactive users; no significant age difference was found (P=0.200).

To provide an objective measure of application adoption, usage data for each HCGM participant were obtained from the application developers. Because much of the study's first week was spent onboarding and instructing participant, the first week was not included in the analysis. Of 43 individuals enrolled in the study for at least one of the seven remaining weeks, 56% sent a total of 5 texts, 44% sent 10 texts, and 28% sent 20 texts. HCGM users on three teams sent an aggregate mean of 123 texts/week. Data on number of messages received by each user were not available.

Perceived Effectiveness: Paging Versus HCGM

In post‐study surveys, HCGM participants rated HCGM significantly higher (P<0.05) than paging (Table 3) in terms of ability to communicate thoughts clearly (P=0.010) and efficiently (P=0.009). HCGM was also deemed more effective at integrating into workflow during rounds (P=0.018) and patient discharge (P=0.012). Overall satisfaction with HCGM was also significantly higher (P=0.003).

Comparison of Pre‐ and Post‐study Perceived Effectiveness of the Hospital Paging System

In post‐study evaluations, both control and HCGM participants rated the paging system's effectiveness less favorably (P<0.05) compared to baseline in terms of ability to receive messages/stay informed in real time (control P=0.002, HCGM P=0.031) (Table 4). Controls also reported a decrease from baseline in perceived effectiveness of paging in terms of ability to send messages (P=0.019) and integrate into workflow during patient admissions (P=0.020). HCGM participants found paging less effective at communicating thoughts clearly (P=0.004) and efficiently (P=0.018). No significant differences existed between control and HCGM groups' average assessments of paging at the conclusion of the study (see Supporting Table 2, in the online version of this article).

HCGM User Experience

When asked if they would recommend using an HCGM system to facilitate communication on the internal medicine wards, 85% of HCGM participants replied yes, 15% reported not sure, and 0% reported no. Based on free response entries, HCGM's most effective features (Table 5) included ease of use, group texting capacity, and speed (32.4%, 32.4%, and 23.5% of 34 respondents, respectively); its most ineffective aspects (Table 5) included lack of ubiquity, inconsistent usage by those with access to the application, and reliability of message transmission (30.3%, 24.2%, and 15.2% of 33 respondents, respectively).

Limitations

Our study had several limitations. HCGM was available only to individuals in the experimental arm of the study; most members of the internal medicine department and all other departments were not reachable through the application. This lack of ubiquity was a frequently cited frustration. Among individuals to whom HCGM was made available, barriers to adoption included: close proximity to would‐be message recipients, concern that smartphone usage in front of patients might appear unprofessional, and inconsistent or dropped service (weak or no Wi‐Fi signal in some areas). A technical problem with the Android platform midway through the study served as a potential frustration to several participants.

Due to the aforementioned issues, some participants used the HCGM application in a very limited way. We also did not replace hospital pagers (infeasible in this hospital setting); the HCGM application was added as a supplemental system. Future studies might explore the replacement of paging systems with HCGM‐type applications, as well as delve further into quantitative patient care outcomes.

It should be noted that the start of the study unintentionally coincided with the start of new interns and medical students in the hospital. Although it is possible that their relative unfamiliarity with the hospital may have made them more amenable to adopting a new technology, it is also possible that they may have been less likely to do so in the midst of such a major transitional period. Finally, this was a single‐site study, and as such, its findings may not be broadly generalizable. More research on such interventions is warranted, particularly in the context of current insecure communication methods such as paging that may make hospital‐wide adoption of new methods of secure communication, such as HCGM, mandatory.