Hospitalization can be considered a teachable moment for smoking cessation13 for the 6.5 million adult smokers who are hospitalized in the United States each year.4 Smokers who receive tobacco treatment during hospitalization and outpatient follow‐up treatment for at least 1 month are more likely to quit than patients who receive no treatment.5, 6

Unless tobacco treatment is explicitly delegated to other providers, physicians shoulder the responsibility of encouraging smokers to quit and prescribing smoking cessation medications. This is problematic in that physicians sometimes fail to counsel their patients about quitting smoking7, 8 or recommend outpatient follow‐up.9 Few hospitals provide comprehensive treatment. In a review of 33 studies on the prevalence of smoking care delivery in hospitals, 3 hospitals reported they provided advice to quit alone, 29 provided advice plus counseling and assistance in quitting, and 8 provided advice or prescription for cessation pharmacotherapy.9 Although post‐discharge support is a key component of effective treatment for hospitalized smokers,6 only 11 reported providing follow‐up treatment, or referral for follow‐up treatment, after discharge. Among these 11 hospitals, respondents reported they provided referral or follow‐up to 1% to 74% of their smokers, with a median percentage of 24%. The 1 study that specified the type of outpatient treatment provided reported the hospital provided the state quitline number to smokers.

Instituting a dedicated smoking cessation program may enhance inpatient treatment, outpatient follow‐up, and treatment outcomes. Two studies have found that institutional smoking cessation programs increased the likelihood that patients would receive treatment and quit compared to hospitals without dedicated programs.10, 11

Although many US hospitals are developing programs to provide systematic treatment for tobacco dependence,9 little is known regarding how programs structure their staff, enroll patients, or provide treatment to patients that smoke. Instituting tobacco treatment services usually requires policy change and system‐wide approaches with quality improvement endpoint goals.8, 1214 In the United States, elements of these services include: 1) developing a cadre of trained tobacco treatment specialists, 2) implementing hospital systems for identifying smokers and referring them to the service, 3) providing inpatient treatment based on current treatment guidelines15 and 4) providing or facilitating follow‐up treatment after discharge, often via fax‐referral to tobacco quitlines. This systematic approach is still lacking in many hospitals.

To date, few evaluations of dedicated hospital‐based smoking cessation programs have been reported in the literature.8, 11 The purpose of this study is to describe patient characteristics and outcomes of a dedicated tobacco treatment service, with paid staff, in a large academic medical center. We describe treatment protocols, profile patients served, treatments provided, and summarization of 6‐month post‐discharge outcomes for smokers referred to the UKanQuit service over a 1‐year period. We close with lessons learned on how to improve the delivery of tobacco treatment to hospitalized patients.

Methods

Program Intervention

UKanQuit staff visit patients at the bedside to deliver tobacco treatment. This consists of: (a) assessing withdrawal; (b) working with the health care team to adjust nicotine replacement to keep the patient comfortable; (c) assessing patients' interest in quitting smoking; (d) providing brief motivational intervention to patients not interested in quitting; and (e) providing assistance in quitting (developing a quit plan, arranging for medications on discharge) to patients interested in quitting (Figure 1). UKanQuit staff recommend medications based on the patients' level of dependence, history of cessation, and cessation medication preferences. The recommendation is communicated in person and by chart documentation to the medical team, usually by the nursing staff. The patients' resident or attending physician makes the final determination regarding medication provided. The hospital has nicotine replacement therapy (NRT; patch, gum, and lozenge), bupropion, and varenicline in its formulary. Patients are then offered an option of fax referral to the state tobacco quitline for follow‐up counseling. UKanQuit staff documents the services provided in the EMR via SOAR (Subjective, Objective, Assessment and Referral) notes.

Figure 1

Results

Baseline

Within the study period (September 1, 2007 to August 31, 2008), 22,624 patients were admitted to the medical center (Figure 2). A total of 4150 were current smokers (ie, smoked within the past 30 days). UKanQuit staff met with 513 (68%) of 753 patients referred to the service. Some of the reasons why 32% of referred patients were not seen by the UKanQuit staff have been described in our previous paper.17 These include patient was asleep, doctor in the room, out of bed for procedure, and unable to speak. Table 1 displays the characteristics of 513 smokers treated by UKanQuit from September 1, 2007 to August 31, 2008. Patients were predominantly white (74%) with mean age of 50 years. Slightly more than half of smokers were male (57%). They had smoked an average of 18 cigarettes per day for a mean duration of 29 years, and over half (58%) smoked within 5 minutes of waking suggesting a high level of dependence. On a scale of 1 to 10 the mean interest in quitting was 7.9 (SD 2.9) and the mean craving score on a scale of 0 to 4 was 1.2 (SD 1.4) suggesting slight to mild craving.

Figure 2

Table 1.Demographic, Smoking Characteristics and Treatment Provided to 513 Patients Seen by UKanQuit Service From Sept 1, 2007 to Aug 31, 2008

Characteristics	Treated (n = 513)
NOTE: For each variable, subsamples were slightly different from total sample due to missing data. Missing data were not included in the analysis. Abbreviations: AA, Afro American; SD, standard deviation. * Interest in quitting range from 0 to 10. craving range from 0 to 4.
Demographics
Mean age (SD), years	50.2 (13.6)
Male, n (%)	291 (56.7)
Ethnicity, n (%)
White	371 (73.6)
AA	107 (21.2)
Latino	18 (3.6)
Other	8 (1.6)
Referral source, n (%)
Nursing profile	477 (94.1)
Physician	5 (1.0)
Other	25 (4.9)
Smoking characteristics
Mean number of years smoked (SD)	28.9 (14.6)
Smokes within 5 minutes of waking n (%)	270 (58.3)
Mean cigarettes smoked per day (SD)	18.4 (12.6)
Mean interest in quitting (SD)*	7.9 (2.9)
Mean craving (SD)	1.2 (1.4)
Tobacco treatment provided
Counseling
Average time spent with patients (SD)	19.9 (9.1)
Received information packet, n (%)	490 (97.4)
Set goals for quitting, n (%)	352 (73.3)
Had quit plan, n (%)	151 (33.2)
Accepted fax referral to quitline, n (%)	277 (55.8)
Opted for UKanQuit counseling, n (%)	29 (5.9)
Medication
On smoking cessation medication, n (%)	133 (26.2)
Interested in receiving or changing smoking cessation medication, n (%)	132 (26.7)
Added or changed smoking cessation medication, n (%)	195 (40.5)
Discharge med, n (%)	196 (40.2)

Discussion

Among patients served by this inpatient program, interest in quitting was high but administration of inpatient medications upon admission was low (26%). Nearly all patients were provided with written materials, a majority set some form of goal for quitting or cutting down, and many developed quit plans and received assistance adjusting inpatient and/or discharge medications. After discharge, the majority of study participants made unassisted quit attempts, as utilization of medications and quitline services was suboptimal. Fax‐referral to quitline may not, on its own, fulfill guideline recommendations for post‐discharge follow‐up.

Our intent‐to‐treat quit rate was about half of what was found in Taylor et al.'s11 hospital program dissemination trial. Their program may have had better effects as it was somewhat more intensive. It included at least 1 follow‐up phone call immediately after discharge, as well as an accompanying video and relaxation audiotape or compact disc. However, differences in outcomes may also be due to large differences between the study populations. Hospitals participating in Taylor's study only conducted intervention and outcome assessment among smokers who were ready to quit, willing to enroll in a clinical trial, and willing to complete informed consent. Our intervention and outcome data included patients who agreed to speak with UKanQuit staff, regardless of readiness to quit. Our participants did not have to complete informed consent and enroll in a trial as our analyses were conducted post hoc. Our study outcomes might better reflect quit rates for a program serving all smokers, at all levels of readiness to quit, in actual hospital practices. The mean reduction in cigarette smoking among smokers who continued to smoke at 6 months' follow‐up was statistically significant. However, findings from a lung health study show that 50% or more reduction in smoking was ultimately related to successful quitting.23

Hospitalization can be considered a teachable moment for smoking cessation13 for the 6.5 million adult smokers who are hospitalized in the United States each year.4 Smokers who receive tobacco treatment during hospitalization and outpatient follow‐up treatment for at least 1 month are more likely to quit than patients who receive no treatment.5, 6

Unless tobacco treatment is explicitly delegated to other providers, physicians shoulder the responsibility of encouraging smokers to quit and prescribing smoking cessation medications. This is problematic in that physicians sometimes fail to counsel their patients about quitting smoking7, 8 or recommend outpatient follow‐up.9 Few hospitals provide comprehensive treatment. In a review of 33 studies on the prevalence of smoking care delivery in hospitals, 3 hospitals reported they provided advice to quit alone, 29 provided advice plus counseling and assistance in quitting, and 8 provided advice or prescription for cessation pharmacotherapy.9 Although post‐discharge support is a key component of effective treatment for hospitalized smokers,6 only 11 reported providing follow‐up treatment, or referral for follow‐up treatment, after discharge. Among these 11 hospitals, respondents reported they provided referral or follow‐up to 1% to 74% of their smokers, with a median percentage of 24%. The 1 study that specified the type of outpatient treatment provided reported the hospital provided the state quitline number to smokers.

Instituting a dedicated smoking cessation program may enhance inpatient treatment, outpatient follow‐up, and treatment outcomes. Two studies have found that institutional smoking cessation programs increased the likelihood that patients would receive treatment and quit compared to hospitals without dedicated programs.10, 11

Although many US hospitals are developing programs to provide systematic treatment for tobacco dependence,9 little is known regarding how programs structure their staff, enroll patients, or provide treatment to patients that smoke. Instituting tobacco treatment services usually requires policy change and system‐wide approaches with quality improvement endpoint goals.8, 1214 In the United States, elements of these services include: 1) developing a cadre of trained tobacco treatment specialists, 2) implementing hospital systems for identifying smokers and referring them to the service, 3) providing inpatient treatment based on current treatment guidelines15 and 4) providing or facilitating follow‐up treatment after discharge, often via fax‐referral to tobacco quitlines. This systematic approach is still lacking in many hospitals.

To date, few evaluations of dedicated hospital‐based smoking cessation programs have been reported in the literature.8, 11 The purpose of this study is to describe patient characteristics and outcomes of a dedicated tobacco treatment service, with paid staff, in a large academic medical center. We describe treatment protocols, profile patients served, treatments provided, and summarization of 6‐month post‐discharge outcomes for smokers referred to the UKanQuit service over a 1‐year period. We close with lessons learned on how to improve the delivery of tobacco treatment to hospitalized patients.

Methods

Program Intervention

UKanQuit staff visit patients at the bedside to deliver tobacco treatment. This consists of: (a) assessing withdrawal; (b) working with the health care team to adjust nicotine replacement to keep the patient comfortable; (c) assessing patients' interest in quitting smoking; (d) providing brief motivational intervention to patients not interested in quitting; and (e) providing assistance in quitting (developing a quit plan, arranging for medications on discharge) to patients interested in quitting (Figure 1). UKanQuit staff recommend medications based on the patients' level of dependence, history of cessation, and cessation medication preferences. The recommendation is communicated in person and by chart documentation to the medical team, usually by the nursing staff. The patients' resident or attending physician makes the final determination regarding medication provided. The hospital has nicotine replacement therapy (NRT; patch, gum, and lozenge), bupropion, and varenicline in its formulary. Patients are then offered an option of fax referral to the state tobacco quitline for follow‐up counseling. UKanQuit staff documents the services provided in the EMR via SOAR (Subjective, Objective, Assessment and Referral) notes.

Figure 1

Results

Baseline

Within the study period (September 1, 2007 to August 31, 2008), 22,624 patients were admitted to the medical center (Figure 2). A total of 4150 were current smokers (ie, smoked within the past 30 days). UKanQuit staff met with 513 (68%) of 753 patients referred to the service. Some of the reasons why 32% of referred patients were not seen by the UKanQuit staff have been described in our previous paper.17 These include patient was asleep, doctor in the room, out of bed for procedure, and unable to speak. Table 1 displays the characteristics of 513 smokers treated by UKanQuit from September 1, 2007 to August 31, 2008. Patients were predominantly white (74%) with mean age of 50 years. Slightly more than half of smokers were male (57%). They had smoked an average of 18 cigarettes per day for a mean duration of 29 years, and over half (58%) smoked within 5 minutes of waking suggesting a high level of dependence. On a scale of 1 to 10 the mean interest in quitting was 7.9 (SD 2.9) and the mean craving score on a scale of 0 to 4 was 1.2 (SD 1.4) suggesting slight to mild craving.

Figure 2

Table 1.Demographic, Smoking Characteristics and Treatment Provided to 513 Patients Seen by UKanQuit Service From Sept 1, 2007 to Aug 31, 2008

Characteristics	Treated (n = 513)
NOTE: For each variable, subsamples were slightly different from total sample due to missing data. Missing data were not included in the analysis. Abbreviations: AA, Afro American; SD, standard deviation. * Interest in quitting range from 0 to 10. craving range from 0 to 4.
Demographics
Mean age (SD), years	50.2 (13.6)
Male, n (%)	291 (56.7)
Ethnicity, n (%)
White	371 (73.6)
AA	107 (21.2)
Latino	18 (3.6)
Other	8 (1.6)
Referral source, n (%)
Nursing profile	477 (94.1)
Physician	5 (1.0)
Other	25 (4.9)
Smoking characteristics
Mean number of years smoked (SD)	28.9 (14.6)
Smokes within 5 minutes of waking n (%)	270 (58.3)
Mean cigarettes smoked per day (SD)	18.4 (12.6)
Mean interest in quitting (SD)*	7.9 (2.9)
Mean craving (SD)	1.2 (1.4)
Tobacco treatment provided
Counseling
Average time spent with patients (SD)	19.9 (9.1)
Received information packet, n (%)	490 (97.4)
Set goals for quitting, n (%)	352 (73.3)
Had quit plan, n (%)	151 (33.2)
Accepted fax referral to quitline, n (%)	277 (55.8)
Opted for UKanQuit counseling, n (%)	29 (5.9)
Medication
On smoking cessation medication, n (%)	133 (26.2)
Interested in receiving or changing smoking cessation medication, n (%)	132 (26.7)
Added or changed smoking cessation medication, n (%)	195 (40.5)
Discharge med, n (%)	196 (40.2)

Discussion

Among patients served by this inpatient program, interest in quitting was high but administration of inpatient medications upon admission was low (26%). Nearly all patients were provided with written materials, a majority set some form of goal for quitting or cutting down, and many developed quit plans and received assistance adjusting inpatient and/or discharge medications. After discharge, the majority of study participants made unassisted quit attempts, as utilization of medications and quitline services was suboptimal. Fax‐referral to quitline may not, on its own, fulfill guideline recommendations for post‐discharge follow‐up.

Our intent‐to‐treat quit rate was about half of what was found in Taylor et al.'s11 hospital program dissemination trial. Their program may have had better effects as it was somewhat more intensive. It included at least 1 follow‐up phone call immediately after discharge, as well as an accompanying video and relaxation audiotape or compact disc. However, differences in outcomes may also be due to large differences between the study populations. Hospitals participating in Taylor's study only conducted intervention and outcome assessment among smokers who were ready to quit, willing to enroll in a clinical trial, and willing to complete informed consent. Our intervention and outcome data included patients who agreed to speak with UKanQuit staff, regardless of readiness to quit. Our participants did not have to complete informed consent and enroll in a trial as our analyses were conducted post hoc. Our study outcomes might better reflect quit rates for a program serving all smokers, at all levels of readiness to quit, in actual hospital practices. The mean reduction in cigarette smoking among smokers who continued to smoke at 6 months' follow‐up was statistically significant. However, findings from a lung health study show that 50% or more reduction in smoking was ultimately related to successful quitting.23

Hospitalization can be considered a teachable moment for smoking cessation13 for the 6.5 million adult smokers who are hospitalized in the United States each year.4 Smokers who receive tobacco treatment during hospitalization and outpatient follow‐up treatment for at least 1 month are more likely to quit than patients who receive no treatment.5, 6

Unless tobacco treatment is explicitly delegated to other providers, physicians shoulder the responsibility of encouraging smokers to quit and prescribing smoking cessation medications. This is problematic in that physicians sometimes fail to counsel their patients about quitting smoking7, 8 or recommend outpatient follow‐up.9 Few hospitals provide comprehensive treatment. In a review of 33 studies on the prevalence of smoking care delivery in hospitals, 3 hospitals reported they provided advice to quit alone, 29 provided advice plus counseling and assistance in quitting, and 8 provided advice or prescription for cessation pharmacotherapy.9 Although post‐discharge support is a key component of effective treatment for hospitalized smokers,6 only 11 reported providing follow‐up treatment, or referral for follow‐up treatment, after discharge. Among these 11 hospitals, respondents reported they provided referral or follow‐up to 1% to 74% of their smokers, with a median percentage of 24%. The 1 study that specified the type of outpatient treatment provided reported the hospital provided the state quitline number to smokers.

Instituting a dedicated smoking cessation program may enhance inpatient treatment, outpatient follow‐up, and treatment outcomes. Two studies have found that institutional smoking cessation programs increased the likelihood that patients would receive treatment and quit compared to hospitals without dedicated programs.10, 11

Although many US hospitals are developing programs to provide systematic treatment for tobacco dependence,9 little is known regarding how programs structure their staff, enroll patients, or provide treatment to patients that smoke. Instituting tobacco treatment services usually requires policy change and system‐wide approaches with quality improvement endpoint goals.8, 1214 In the United States, elements of these services include: 1) developing a cadre of trained tobacco treatment specialists, 2) implementing hospital systems for identifying smokers and referring them to the service, 3) providing inpatient treatment based on current treatment guidelines15 and 4) providing or facilitating follow‐up treatment after discharge, often via fax‐referral to tobacco quitlines. This systematic approach is still lacking in many hospitals.

To date, few evaluations of dedicated hospital‐based smoking cessation programs have been reported in the literature.8, 11 The purpose of this study is to describe patient characteristics and outcomes of a dedicated tobacco treatment service, with paid staff, in a large academic medical center. We describe treatment protocols, profile patients served, treatments provided, and summarization of 6‐month post‐discharge outcomes for smokers referred to the UKanQuit service over a 1‐year period. We close with lessons learned on how to improve the delivery of tobacco treatment to hospitalized patients.

Methods

Program Intervention

UKanQuit staff visit patients at the bedside to deliver tobacco treatment. This consists of: (a) assessing withdrawal; (b) working with the health care team to adjust nicotine replacement to keep the patient comfortable; (c) assessing patients' interest in quitting smoking; (d) providing brief motivational intervention to patients not interested in quitting; and (e) providing assistance in quitting (developing a quit plan, arranging for medications on discharge) to patients interested in quitting (Figure 1). UKanQuit staff recommend medications based on the patients' level of dependence, history of cessation, and cessation medication preferences. The recommendation is communicated in person and by chart documentation to the medical team, usually by the nursing staff. The patients' resident or attending physician makes the final determination regarding medication provided. The hospital has nicotine replacement therapy (NRT; patch, gum, and lozenge), bupropion, and varenicline in its formulary. Patients are then offered an option of fax referral to the state tobacco quitline for follow‐up counseling. UKanQuit staff documents the services provided in the EMR via SOAR (Subjective, Objective, Assessment and Referral) notes.

Figure 1

Results

Baseline

Within the study period (September 1, 2007 to August 31, 2008), 22,624 patients were admitted to the medical center (Figure 2). A total of 4150 were current smokers (ie, smoked within the past 30 days). UKanQuit staff met with 513 (68%) of 753 patients referred to the service. Some of the reasons why 32% of referred patients were not seen by the UKanQuit staff have been described in our previous paper.17 These include patient was asleep, doctor in the room, out of bed for procedure, and unable to speak. Table 1 displays the characteristics of 513 smokers treated by UKanQuit from September 1, 2007 to August 31, 2008. Patients were predominantly white (74%) with mean age of 50 years. Slightly more than half of smokers were male (57%). They had smoked an average of 18 cigarettes per day for a mean duration of 29 years, and over half (58%) smoked within 5 minutes of waking suggesting a high level of dependence. On a scale of 1 to 10 the mean interest in quitting was 7.9 (SD 2.9) and the mean craving score on a scale of 0 to 4 was 1.2 (SD 1.4) suggesting slight to mild craving.

Figure 2

Table 1.Demographic, Smoking Characteristics and Treatment Provided to 513 Patients Seen by UKanQuit Service From Sept 1, 2007 to Aug 31, 2008

Characteristics	Treated (n = 513)
NOTE: For each variable, subsamples were slightly different from total sample due to missing data. Missing data were not included in the analysis. Abbreviations: AA, Afro American; SD, standard deviation. * Interest in quitting range from 0 to 10. craving range from 0 to 4.
Demographics
Mean age (SD), years	50.2 (13.6)
Male, n (%)	291 (56.7)
Ethnicity, n (%)
White	371 (73.6)
AA	107 (21.2)
Latino	18 (3.6)
Other	8 (1.6)
Referral source, n (%)
Nursing profile	477 (94.1)
Physician	5 (1.0)
Other	25 (4.9)
Smoking characteristics
Mean number of years smoked (SD)	28.9 (14.6)
Smokes within 5 minutes of waking n (%)	270 (58.3)
Mean cigarettes smoked per day (SD)	18.4 (12.6)
Mean interest in quitting (SD)*	7.9 (2.9)
Mean craving (SD)	1.2 (1.4)
Tobacco treatment provided
Counseling
Average time spent with patients (SD)	19.9 (9.1)
Received information packet, n (%)	490 (97.4)
Set goals for quitting, n (%)	352 (73.3)
Had quit plan, n (%)	151 (33.2)
Accepted fax referral to quitline, n (%)	277 (55.8)
Opted for UKanQuit counseling, n (%)	29 (5.9)
Medication
On smoking cessation medication, n (%)	133 (26.2)
Interested in receiving or changing smoking cessation medication, n (%)	132 (26.7)
Added or changed smoking cessation medication, n (%)	195 (40.5)
Discharge med, n (%)	196 (40.2)

Discussion

Among patients served by this inpatient program, interest in quitting was high but administration of inpatient medications upon admission was low (26%). Nearly all patients were provided with written materials, a majority set some form of goal for quitting or cutting down, and many developed quit plans and received assistance adjusting inpatient and/or discharge medications. After discharge, the majority of study participants made unassisted quit attempts, as utilization of medications and quitline services was suboptimal. Fax‐referral to quitline may not, on its own, fulfill guideline recommendations for post‐discharge follow‐up.

Our intent‐to‐treat quit rate was about half of what was found in Taylor et al.'s11 hospital program dissemination trial. Their program may have had better effects as it was somewhat more intensive. It included at least 1 follow‐up phone call immediately after discharge, as well as an accompanying video and relaxation audiotape or compact disc. However, differences in outcomes may also be due to large differences between the study populations. Hospitals participating in Taylor's study only conducted intervention and outcome assessment among smokers who were ready to quit, willing to enroll in a clinical trial, and willing to complete informed consent. Our intervention and outcome data included patients who agreed to speak with UKanQuit staff, regardless of readiness to quit. Our participants did not have to complete informed consent and enroll in a trial as our analyses were conducted post hoc. Our study outcomes might better reflect quit rates for a program serving all smokers, at all levels of readiness to quit, in actual hospital practices. The mean reduction in cigarette smoking among smokers who continued to smoke at 6 months' follow‐up was statistically significant. However, findings from a lung health study show that 50% or more reduction in smoking was ultimately related to successful quitting.23

The incidence of sudden pediatric cardiac or respiratory arrest is low.1 Most inpatient pediatric arrests appear to occur as progression of respiratory distress or shock.2 The outcome of inpatient pediatric cardiorespiratory arrests continues to be poor, emphasizing the need for early recognition and intervention. In 2006, as part of its 100,000 Lives Campaign the Institute for Healthcare Improvement recommended the implementation of Rapid Response Teams (RRT) as 1 of the strategies to reduce the number of preventable inpatient deaths.3 We reviewed all emergency response team (ERT) activations for last 13 years at The Children's Hospital in Denver, CO to assist in the development of a new RRT and to identify at risk populations, situations, and system processes the RRT should address.

This is a retrospective review of 13 years of data collection on ERT activations at The Children's Hospital in Denver, CO. We describe demographic and clinical variables, including outcomes of ERT activations at a free‐standing tertiary care children's hospital.

Background/Methods

The Children's Hospital (TCH) is a 270 inpatient bed tertiary care free standing children's hospital associated with University of Colorado at Denver Health Sciences Center. The current distribution of inpatient beds includes: 168 medical/surgical beds, 102 critical care beds (26 pediatric intensive care unit, 16 cardiac intensive care unit, and 60 neonatal intensive care unit). In 2006, nearly 10,000 patients were admitted for care at this hospital with an average inpatient stay of 6 days. TCH is a Level One Regional Trauma Center serving a catchment area of seven states, and a major transplant center for heart, solid organs, and bone marrow.

The history of the TCH Emergency Response Team dates back to 1990 when we first began to follow cardiorespiratory arrests in noncritical care areas of the hospital. In 1992, a Cardiac Or Respiratory event (COR) team including the most senior in‐house specialist was available 24 hours a day, 7 days a week to respond to all arrests within the hospital. The COR committee provided oversight and monitoring of the arrest events, including standardization of crash carts and the development of mock codes to ensure that the responding personnel were qualified in resuscitation practices. The COR team has evolved over the ensuing years, including a name change to the Emergency Response Team (ERT); now a single number activated by any medical staff is used to call the operator who activates the ERT via overhead paging and via code pager system. The 14 member Emergency Response Team consists of PICU/CICU/anesthesia/surgical fellows, ED attending, in‐house residents, PICU/CICU/ED charge nurse, nursing supervisor, resource RN, pharmacist, respiratory therapist, and a messenger.

A database has been maintained by 1 of the authors (DBH) since the inception of the COR/ERT at TCH 16 years ago. This is a retrospective review of this database of ERT activations. An ERT activation could have been triggered by any event that was felt to be emergent, life threatening, and/or needing immediate medical attention. After the event, a debriefing form was filled out about the event. Data collected included date, time, medical record number, location, primary care service, age, sex, primary and secondary diagnoses, and disposition. Data not captured in this database included initial rhythm, need for compressions, cardiac medications, defibrillation or intubation.

Analyses were performed on data collected from January 1993 through April 2007. Medical records of the documented ERT activations were reviewed for missing information and/or clarification of the events. Categories entered in Statistical Package for Social Statistics (SPSS) were similar to information included in the debriefing form: age, sex, admission diagnosis, precipitating event, percentage of admissions, acute vs. chronic diagnosis, winter vs. nonwinter months (October‐March/April‐September), day (6 am‐6 pm) and night (6 pm‐6 am) shifts, survival of ERT activation, survival to discharge, and primary attending service. Data were analyzed using SPSS 16.0 (2007, Chicago, IL). The study was approved by the Colorado Multiple Institutional Review Board.

Results/Conclusion

There were 1537 ERT activations in the database. A total of 203 ERT activations were eliminated from the database: 177 were eliminated from analysis because of missing age, admission diagnosis or time of day of activation, and 26 were ERT activations that had been triggered on adult visitors or adult employees. The remaining 1334 ERT activations were included for analysis.

Table 1 shows the demographics of the patients. The median age was 1.8 years, with a range of 0 to 29 years. A total of 39%(511) of all ERT activations occurred in patients under the age of 1 year with the highest incidence between 1 month and 1 year. Overall, the children at highest risk were males less than 1 year of age with a chronic diagnosis. In addition, time of day and time of year of ERT activations were analyzed as shown. There was no statistical difference between nonwinter (April‐September) and winter (October‐March) months. Statistically, there were significantly more ERT activations during day shifts (6 am‐6 pm) as compared to night shifts (P < 0.001).

The most common admission diagnosis (Table 2) and underlying chronic condition was cardiac disease; other common admission diagnoses were infectious disease, trauma, and pulmonary disease. The medical categories of admission diagnosis included congenital/metabolic (39%), gastrointestinal (29%), renal (18%), rheumatology (4%), toxicology (4%), psychiatry (3%), endocrine (2%), and allergy (1%). The surgery category of admission diagnosis included otolaryngology (63%), orthopedics (28%), urology (4%), dental (4%), and ophthalmology (1%).

Finally, the patients' survival rate after an ERT itself was to be 90% (Table 3), with an overall survival rate to discharge of 78% (Table 4). Survival rate to discharge of those patients who survived the ERT event was 87%. Two patients were missing survival event data and 137 patients were missing survival to discharge data.

Discussion

We present a retrospective review of 1334 emergency response team activations over 13 years at an academic free‐standing tertiary care children's hospital. In keeping with previous reports, we found that children less than 1 year of age were at the highest risk for activation of the emergency response team.1, 4, 5

The National Registry for CardioPulmonary Resuscitation (NRCPR) database cite respiratory failure (asphyxia) and circulatory shock (ischemia) as the most common causes of in‐hospital cardiac arrests.1 Additionally, more than half of pediatric patients that experience a cardiopulmonary arrest have an underlying chronic illness.1, 2, 4, 5, 6 These are similar to our findings that chronic pulmonary and cardiac diseases were among the most frequent admission diagnosis.

Unlike Peberdy et al.,7we did not find an increase in the number of emergency response team activations at night or on weekends. Instead, we found that an ERT activation was more likely to be requested during the day shifts (6 am‐6 pm) which is a similar to that reported by Jones et al.8 Similarly, Jones et al.8 reported that the hourly rate of their medical emergency team activation was greater during the time between 8 am and 6 pm.

Our overall survival rate (78%) to discharge after an ERT event was much higher than what has been reported by Topjian et al.9 (25%). This likely reflects our inclusion of all emergency response team activations, not just apneic and asystolic arrests. The improved survival rate may also be influenced by the 24/7 presence of pediatric fellows, residents and surgeons in our hospital, which has been associated with improved 24‐hour survival for children receiving in‐hospital cardiopulmonary resuscitation.10

There are several limitations to our data collection and this report. This was a retrospective review and as previously noted some medical details were absent resulting in the exclusion of some cases. We also found that the original debriefing form which was used as the basis for the database did not include some important clinical variable, such as vital signs, more detailed events such as rhythm, medications used, and deficits or changes in baseline function. We suggest including multiple variables in a future multicenter study of pediatric RRT's: facility, admitting service, admission diagnosis, age (chronologic and gestational), sex, how long patient has been in hospital prior to the event, past medical history, vitals signs before and after activation, time/date, location, any precipitating events and actions taken by the RRT (eg, cardiopulmonary resuscitation, defibrillation, emergent intubation, and other emergent interventions), medications, survival of event, survival to discharge, deficits or changes in baseline function after the event and to discharge. In particular, information about history of prematurity would have been helpful in assessing further risk factors. Analysis of survival with or without significant deficits or changes in baseline function would be another useful outcome measure. Figure 1 is an example of a debriefing form a multi‐institutional study or hospitalist led quality improvement project may use to collect this data.

Figure 1

We were able to identify a population of higher‐risk patients (less than 1 year of age with comorbidities and an admission diagnosis of cardiac or respiratory disease) to focus our educational efforts on earlier recognition of patient deterioration for both inpatient ward staff and RRT responders. These findings may assist in future quality assurance issues such as patient placement and early ICU admissions depending on age, chronic conditions, and/or admission diagnosis. Future directions should include multi‐center study of RRT to improve external validity. In addition, a more careful analysis of events surrounding the activation, including incorporating such tools as the Pediatric Early Warning (PEW) Score,11 may further assist hospitals and practitioners identify hospitalized children at risk for deterioration on the inpatient ward.

The incidence of sudden pediatric cardiac or respiratory arrest is low.1 Most inpatient pediatric arrests appear to occur as progression of respiratory distress or shock.2 The outcome of inpatient pediatric cardiorespiratory arrests continues to be poor, emphasizing the need for early recognition and intervention. In 2006, as part of its 100,000 Lives Campaign the Institute for Healthcare Improvement recommended the implementation of Rapid Response Teams (RRT) as 1 of the strategies to reduce the number of preventable inpatient deaths.3 We reviewed all emergency response team (ERT) activations for last 13 years at The Children's Hospital in Denver, CO to assist in the development of a new RRT and to identify at risk populations, situations, and system processes the RRT should address.

This is a retrospective review of 13 years of data collection on ERT activations at The Children's Hospital in Denver, CO. We describe demographic and clinical variables, including outcomes of ERT activations at a free‐standing tertiary care children's hospital.

Background/Methods

The Children's Hospital (TCH) is a 270 inpatient bed tertiary care free standing children's hospital associated with University of Colorado at Denver Health Sciences Center. The current distribution of inpatient beds includes: 168 medical/surgical beds, 102 critical care beds (26 pediatric intensive care unit, 16 cardiac intensive care unit, and 60 neonatal intensive care unit). In 2006, nearly 10,000 patients were admitted for care at this hospital with an average inpatient stay of 6 days. TCH is a Level One Regional Trauma Center serving a catchment area of seven states, and a major transplant center for heart, solid organs, and bone marrow.

The history of the TCH Emergency Response Team dates back to 1990 when we first began to follow cardiorespiratory arrests in noncritical care areas of the hospital. In 1992, a Cardiac Or Respiratory event (COR) team including the most senior in‐house specialist was available 24 hours a day, 7 days a week to respond to all arrests within the hospital. The COR committee provided oversight and monitoring of the arrest events, including standardization of crash carts and the development of mock codes to ensure that the responding personnel were qualified in resuscitation practices. The COR team has evolved over the ensuing years, including a name change to the Emergency Response Team (ERT); now a single number activated by any medical staff is used to call the operator who activates the ERT via overhead paging and via code pager system. The 14 member Emergency Response Team consists of PICU/CICU/anesthesia/surgical fellows, ED attending, in‐house residents, PICU/CICU/ED charge nurse, nursing supervisor, resource RN, pharmacist, respiratory therapist, and a messenger.

A database has been maintained by 1 of the authors (DBH) since the inception of the COR/ERT at TCH 16 years ago. This is a retrospective review of this database of ERT activations. An ERT activation could have been triggered by any event that was felt to be emergent, life threatening, and/or needing immediate medical attention. After the event, a debriefing form was filled out about the event. Data collected included date, time, medical record number, location, primary care service, age, sex, primary and secondary diagnoses, and disposition. Data not captured in this database included initial rhythm, need for compressions, cardiac medications, defibrillation or intubation.

Analyses were performed on data collected from January 1993 through April 2007. Medical records of the documented ERT activations were reviewed for missing information and/or clarification of the events. Categories entered in Statistical Package for Social Statistics (SPSS) were similar to information included in the debriefing form: age, sex, admission diagnosis, precipitating event, percentage of admissions, acute vs. chronic diagnosis, winter vs. nonwinter months (October‐March/April‐September), day (6 am‐6 pm) and night (6 pm‐6 am) shifts, survival of ERT activation, survival to discharge, and primary attending service. Data were analyzed using SPSS 16.0 (2007, Chicago, IL). The study was approved by the Colorado Multiple Institutional Review Board.

Results/Conclusion

There were 1537 ERT activations in the database. A total of 203 ERT activations were eliminated from the database: 177 were eliminated from analysis because of missing age, admission diagnosis or time of day of activation, and 26 were ERT activations that had been triggered on adult visitors or adult employees. The remaining 1334 ERT activations were included for analysis.

Table 1 shows the demographics of the patients. The median age was 1.8 years, with a range of 0 to 29 years. A total of 39%(511) of all ERT activations occurred in patients under the age of 1 year with the highest incidence between 1 month and 1 year. Overall, the children at highest risk were males less than 1 year of age with a chronic diagnosis. In addition, time of day and time of year of ERT activations were analyzed as shown. There was no statistical difference between nonwinter (April‐September) and winter (October‐March) months. Statistically, there were significantly more ERT activations during day shifts (6 am‐6 pm) as compared to night shifts (P < 0.001).

The most common admission diagnosis (Table 2) and underlying chronic condition was cardiac disease; other common admission diagnoses were infectious disease, trauma, and pulmonary disease. The medical categories of admission diagnosis included congenital/metabolic (39%), gastrointestinal (29%), renal (18%), rheumatology (4%), toxicology (4%), psychiatry (3%), endocrine (2%), and allergy (1%). The surgery category of admission diagnosis included otolaryngology (63%), orthopedics (28%), urology (4%), dental (4%), and ophthalmology (1%).

Finally, the patients' survival rate after an ERT itself was to be 90% (Table 3), with an overall survival rate to discharge of 78% (Table 4). Survival rate to discharge of those patients who survived the ERT event was 87%. Two patients were missing survival event data and 137 patients were missing survival to discharge data.

Discussion

We present a retrospective review of 1334 emergency response team activations over 13 years at an academic free‐standing tertiary care children's hospital. In keeping with previous reports, we found that children less than 1 year of age were at the highest risk for activation of the emergency response team.1, 4, 5

The National Registry for CardioPulmonary Resuscitation (NRCPR) database cite respiratory failure (asphyxia) and circulatory shock (ischemia) as the most common causes of in‐hospital cardiac arrests.1 Additionally, more than half of pediatric patients that experience a cardiopulmonary arrest have an underlying chronic illness.1, 2, 4, 5, 6 These are similar to our findings that chronic pulmonary and cardiac diseases were among the most frequent admission diagnosis.

Unlike Peberdy et al.,7we did not find an increase in the number of emergency response team activations at night or on weekends. Instead, we found that an ERT activation was more likely to be requested during the day shifts (6 am‐6 pm) which is a similar to that reported by Jones et al.8 Similarly, Jones et al.8 reported that the hourly rate of their medical emergency team activation was greater during the time between 8 am and 6 pm.

Our overall survival rate (78%) to discharge after an ERT event was much higher than what has been reported by Topjian et al.9 (25%). This likely reflects our inclusion of all emergency response team activations, not just apneic and asystolic arrests. The improved survival rate may also be influenced by the 24/7 presence of pediatric fellows, residents and surgeons in our hospital, which has been associated with improved 24‐hour survival for children receiving in‐hospital cardiopulmonary resuscitation.10

There are several limitations to our data collection and this report. This was a retrospective review and as previously noted some medical details were absent resulting in the exclusion of some cases. We also found that the original debriefing form which was used as the basis for the database did not include some important clinical variable, such as vital signs, more detailed events such as rhythm, medications used, and deficits or changes in baseline function. We suggest including multiple variables in a future multicenter study of pediatric RRT's: facility, admitting service, admission diagnosis, age (chronologic and gestational), sex, how long patient has been in hospital prior to the event, past medical history, vitals signs before and after activation, time/date, location, any precipitating events and actions taken by the RRT (eg, cardiopulmonary resuscitation, defibrillation, emergent intubation, and other emergent interventions), medications, survival of event, survival to discharge, deficits or changes in baseline function after the event and to discharge. In particular, information about history of prematurity would have been helpful in assessing further risk factors. Analysis of survival with or without significant deficits or changes in baseline function would be another useful outcome measure. Figure 1 is an example of a debriefing form a multi‐institutional study or hospitalist led quality improvement project may use to collect this data.

Figure 1

We were able to identify a population of higher‐risk patients (less than 1 year of age with comorbidities and an admission diagnosis of cardiac or respiratory disease) to focus our educational efforts on earlier recognition of patient deterioration for both inpatient ward staff and RRT responders. These findings may assist in future quality assurance issues such as patient placement and early ICU admissions depending on age, chronic conditions, and/or admission diagnosis. Future directions should include multi‐center study of RRT to improve external validity. In addition, a more careful analysis of events surrounding the activation, including incorporating such tools as the Pediatric Early Warning (PEW) Score,11 may further assist hospitals and practitioners identify hospitalized children at risk for deterioration on the inpatient ward.

The incidence of sudden pediatric cardiac or respiratory arrest is low.1 Most inpatient pediatric arrests appear to occur as progression of respiratory distress or shock.2 The outcome of inpatient pediatric cardiorespiratory arrests continues to be poor, emphasizing the need for early recognition and intervention. In 2006, as part of its 100,000 Lives Campaign the Institute for Healthcare Improvement recommended the implementation of Rapid Response Teams (RRT) as 1 of the strategies to reduce the number of preventable inpatient deaths.3 We reviewed all emergency response team (ERT) activations for last 13 years at The Children's Hospital in Denver, CO to assist in the development of a new RRT and to identify at risk populations, situations, and system processes the RRT should address.

This is a retrospective review of 13 years of data collection on ERT activations at The Children's Hospital in Denver, CO. We describe demographic and clinical variables, including outcomes of ERT activations at a free‐standing tertiary care children's hospital.

Background/Methods

The Children's Hospital (TCH) is a 270 inpatient bed tertiary care free standing children's hospital associated with University of Colorado at Denver Health Sciences Center. The current distribution of inpatient beds includes: 168 medical/surgical beds, 102 critical care beds (26 pediatric intensive care unit, 16 cardiac intensive care unit, and 60 neonatal intensive care unit). In 2006, nearly 10,000 patients were admitted for care at this hospital with an average inpatient stay of 6 days. TCH is a Level One Regional Trauma Center serving a catchment area of seven states, and a major transplant center for heart, solid organs, and bone marrow.

The history of the TCH Emergency Response Team dates back to 1990 when we first began to follow cardiorespiratory arrests in noncritical care areas of the hospital. In 1992, a Cardiac Or Respiratory event (COR) team including the most senior in‐house specialist was available 24 hours a day, 7 days a week to respond to all arrests within the hospital. The COR committee provided oversight and monitoring of the arrest events, including standardization of crash carts and the development of mock codes to ensure that the responding personnel were qualified in resuscitation practices. The COR team has evolved over the ensuing years, including a name change to the Emergency Response Team (ERT); now a single number activated by any medical staff is used to call the operator who activates the ERT via overhead paging and via code pager system. The 14 member Emergency Response Team consists of PICU/CICU/anesthesia/surgical fellows, ED attending, in‐house residents, PICU/CICU/ED charge nurse, nursing supervisor, resource RN, pharmacist, respiratory therapist, and a messenger.

A database has been maintained by 1 of the authors (DBH) since the inception of the COR/ERT at TCH 16 years ago. This is a retrospective review of this database of ERT activations. An ERT activation could have been triggered by any event that was felt to be emergent, life threatening, and/or needing immediate medical attention. After the event, a debriefing form was filled out about the event. Data collected included date, time, medical record number, location, primary care service, age, sex, primary and secondary diagnoses, and disposition. Data not captured in this database included initial rhythm, need for compressions, cardiac medications, defibrillation or intubation.

Analyses were performed on data collected from January 1993 through April 2007. Medical records of the documented ERT activations were reviewed for missing information and/or clarification of the events. Categories entered in Statistical Package for Social Statistics (SPSS) were similar to information included in the debriefing form: age, sex, admission diagnosis, precipitating event, percentage of admissions, acute vs. chronic diagnosis, winter vs. nonwinter months (October‐March/April‐September), day (6 am‐6 pm) and night (6 pm‐6 am) shifts, survival of ERT activation, survival to discharge, and primary attending service. Data were analyzed using SPSS 16.0 (2007, Chicago, IL). The study was approved by the Colorado Multiple Institutional Review Board.

Results/Conclusion

There were 1537 ERT activations in the database. A total of 203 ERT activations were eliminated from the database: 177 were eliminated from analysis because of missing age, admission diagnosis or time of day of activation, and 26 were ERT activations that had been triggered on adult visitors or adult employees. The remaining 1334 ERT activations were included for analysis.

Table 1 shows the demographics of the patients. The median age was 1.8 years, with a range of 0 to 29 years. A total of 39%(511) of all ERT activations occurred in patients under the age of 1 year with the highest incidence between 1 month and 1 year. Overall, the children at highest risk were males less than 1 year of age with a chronic diagnosis. In addition, time of day and time of year of ERT activations were analyzed as shown. There was no statistical difference between nonwinter (April‐September) and winter (October‐March) months. Statistically, there were significantly more ERT activations during day shifts (6 am‐6 pm) as compared to night shifts (P < 0.001).

The most common admission diagnosis (Table 2) and underlying chronic condition was cardiac disease; other common admission diagnoses were infectious disease, trauma, and pulmonary disease. The medical categories of admission diagnosis included congenital/metabolic (39%), gastrointestinal (29%), renal (18%), rheumatology (4%), toxicology (4%), psychiatry (3%), endocrine (2%), and allergy (1%). The surgery category of admission diagnosis included otolaryngology (63%), orthopedics (28%), urology (4%), dental (4%), and ophthalmology (1%).

Finally, the patients' survival rate after an ERT itself was to be 90% (Table 3), with an overall survival rate to discharge of 78% (Table 4). Survival rate to discharge of those patients who survived the ERT event was 87%. Two patients were missing survival event data and 137 patients were missing survival to discharge data.

Discussion

We present a retrospective review of 1334 emergency response team activations over 13 years at an academic free‐standing tertiary care children's hospital. In keeping with previous reports, we found that children less than 1 year of age were at the highest risk for activation of the emergency response team.1, 4, 5

The National Registry for CardioPulmonary Resuscitation (NRCPR) database cite respiratory failure (asphyxia) and circulatory shock (ischemia) as the most common causes of in‐hospital cardiac arrests.1 Additionally, more than half of pediatric patients that experience a cardiopulmonary arrest have an underlying chronic illness.1, 2, 4, 5, 6 These are similar to our findings that chronic pulmonary and cardiac diseases were among the most frequent admission diagnosis.

Unlike Peberdy et al.,7we did not find an increase in the number of emergency response team activations at night or on weekends. Instead, we found that an ERT activation was more likely to be requested during the day shifts (6 am‐6 pm) which is a similar to that reported by Jones et al.8 Similarly, Jones et al.8 reported that the hourly rate of their medical emergency team activation was greater during the time between 8 am and 6 pm.

Our overall survival rate (78%) to discharge after an ERT event was much higher than what has been reported by Topjian et al.9 (25%). This likely reflects our inclusion of all emergency response team activations, not just apneic and asystolic arrests. The improved survival rate may also be influenced by the 24/7 presence of pediatric fellows, residents and surgeons in our hospital, which has been associated with improved 24‐hour survival for children receiving in‐hospital cardiopulmonary resuscitation.10

There are several limitations to our data collection and this report. This was a retrospective review and as previously noted some medical details were absent resulting in the exclusion of some cases. We also found that the original debriefing form which was used as the basis for the database did not include some important clinical variable, such as vital signs, more detailed events such as rhythm, medications used, and deficits or changes in baseline function. We suggest including multiple variables in a future multicenter study of pediatric RRT's: facility, admitting service, admission diagnosis, age (chronologic and gestational), sex, how long patient has been in hospital prior to the event, past medical history, vitals signs before and after activation, time/date, location, any precipitating events and actions taken by the RRT (eg, cardiopulmonary resuscitation, defibrillation, emergent intubation, and other emergent interventions), medications, survival of event, survival to discharge, deficits or changes in baseline function after the event and to discharge. In particular, information about history of prematurity would have been helpful in assessing further risk factors. Analysis of survival with or without significant deficits or changes in baseline function would be another useful outcome measure. Figure 1 is an example of a debriefing form a multi‐institutional study or hospitalist led quality improvement project may use to collect this data.

Figure 1

We were able to identify a population of higher‐risk patients (less than 1 year of age with comorbidities and an admission diagnosis of cardiac or respiratory disease) to focus our educational efforts on earlier recognition of patient deterioration for both inpatient ward staff and RRT responders. These findings may assist in future quality assurance issues such as patient placement and early ICU admissions depending on age, chronic conditions, and/or admission diagnosis. Future directions should include multi‐center study of RRT to improve external validity. In addition, a more careful analysis of events surrounding the activation, including incorporating such tools as the Pediatric Early Warning (PEW) Score,11 may further assist hospitals and practitioners identify hospitalized children at risk for deterioration on the inpatient ward.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.