Nancy L. Novotny, PhD

During the transition from inpatient to outpatient care, patients are vulnerable to adverse events.1 Poor communication between hospital personnel and either the patient or the outpatient primary care physician has been associated with preventable or ameliorable adverse events after discharge.1 Systematic reviews confirm that discharge communication is often delayed, inaccurate, or ineffective.2, 3

Discharge communication failures may occur if hospital processes rely on dictated discharge summaries.2 For several reasons, discharge summaries are inadequate for communication. Most patients complete their initial posthospital clinic visit before their primary care physician receives the discharge summary.4 For many patients, the discharge summary is unavailable for all posthospital visits.4 Discharge summaries often fail as communication because they are not generated or transmitted.4

Recommendations to improve discharge communication include the use of health information technology.2, 5 The benefits of computer‐generated discharge summaries include decreases in delivery time for discharge communications.2 The benefits of computerized physician order entry (CPOE) include reduction of medical errors.6 These theoretical benefits create a rationale for clinical trials to measure improvements after discharge software applications with CPOE.5

In an effort to improve discharge communication and clinically relevant outcomes, we performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The clustered design followed recommendations from a systematic review of discharge interventions.3 We applied our research intervention at the physician level and measured outcomes at the patient level. Our objective was to assess the benefit of discharge software with CPOE vs. usual care when used to discharge patients at high risk for repeat admission. In a previous work, we reported that discharge software did not reduce rates of hospital readmission, emergency department visits, or postdischarge adverse events due to medical management.7 In the present article, we compare secondary outcomes after the research intervention: perceptions of the discharge from the perspectives of patients, primary care physicians, and hospital physicians.

Methods

The trial design was a cluster randomized, controlled trial. The setting was the postdischarge environment following index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). We approached 6884 patients during their index hospitalization. After excluding 6253 ineligible patients, we enrolled 631 willing patients (Supplementary Appendix). As depicted in Figure 1, the most common reason for ineligibility occurred for patients with Pra score <0.40 (2168/6253 exclusions; 34.7%). We followed 631 patients who received the discharge intervention (Figure 1). There was no differential dropout between the interventions. Protocol deviations were rare, 0.5% (3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly assigned hospital physicians and their patients are in Table 1. Most of the hospital physicians were residents in the first year of postgraduate training.

Figure 1

We assessed the patient's perception of discharge preparedness. One week after discharge, research personnel interviewed 92.4% (292/316) of patients in the discharge software group and 92.4% (291/315) in the usual care group. The mean (SD) B‐PREPARED scores for discharge preparedness were 17.7 (4.1) in the discharge software group and 17.2 (4.0) in the usual care group. In the generalized estimating equation that accounted for potential clustering within hospital physicians, the parameter estimate for the intervention variable coefficient was small but significant (P = 0.042; Table 2). Patients in the discharge software group had slightly better perceptions of their discharge preparedness.

Another outcome was the patient's satisfaction with information about discharge medications (Table 2). One week after discharge, mean (SD) SIMS scores for satisfaction were 12.3 (4.8) in the discharge software group and 12.1 (4.6) in the usual care group. The generalized estimating equation revealed an insignificant coefficient for the intervention variable (P = 0.567; Table 2).

We assessed the outpatient physician perception of the discharge with a questionnaire sent 10 days after discharge. We received 496 out of 631 questionnaires (78.6%) from outpatient practitioners and the median response time was 19 days after the date of discharge. The practitioner specialty was internal medicine for 38.9% (193/496), family medicine for 27.2% (135/496), medicine‐pediatrics for 27.0% (134/496), advance practice nurse for 4.4% (22/496), other physician specialties for 2.0% (10/496), and physician assistant for 0.4% (2/496). We excluded 18 questionnaires from analysis because outpatient practitioners failed to answer 2 or more items in the Modified Physician‐PREPARED scale. When we compared baseline characteristics for patients who had complete questionnaires vs. patients with nonrespondent or excluded questionnaires, we found no significant differences (data available upon request). Among the discharge software group, 72.2% (228/316) of patients had complete questionnaires from their outpatient physicians. The response rate with complete questionnaires was 79.4% (250/315) of patients assigned to usual care. On the Modified Physician‐PREPARED scale, the mean (SD) scores were 17.2 (3.8) for the discharge software group and 16.5 (3.9) for the usual care group. The parameter estimate from the generalized estimating equation was significant (P = 0.027; Table 2). Outpatient physicians had slightly better perception of discharge quality for patients assigned to discharge software.

In the questionnaire sent to outpatient practitioners, we requested additional information about discharge communication. When asked about timeliness, outpatient physicians perceived no faster communication with the discharge software (Table 3). We asked about the media for discharge information exchange. It was uncommon for community physicians to receive discharge information via electronic mail (Table 3). Outpatient physicians acknowledged receipt of a minority of facsimile transmissions with no significant difference between discharge software vs. usual care (Table 3). Investigators documented facsimile transmission of the output from the discharge software to outpatient practitioners. Transmission occurred on the first business day after discharge. Despite the documentation of all facsimile transmissions, only 23.4% of patients assigned to discharge software had community practitioners who acknowledged receipt.

In exploratory analyses, we evaluated the effect of hospital physician level of training. We wondered if discharging physician experience or seniority affected perceptions of patients or primary care physicians. We entered level of training as a covariate in generalized estimating equations. When patient perception of discharge preparedness (B‐PREPARED) was the dependent variable, then physician level of training had a nonsignificant coefficient (P > 0.219). Likewise, physician level of training was nonsignificant in models of patient satisfaction with medication information, SIMS (P > 0.068), and outpatient physician perception, Modified Physician‐PREPARED (P > 0.177). We concluded that physician level of training had no influence on the patient‐level outcomes assessed in our study.

We compared the satisfaction of hospital physicians who used the discharge software and the usual care discharge. The proportions of hospital physicians who returned questionnaires were 85.7% (30/35) in the discharge software group and 97% (34/35) in the usual care group. After using their assigned discharge process for at least 6 months, discharge software users had mean (SD) satisfaction 7.4 (1.4) vs. 7.9 (1.4) for usual care physicians (difference = 0.5; 95% CI = 0.2‐1.3; P = 0.129). The effort for discharge software users was more difficult than the effort for usual care (mean [SD] effort = 6.5 [1.9] vs. 7.9 [2.1], respectively). The mean difference in effort was significant (difference = 1.4; 95% CI = 0.3‐2.4; P = 0.011). We reviewed free‐text comments on hospital physician questionnaires. The common theme was software users spent more time to complete discharges. We did not perform time‐motion assessments so we cannot confirm or refute these impressions. Even though hospital physicians found the discharge software significantly more difficult, they did not report a significant decrease in their satisfaction between the 2 discharge interventions.

The cluster design of our trial assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for B‐PREPARED, SIMS, and Modified Physician‐PREPARED. For all of these outcome variables, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on CIs for intervention coefficients (Table 2).

We evaluated the adequacy of the blind for outcome assessors who interviewed patients for B‐PREPARED and SIMS. The guesses of outcomes assessors were unrelated to true intervention assignment (P = 0.253). We interpreted the blind as adequate.

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software vs. usual care discharge. The discharge software incorporated the ASTM (American Society for Testing and Material) Continuity of Care Record (CCR) standards.19 The CCR is a patient health summary standard with widespread support from medical and specialty organizations. The rationale for the CCR was the need for continuity of care from 1 provider or practitioner to any other practitioner. Our discharge software had the same rationale as the CCR and included a subset of the clinical content specified by the CCR. Like the CCR, our discharge software produced concise reports, and emphasized a brief, postdischarge, care plan. Since we included clinical data elements recommended by the CCR, we hypothesized our discharge software would produce clinically relevant improvements.

Our discharge software also implemented elements of high‐quality discharge planning and communication endorsed by the Society of Hospital Medicine.20 For example, the discharge software produced a legible, typed, discharge plan for the patient or caregiver with medication instructions, follow‐up tests, studies, and appointments. The discharge software generated a discharge summary for the outpatient primary care physician and other clinicians who provided postdischarge care. The summary included discharge diagnoses, key findings and test results, follow‐up appointments, pending diagnostic tests, documentation of patient education, reconciled medication list, and contact information for the hospital physician. The discharge software compiled data for purposes of benchmarking, measurement, and continuous quality improvement. We thought our implementation of discharge software would lead to improved outcomes.

Despite our deployment of recommended strategies, we detected only small increases in patient perceptions of discharge preparedness. We do not know if small changes in B‐PREPARED values were clinically important. We found no improvement in patient satisfaction with medication information. Our results are consistent with systematic reviews that revealed limited benefit of interventions other than discharge planning with postdischarge support.21 Since our discharge software was added to robust discharge planning and support, we possibly had limited ability to detect benefit unless the intervention had a large effect size.

Our discharge software caused a small increase in positive perception reported by outpatient physicians. Small changes in the Modified Physician‐PREPARED had uncertain clinical relevance. Potential delays imposed by our distribution method may have contributed to our findings. Output from our discharge software went to community physicians via facsimile transmission with backup copies via standard U.S. mail. Our distribution system responded to several realities. Most community physicians in our area had no access to interoperable electronic medical records or secured e‐mail. In addition, electronic transmittal of prescriptions was not commonplace. Our discharge intervention did not control the flow of information inside the offices of outpatient physicians. We did not know if our facsimile transmissions joined piles of unread laboratory and imaging reports on the desks of busy primary care physicians. Despite the limited technology available to community physicians, they perceived communication generated by the software to be an improvement over the handwritten process. Our results support previous studies in which physicians preferred computer‐generated discharge summaries and summaries in standardized formats.2224

One of the limitations of our trial design was the unmasked intervention. Hospital physicians assigned to usual care might have improved their handwritten and verbal discharge communication after observation of their colleagues assigned to discharge software. This phenomenon is encountered in unmasked trials and is called contamination. We attempted to minimize contamination when we blocked usual care physicians from access to the discharge software. However, we could not eliminate cross‐talk among unmasked hospital physicians who worked together in close proximity during 27 months of patient enrollment. Some contamination was inevitable. When contamination occurred, there was bias toward the null (increased type II error).

Another limitation was the large proportion of hospital physicians in the first year of postgraduate training. There was a potential for variance from multilevel clusters with patient‐level outcomes clustered within first‐year hospital physicians who were clustered within teams supervised by senior resident or attending physicians. Our results argued against hierarchical clusters because intracluster correlation coefficients were negligible. Furthermore, our exploratory analysis suggested physician training level had no influence on patient outcomes measured in our study. We speculate the highly structured discharge process for both usual care and software minimized variance attributable to physician training level.

The research intervention in our trial was a stand‐alone software application. The discharge software did not integrate with the hospital electronic medical record. Consequently, hospital physician users had to reenter patient demographic data and prescription data that already existed in the electronic record. Data reentry probably caused hospital physicians to attribute greater effort to the discharge software.

In our study, hospital physicians incorporated discharge software with CPOE into their clinical workflow without deterioration in their satisfaction. Our experience may inform the decisions of hospital personnel who design health information systems. When designing discharge functions, developers should consider medication reconciliation and the standards of the CCR.19 Modules within the discharge software would likely be more efficient with prepopulated data from the electronic record. Then users could shift their work from data entry to data verification and possibly mitigate their perceived effort. Software developers may wish to explore options for data transmission to community physicians: secure e‐mail, automated fax servers, and direct digital file transfer. Future studies should test the acceptability of discharge functions incorporated within electronic health records with robust clinical decision support.

Our results apply to a population of adults of all ages with high risk for readmission. The results may not generalize to children, surgical patients, or people with low risk for readmission. All of the patients in our study were discharged to home. The exclusion of other discharge destinations helped us to enroll a homogenous cohort. However, the exclusion criteria did not allow us to generalize our results to patients discharged to nursing homes, inpatient rehabilitation units, or other acute care facilities. We designed the intervention to apply to the hospitalist model, in which responsibility for patient care transitions to a different physician after discharge. The results of our study do not apply when the inpatient and outpatient physician are the same. Since we enrolled general internal medicine hospital physicians, our results may not generalize to care provided by other specialists.

Conclusions

A discharge software application with CPOE improved perceptions of the hospital discharge process for patients and their outpatient physicians. When compared to the handwritten discharge process, the improvements were small in magnitude. Hospital physicians who used the discharge software reported more effort but otherwise no decrement in their satisfaction with the discharge process.

During the transition from inpatient to outpatient care, patients are vulnerable to adverse events.1 Poor communication between hospital personnel and either the patient or the outpatient primary care physician has been associated with preventable or ameliorable adverse events after discharge.1 Systematic reviews confirm that discharge communication is often delayed, inaccurate, or ineffective.2, 3

Discharge communication failures may occur if hospital processes rely on dictated discharge summaries.2 For several reasons, discharge summaries are inadequate for communication. Most patients complete their initial posthospital clinic visit before their primary care physician receives the discharge summary.4 For many patients, the discharge summary is unavailable for all posthospital visits.4 Discharge summaries often fail as communication because they are not generated or transmitted.4

Recommendations to improve discharge communication include the use of health information technology.2, 5 The benefits of computer‐generated discharge summaries include decreases in delivery time for discharge communications.2 The benefits of computerized physician order entry (CPOE) include reduction of medical errors.6 These theoretical benefits create a rationale for clinical trials to measure improvements after discharge software applications with CPOE.5

In an effort to improve discharge communication and clinically relevant outcomes, we performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The clustered design followed recommendations from a systematic review of discharge interventions.3 We applied our research intervention at the physician level and measured outcomes at the patient level. Our objective was to assess the benefit of discharge software with CPOE vs. usual care when used to discharge patients at high risk for repeat admission. In a previous work, we reported that discharge software did not reduce rates of hospital readmission, emergency department visits, or postdischarge adverse events due to medical management.7 In the present article, we compare secondary outcomes after the research intervention: perceptions of the discharge from the perspectives of patients, primary care physicians, and hospital physicians.

Methods

The trial design was a cluster randomized, controlled trial. The setting was the postdischarge environment following index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). We approached 6884 patients during their index hospitalization. After excluding 6253 ineligible patients, we enrolled 631 willing patients (Supplementary Appendix). As depicted in Figure 1, the most common reason for ineligibility occurred for patients with Pra score <0.40 (2168/6253 exclusions; 34.7%). We followed 631 patients who received the discharge intervention (Figure 1). There was no differential dropout between the interventions. Protocol deviations were rare, 0.5% (3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly assigned hospital physicians and their patients are in Table 1. Most of the hospital physicians were residents in the first year of postgraduate training.

Figure 1

We assessed the patient's perception of discharge preparedness. One week after discharge, research personnel interviewed 92.4% (292/316) of patients in the discharge software group and 92.4% (291/315) in the usual care group. The mean (SD) B‐PREPARED scores for discharge preparedness were 17.7 (4.1) in the discharge software group and 17.2 (4.0) in the usual care group. In the generalized estimating equation that accounted for potential clustering within hospital physicians, the parameter estimate for the intervention variable coefficient was small but significant (P = 0.042; Table 2). Patients in the discharge software group had slightly better perceptions of their discharge preparedness.

Another outcome was the patient's satisfaction with information about discharge medications (Table 2). One week after discharge, mean (SD) SIMS scores for satisfaction were 12.3 (4.8) in the discharge software group and 12.1 (4.6) in the usual care group. The generalized estimating equation revealed an insignificant coefficient for the intervention variable (P = 0.567; Table 2).

We assessed the outpatient physician perception of the discharge with a questionnaire sent 10 days after discharge. We received 496 out of 631 questionnaires (78.6%) from outpatient practitioners and the median response time was 19 days after the date of discharge. The practitioner specialty was internal medicine for 38.9% (193/496), family medicine for 27.2% (135/496), medicine‐pediatrics for 27.0% (134/496), advance practice nurse for 4.4% (22/496), other physician specialties for 2.0% (10/496), and physician assistant for 0.4% (2/496). We excluded 18 questionnaires from analysis because outpatient practitioners failed to answer 2 or more items in the Modified Physician‐PREPARED scale. When we compared baseline characteristics for patients who had complete questionnaires vs. patients with nonrespondent or excluded questionnaires, we found no significant differences (data available upon request). Among the discharge software group, 72.2% (228/316) of patients had complete questionnaires from their outpatient physicians. The response rate with complete questionnaires was 79.4% (250/315) of patients assigned to usual care. On the Modified Physician‐PREPARED scale, the mean (SD) scores were 17.2 (3.8) for the discharge software group and 16.5 (3.9) for the usual care group. The parameter estimate from the generalized estimating equation was significant (P = 0.027; Table 2). Outpatient physicians had slightly better perception of discharge quality for patients assigned to discharge software.

In the questionnaire sent to outpatient practitioners, we requested additional information about discharge communication. When asked about timeliness, outpatient physicians perceived no faster communication with the discharge software (Table 3). We asked about the media for discharge information exchange. It was uncommon for community physicians to receive discharge information via electronic mail (Table 3). Outpatient physicians acknowledged receipt of a minority of facsimile transmissions with no significant difference between discharge software vs. usual care (Table 3). Investigators documented facsimile transmission of the output from the discharge software to outpatient practitioners. Transmission occurred on the first business day after discharge. Despite the documentation of all facsimile transmissions, only 23.4% of patients assigned to discharge software had community practitioners who acknowledged receipt.

In exploratory analyses, we evaluated the effect of hospital physician level of training. We wondered if discharging physician experience or seniority affected perceptions of patients or primary care physicians. We entered level of training as a covariate in generalized estimating equations. When patient perception of discharge preparedness (B‐PREPARED) was the dependent variable, then physician level of training had a nonsignificant coefficient (P > 0.219). Likewise, physician level of training was nonsignificant in models of patient satisfaction with medication information, SIMS (P > 0.068), and outpatient physician perception, Modified Physician‐PREPARED (P > 0.177). We concluded that physician level of training had no influence on the patient‐level outcomes assessed in our study.

We compared the satisfaction of hospital physicians who used the discharge software and the usual care discharge. The proportions of hospital physicians who returned questionnaires were 85.7% (30/35) in the discharge software group and 97% (34/35) in the usual care group. After using their assigned discharge process for at least 6 months, discharge software users had mean (SD) satisfaction 7.4 (1.4) vs. 7.9 (1.4) for usual care physicians (difference = 0.5; 95% CI = 0.2‐1.3; P = 0.129). The effort for discharge software users was more difficult than the effort for usual care (mean [SD] effort = 6.5 [1.9] vs. 7.9 [2.1], respectively). The mean difference in effort was significant (difference = 1.4; 95% CI = 0.3‐2.4; P = 0.011). We reviewed free‐text comments on hospital physician questionnaires. The common theme was software users spent more time to complete discharges. We did not perform time‐motion assessments so we cannot confirm or refute these impressions. Even though hospital physicians found the discharge software significantly more difficult, they did not report a significant decrease in their satisfaction between the 2 discharge interventions.

The cluster design of our trial assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for B‐PREPARED, SIMS, and Modified Physician‐PREPARED. For all of these outcome variables, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on CIs for intervention coefficients (Table 2).

We evaluated the adequacy of the blind for outcome assessors who interviewed patients for B‐PREPARED and SIMS. The guesses of outcomes assessors were unrelated to true intervention assignment (P = 0.253). We interpreted the blind as adequate.

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software vs. usual care discharge. The discharge software incorporated the ASTM (American Society for Testing and Material) Continuity of Care Record (CCR) standards.19 The CCR is a patient health summary standard with widespread support from medical and specialty organizations. The rationale for the CCR was the need for continuity of care from 1 provider or practitioner to any other practitioner. Our discharge software had the same rationale as the CCR and included a subset of the clinical content specified by the CCR. Like the CCR, our discharge software produced concise reports, and emphasized a brief, postdischarge, care plan. Since we included clinical data elements recommended by the CCR, we hypothesized our discharge software would produce clinically relevant improvements.

Our discharge software also implemented elements of high‐quality discharge planning and communication endorsed by the Society of Hospital Medicine.20 For example, the discharge software produced a legible, typed, discharge plan for the patient or caregiver with medication instructions, follow‐up tests, studies, and appointments. The discharge software generated a discharge summary for the outpatient primary care physician and other clinicians who provided postdischarge care. The summary included discharge diagnoses, key findings and test results, follow‐up appointments, pending diagnostic tests, documentation of patient education, reconciled medication list, and contact information for the hospital physician. The discharge software compiled data for purposes of benchmarking, measurement, and continuous quality improvement. We thought our implementation of discharge software would lead to improved outcomes.

Despite our deployment of recommended strategies, we detected only small increases in patient perceptions of discharge preparedness. We do not know if small changes in B‐PREPARED values were clinically important. We found no improvement in patient satisfaction with medication information. Our results are consistent with systematic reviews that revealed limited benefit of interventions other than discharge planning with postdischarge support.21 Since our discharge software was added to robust discharge planning and support, we possibly had limited ability to detect benefit unless the intervention had a large effect size.

Our discharge software caused a small increase in positive perception reported by outpatient physicians. Small changes in the Modified Physician‐PREPARED had uncertain clinical relevance. Potential delays imposed by our distribution method may have contributed to our findings. Output from our discharge software went to community physicians via facsimile transmission with backup copies via standard U.S. mail. Our distribution system responded to several realities. Most community physicians in our area had no access to interoperable electronic medical records or secured e‐mail. In addition, electronic transmittal of prescriptions was not commonplace. Our discharge intervention did not control the flow of information inside the offices of outpatient physicians. We did not know if our facsimile transmissions joined piles of unread laboratory and imaging reports on the desks of busy primary care physicians. Despite the limited technology available to community physicians, they perceived communication generated by the software to be an improvement over the handwritten process. Our results support previous studies in which physicians preferred computer‐generated discharge summaries and summaries in standardized formats.2224

One of the limitations of our trial design was the unmasked intervention. Hospital physicians assigned to usual care might have improved their handwritten and verbal discharge communication after observation of their colleagues assigned to discharge software. This phenomenon is encountered in unmasked trials and is called contamination. We attempted to minimize contamination when we blocked usual care physicians from access to the discharge software. However, we could not eliminate cross‐talk among unmasked hospital physicians who worked together in close proximity during 27 months of patient enrollment. Some contamination was inevitable. When contamination occurred, there was bias toward the null (increased type II error).

Another limitation was the large proportion of hospital physicians in the first year of postgraduate training. There was a potential for variance from multilevel clusters with patient‐level outcomes clustered within first‐year hospital physicians who were clustered within teams supervised by senior resident or attending physicians. Our results argued against hierarchical clusters because intracluster correlation coefficients were negligible. Furthermore, our exploratory analysis suggested physician training level had no influence on patient outcomes measured in our study. We speculate the highly structured discharge process for both usual care and software minimized variance attributable to physician training level.

The research intervention in our trial was a stand‐alone software application. The discharge software did not integrate with the hospital electronic medical record. Consequently, hospital physician users had to reenter patient demographic data and prescription data that already existed in the electronic record. Data reentry probably caused hospital physicians to attribute greater effort to the discharge software.

In our study, hospital physicians incorporated discharge software with CPOE into their clinical workflow without deterioration in their satisfaction. Our experience may inform the decisions of hospital personnel who design health information systems. When designing discharge functions, developers should consider medication reconciliation and the standards of the CCR.19 Modules within the discharge software would likely be more efficient with prepopulated data from the electronic record. Then users could shift their work from data entry to data verification and possibly mitigate their perceived effort. Software developers may wish to explore options for data transmission to community physicians: secure e‐mail, automated fax servers, and direct digital file transfer. Future studies should test the acceptability of discharge functions incorporated within electronic health records with robust clinical decision support.

Our results apply to a population of adults of all ages with high risk for readmission. The results may not generalize to children, surgical patients, or people with low risk for readmission. All of the patients in our study were discharged to home. The exclusion of other discharge destinations helped us to enroll a homogenous cohort. However, the exclusion criteria did not allow us to generalize our results to patients discharged to nursing homes, inpatient rehabilitation units, or other acute care facilities. We designed the intervention to apply to the hospitalist model, in which responsibility for patient care transitions to a different physician after discharge. The results of our study do not apply when the inpatient and outpatient physician are the same. Since we enrolled general internal medicine hospital physicians, our results may not generalize to care provided by other specialists.

Conclusions

A discharge software application with CPOE improved perceptions of the hospital discharge process for patients and their outpatient physicians. When compared to the handwritten discharge process, the improvements were small in magnitude. Hospital physicians who used the discharge software reported more effort but otherwise no decrement in their satisfaction with the discharge process.

During the transition from inpatient to outpatient care, patients are vulnerable to adverse events.1 Poor communication between hospital personnel and either the patient or the outpatient primary care physician has been associated with preventable or ameliorable adverse events after discharge.1 Systematic reviews confirm that discharge communication is often delayed, inaccurate, or ineffective.2, 3

Discharge communication failures may occur if hospital processes rely on dictated discharge summaries.2 For several reasons, discharge summaries are inadequate for communication. Most patients complete their initial posthospital clinic visit before their primary care physician receives the discharge summary.4 For many patients, the discharge summary is unavailable for all posthospital visits.4 Discharge summaries often fail as communication because they are not generated or transmitted.4

Recommendations to improve discharge communication include the use of health information technology.2, 5 The benefits of computer‐generated discharge summaries include decreases in delivery time for discharge communications.2 The benefits of computerized physician order entry (CPOE) include reduction of medical errors.6 These theoretical benefits create a rationale for clinical trials to measure improvements after discharge software applications with CPOE.5

In an effort to improve discharge communication and clinically relevant outcomes, we performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The clustered design followed recommendations from a systematic review of discharge interventions.3 We applied our research intervention at the physician level and measured outcomes at the patient level. Our objective was to assess the benefit of discharge software with CPOE vs. usual care when used to discharge patients at high risk for repeat admission. In a previous work, we reported that discharge software did not reduce rates of hospital readmission, emergency department visits, or postdischarge adverse events due to medical management.7 In the present article, we compare secondary outcomes after the research intervention: perceptions of the discharge from the perspectives of patients, primary care physicians, and hospital physicians.

Methods

The trial design was a cluster randomized, controlled trial. The setting was the postdischarge environment following index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). We approached 6884 patients during their index hospitalization. After excluding 6253 ineligible patients, we enrolled 631 willing patients (Supplementary Appendix). As depicted in Figure 1, the most common reason for ineligibility occurred for patients with Pra score <0.40 (2168/6253 exclusions; 34.7%). We followed 631 patients who received the discharge intervention (Figure 1). There was no differential dropout between the interventions. Protocol deviations were rare, 0.5% (3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly assigned hospital physicians and their patients are in Table 1. Most of the hospital physicians were residents in the first year of postgraduate training.

Figure 1

We assessed the patient's perception of discharge preparedness. One week after discharge, research personnel interviewed 92.4% (292/316) of patients in the discharge software group and 92.4% (291/315) in the usual care group. The mean (SD) B‐PREPARED scores for discharge preparedness were 17.7 (4.1) in the discharge software group and 17.2 (4.0) in the usual care group. In the generalized estimating equation that accounted for potential clustering within hospital physicians, the parameter estimate for the intervention variable coefficient was small but significant (P = 0.042; Table 2). Patients in the discharge software group had slightly better perceptions of their discharge preparedness.

Another outcome was the patient's satisfaction with information about discharge medications (Table 2). One week after discharge, mean (SD) SIMS scores for satisfaction were 12.3 (4.8) in the discharge software group and 12.1 (4.6) in the usual care group. The generalized estimating equation revealed an insignificant coefficient for the intervention variable (P = 0.567; Table 2).

We assessed the outpatient physician perception of the discharge with a questionnaire sent 10 days after discharge. We received 496 out of 631 questionnaires (78.6%) from outpatient practitioners and the median response time was 19 days after the date of discharge. The practitioner specialty was internal medicine for 38.9% (193/496), family medicine for 27.2% (135/496), medicine‐pediatrics for 27.0% (134/496), advance practice nurse for 4.4% (22/496), other physician specialties for 2.0% (10/496), and physician assistant for 0.4% (2/496). We excluded 18 questionnaires from analysis because outpatient practitioners failed to answer 2 or more items in the Modified Physician‐PREPARED scale. When we compared baseline characteristics for patients who had complete questionnaires vs. patients with nonrespondent or excluded questionnaires, we found no significant differences (data available upon request). Among the discharge software group, 72.2% (228/316) of patients had complete questionnaires from their outpatient physicians. The response rate with complete questionnaires was 79.4% (250/315) of patients assigned to usual care. On the Modified Physician‐PREPARED scale, the mean (SD) scores were 17.2 (3.8) for the discharge software group and 16.5 (3.9) for the usual care group. The parameter estimate from the generalized estimating equation was significant (P = 0.027; Table 2). Outpatient physicians had slightly better perception of discharge quality for patients assigned to discharge software.

In the questionnaire sent to outpatient practitioners, we requested additional information about discharge communication. When asked about timeliness, outpatient physicians perceived no faster communication with the discharge software (Table 3). We asked about the media for discharge information exchange. It was uncommon for community physicians to receive discharge information via electronic mail (Table 3). Outpatient physicians acknowledged receipt of a minority of facsimile transmissions with no significant difference between discharge software vs. usual care (Table 3). Investigators documented facsimile transmission of the output from the discharge software to outpatient practitioners. Transmission occurred on the first business day after discharge. Despite the documentation of all facsimile transmissions, only 23.4% of patients assigned to discharge software had community practitioners who acknowledged receipt.

In exploratory analyses, we evaluated the effect of hospital physician level of training. We wondered if discharging physician experience or seniority affected perceptions of patients or primary care physicians. We entered level of training as a covariate in generalized estimating equations. When patient perception of discharge preparedness (B‐PREPARED) was the dependent variable, then physician level of training had a nonsignificant coefficient (P > 0.219). Likewise, physician level of training was nonsignificant in models of patient satisfaction with medication information, SIMS (P > 0.068), and outpatient physician perception, Modified Physician‐PREPARED (P > 0.177). We concluded that physician level of training had no influence on the patient‐level outcomes assessed in our study.

We compared the satisfaction of hospital physicians who used the discharge software and the usual care discharge. The proportions of hospital physicians who returned questionnaires were 85.7% (30/35) in the discharge software group and 97% (34/35) in the usual care group. After using their assigned discharge process for at least 6 months, discharge software users had mean (SD) satisfaction 7.4 (1.4) vs. 7.9 (1.4) for usual care physicians (difference = 0.5; 95% CI = 0.2‐1.3; P = 0.129). The effort for discharge software users was more difficult than the effort for usual care (mean [SD] effort = 6.5 [1.9] vs. 7.9 [2.1], respectively). The mean difference in effort was significant (difference = 1.4; 95% CI = 0.3‐2.4; P = 0.011). We reviewed free‐text comments on hospital physician questionnaires. The common theme was software users spent more time to complete discharges. We did not perform time‐motion assessments so we cannot confirm or refute these impressions. Even though hospital physicians found the discharge software significantly more difficult, they did not report a significant decrease in their satisfaction between the 2 discharge interventions.

The cluster design of our trial assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for B‐PREPARED, SIMS, and Modified Physician‐PREPARED. For all of these outcome variables, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on CIs for intervention coefficients (Table 2).

We evaluated the adequacy of the blind for outcome assessors who interviewed patients for B‐PREPARED and SIMS. The guesses of outcomes assessors were unrelated to true intervention assignment (P = 0.253). We interpreted the blind as adequate.

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software vs. usual care discharge. The discharge software incorporated the ASTM (American Society for Testing and Material) Continuity of Care Record (CCR) standards.19 The CCR is a patient health summary standard with widespread support from medical and specialty organizations. The rationale for the CCR was the need for continuity of care from 1 provider or practitioner to any other practitioner. Our discharge software had the same rationale as the CCR and included a subset of the clinical content specified by the CCR. Like the CCR, our discharge software produced concise reports, and emphasized a brief, postdischarge, care plan. Since we included clinical data elements recommended by the CCR, we hypothesized our discharge software would produce clinically relevant improvements.

Our discharge software also implemented elements of high‐quality discharge planning and communication endorsed by the Society of Hospital Medicine.20 For example, the discharge software produced a legible, typed, discharge plan for the patient or caregiver with medication instructions, follow‐up tests, studies, and appointments. The discharge software generated a discharge summary for the outpatient primary care physician and other clinicians who provided postdischarge care. The summary included discharge diagnoses, key findings and test results, follow‐up appointments, pending diagnostic tests, documentation of patient education, reconciled medication list, and contact information for the hospital physician. The discharge software compiled data for purposes of benchmarking, measurement, and continuous quality improvement. We thought our implementation of discharge software would lead to improved outcomes.

Despite our deployment of recommended strategies, we detected only small increases in patient perceptions of discharge preparedness. We do not know if small changes in B‐PREPARED values were clinically important. We found no improvement in patient satisfaction with medication information. Our results are consistent with systematic reviews that revealed limited benefit of interventions other than discharge planning with postdischarge support.21 Since our discharge software was added to robust discharge planning and support, we possibly had limited ability to detect benefit unless the intervention had a large effect size.

Our discharge software caused a small increase in positive perception reported by outpatient physicians. Small changes in the Modified Physician‐PREPARED had uncertain clinical relevance. Potential delays imposed by our distribution method may have contributed to our findings. Output from our discharge software went to community physicians via facsimile transmission with backup copies via standard U.S. mail. Our distribution system responded to several realities. Most community physicians in our area had no access to interoperable electronic medical records or secured e‐mail. In addition, electronic transmittal of prescriptions was not commonplace. Our discharge intervention did not control the flow of information inside the offices of outpatient physicians. We did not know if our facsimile transmissions joined piles of unread laboratory and imaging reports on the desks of busy primary care physicians. Despite the limited technology available to community physicians, they perceived communication generated by the software to be an improvement over the handwritten process. Our results support previous studies in which physicians preferred computer‐generated discharge summaries and summaries in standardized formats.2224

One of the limitations of our trial design was the unmasked intervention. Hospital physicians assigned to usual care might have improved their handwritten and verbal discharge communication after observation of their colleagues assigned to discharge software. This phenomenon is encountered in unmasked trials and is called contamination. We attempted to minimize contamination when we blocked usual care physicians from access to the discharge software. However, we could not eliminate cross‐talk among unmasked hospital physicians who worked together in close proximity during 27 months of patient enrollment. Some contamination was inevitable. When contamination occurred, there was bias toward the null (increased type II error).

Another limitation was the large proportion of hospital physicians in the first year of postgraduate training. There was a potential for variance from multilevel clusters with patient‐level outcomes clustered within first‐year hospital physicians who were clustered within teams supervised by senior resident or attending physicians. Our results argued against hierarchical clusters because intracluster correlation coefficients were negligible. Furthermore, our exploratory analysis suggested physician training level had no influence on patient outcomes measured in our study. We speculate the highly structured discharge process for both usual care and software minimized variance attributable to physician training level.

The research intervention in our trial was a stand‐alone software application. The discharge software did not integrate with the hospital electronic medical record. Consequently, hospital physician users had to reenter patient demographic data and prescription data that already existed in the electronic record. Data reentry probably caused hospital physicians to attribute greater effort to the discharge software.

In our study, hospital physicians incorporated discharge software with CPOE into their clinical workflow without deterioration in their satisfaction. Our experience may inform the decisions of hospital personnel who design health information systems. When designing discharge functions, developers should consider medication reconciliation and the standards of the CCR.19 Modules within the discharge software would likely be more efficient with prepopulated data from the electronic record. Then users could shift their work from data entry to data verification and possibly mitigate their perceived effort. Software developers may wish to explore options for data transmission to community physicians: secure e‐mail, automated fax servers, and direct digital file transfer. Future studies should test the acceptability of discharge functions incorporated within electronic health records with robust clinical decision support.

Our results apply to a population of adults of all ages with high risk for readmission. The results may not generalize to children, surgical patients, or people with low risk for readmission. All of the patients in our study were discharged to home. The exclusion of other discharge destinations helped us to enroll a homogenous cohort. However, the exclusion criteria did not allow us to generalize our results to patients discharged to nursing homes, inpatient rehabilitation units, or other acute care facilities. We designed the intervention to apply to the hospitalist model, in which responsibility for patient care transitions to a different physician after discharge. The results of our study do not apply when the inpatient and outpatient physician are the same. Since we enrolled general internal medicine hospital physicians, our results may not generalize to care provided by other specialists.

Conclusions

A discharge software application with CPOE improved perceptions of the hospital discharge process for patients and their outpatient physicians. When compared to the handwritten discharge process, the improvements were small in magnitude. Hospital physicians who used the discharge software reported more effort but otherwise no decrement in their satisfaction with the discharge process.

Adverse events occur to patients after their discharge from acute care hospitals.1, 2 Most of these injuries are adverse drug events, procedure‐related events, nosocomial infections, or falls.1 Postdischarge adverse events are associated with several days of symptoms, nonpermanent disability, emergency department visits, or hospital readmission.1, 3 When adverse events are preventable or ameliorable, the most common root cause is poor communication between hospital personnel and either the patient or the outpatient primary care physician.1 In addition, there may be deficits in discharge processes related to assessment and communication of unresolved problems.1 Systematic reviews have shown that discharge communication is an inefficient and error‐prone process.46

One potential solution to poor discharge communication is health information technology.7 An example of technology is discharge software with a computerized physician order entry (CPOE) system. By definition, a CPOE system is a computer‐based system that automates direct entry of orders by physicians and ensures standardized, legible, and complete orders.8 The benefits of CPOE have been tested in other inpatient settings.8, 9 It is logical to consider software applications with CPOE for discharge interventions.7

Several mechanisms explain the potential benefit of discharge software with CPOE.7 Applications with CPOE decrease medication errors.8, 10 Software with decision support could prompt physicians to enter posthospitalization appointment dates and orders for preventive services.11, 12 Discharge software could facilitate medication reconciliation and generate patient instructions and information.4, 1315 The potential benefits of discharge software with CPOE provide a rationale for clinical trials to measure benefits.

Previous studies addressed discharge applications of health information technology. Observational studies recorded outcomes such as physician satisfaction.16, 17 Prior randomized clinical trials measured quality and timeliness of discharge summaries.18 However, these previous trials did not assess clinically relevant outcomes like readmissions, emergency department visits, or adverse events. We performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The rationale for our clustered design complied with recommendations from a systematic review of discharge interventions.5 Our objective was to assess the benefit of discharge software with CPOE when used to discharge patients at high risk for repeat admission. After the intervention, we compared the rates of hospital readmission, emergency department visits, and postdischarge adverse events due to medical management.

Methods

The trial design was a cluster randomized, controlled trial with blinded outcome assessment. Follow‐up occurred until 6 months after discharge from index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. The physician characteristics appear in Table 1. Most of the hospital physicians were interns in the first year of postgraduate training (58.6%; 41/70). We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). The most common reason for hospital physician exclusion applied to resident physicians in their last months of training before graduation or emergency department residents temporarily assigned to internal medicine training. We approached 6,884 patients during their index hospitalization. After excluding 6,253 ineligible patients, we enrolled and followed 631 patients who received the discharge intervention (Figure 1). During 6 months of follow‐up, a small proportion of patients died (3%; 20/631). Hospital records were available for deceased patients and they were included in the analysis. A small proportion (6%; 41/631) of patients withdrew consent or left the trial for other reasons during 6 months. There was no differential dropout between the interventions. Protocol deviations were rare (0.5%; 3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly‐assigned hospital physicians and their patients are in Table 1.

Figure 1

We asked outpatient physicians about their receipt of discharge communication from hospital physicians. The text of the question was, How soon after discharge did you receive any information (in any form) relating to this patient's hospital admission and discharge plans? We mailed the question 10 days after discharge to outpatient physicians designated by patients enrolled in the study. Among patients in the discharge software group, 75.0% (237/316) of their outpatient physicians responded to the question. The response rate was 80.6% (254/315) from physicians who followed patients in the usual care group. Respondents from the discharge software group said within 1‐2 days or within a week for 56.0% (177/316) of patients. Respondents from the usual care group said within 1‐2 days or within a week for 57.1% (180/315) of patients. The difference between the intervention groups, 1.1% (95% CI, 9.2% to 6.9%), was not significant.

The primary, prespecified, outcome of the study was the proportion of patients with at least 1 readmission to the hospital. After intervention with discharge software versus usual care, there was no significant difference in readmission rates (Table 2) or time to first readmission (Figure 2). We screened all baseline variables in Table 1 and sought predictors of readmission to employ in adjusted models. For example, we evaluated physician level of training because we wondered if experience or seniority affected readmission when hospital physicians used the discharge software or usual care discharge. The candidate variable, physician level of training, did not correlate with readmission (rho = 0.066; P = 0.100), so it was dropped from subsequent analyses. After screening all variables in Table 1, we found 4 independent predictors of readmission: previous hospitalizations, previous emergency department visits, heart failure, and physical function. Generalized estimating equations for readmission that adjusted for predictor variables confirmed a negligible parameter estimate for the discharge intervention variable coefficient (Table 2).

Figure 2

We evaluated emergency department visits that were unrelated to readmission as secondary, prespecified, outcomes. The results were similar to readmission results. While the proportion of patients with at least 1 emergency department visit was lower for the discharge software intervention, the difference with usual care was not significant (Table 2). There was no significant difference between interventions for time to first emergency department visit (Figure 3).

Figure 3

Postdischarge adverse events were secondary, prespecified, outcomes. Data for adverse event adjudication were available for 98% (309/316) of discharge software patients and 97% (307/315) of usual care patients. Within 1 month after discharge, 46 patients had adverse events probably or definitely related to medical management. Two patients had 2 events and 1 patient had 3 events. For analysis, we randomly selected 1 event per patient. When comparing patients assigned to discharge software versus usual care, there were no differences in adverse events related to medical management (Table 2). Most of the events were possible adverse drug events (74%; 34/46). The adverse event severity was several days of symptoms or nonpermanent disability for 76% (35/46) of the adverse events. Adjudicators rated 26% (12/46) of the adverse events as preventable and 46% (21/46) as ameliorable. The absolute numbers of events were small. There were no differences between discharge software and usual care patients within adverse event strata defined by type, severity, preventable, or ameliorable (Table 3). For most of the patients with adverse events, the adjudicators could not identify a system problem or preventability category (Table 3). When a deficiency was evident, there was no pattern to suggest a significant difference between discharge software patients versus usual care patients.

When we designed the trial, we assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for readmissions, emergency department visits, and adverse events. For all of these outcomes, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on confidence intervals for intervention coefficients (Table 2).

We performed an exploratory stratified analysis. We evaluated the intervention effect on readmission within subgroups defined by covariates that predicted readmission (Table 4). When the intervention groups were compared within baseline categories of previous hospitalizations, previous emergency department visits, heart failure, and physical functioning, there was a consistent pattern with no differential effect by intervention assignment. None of the intervention coefficients were statistically significant (Table 4).

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software versus usual care handwritten discharge. The discharge software with CPOE implemented elements of high‐quality discharge planning and communication endorsed by the National Quality Forum and systematic reviews.6, 38 Despite theoretical benefits, our discharge software intervention did not reduce readmissions or emergency department visits. What were potential explanations for our results? We assumed an association between postdischarge adverse events and readmissions or emergency department visits.1 Our failure to reduce adverse events might explain the failure to reduce readmissions or emergency department visits. Another potential explanation was related to adverse drug events. Other investigators showed most postdischarge adverse events were adverse drug events and our data confirmed previous studies.1, 2 Medication reconciliation at discharge was a potential mechanism for adverse drug event reduction.14 Medication reconciliation was the standard at the study hospital, so it was unethical to deny reconciliation to patients assigned to either intervention.39 Required medication reconciliation in both groups, by its known effect on preventable adverse drug events, might have reduced the event rates in both groups.14 This possibility is supported by the low rate of adverse events observed in our study compared with other studies.1 We speculate that the low background rate of adverse events at the study hospital may have minimized events in both the discharge software and usual care groups and prevented detection of software benefits, if present.39

One limitation of our study may have been the discharge software. The automated decision support in our software lacked features that might have improved outcomes. For example, the software did not generate a list of diagnostic test results that were pending at the time of discharge. Our software relied on prompts to the physician user that did not specify which tests were pending. The software did not perform error checks on the discharge orders to warn physicians about drug‐drug interactions, therapeutic duplications, or missing items (eg, immunizations, drugs, education). The absence of these software enhancements made our discharge process vulnerable to the lapses and slips of the physician user. Whether or not such enhancements affect clinically relevant outcomes remains a testable hypothesis for future studies.

Another limitation of our study was the outpatient physician response. Discharge software did not increase the proportion of outpatient physicians who said they received communication within 7 days after hospital discharge. Our intervention addressed the sending partner but not the receiving partner in the communication dyad. Our discharge software was not designed to change information flow within the outpatient physician office. We do not know if discharge communication arrived and remained unnoticed until the patient called or visited the outpatient clinic. Future studies of discharge communication should consider a closed loop design to assure receipt and comprehension.

When we designed our study, we expected at least some variance between patient clusters attributable to the physician who performed the discharge. Our analysis of intracluster correlation revealed negligible variance. We speculate the highly‐standardized discharge process implemented by discharge software and usual care at our hospital resulted in minimal variance. Future studies of discharge interventions may consider designs that avoid cluster randomization.

In conclusion, a discharge software application of CPOE did not affect readmissions, emergency department visits, or adverse events after discharge.

Adverse events occur to patients after their discharge from acute care hospitals.1, 2 Most of these injuries are adverse drug events, procedure‐related events, nosocomial infections, or falls.1 Postdischarge adverse events are associated with several days of symptoms, nonpermanent disability, emergency department visits, or hospital readmission.1, 3 When adverse events are preventable or ameliorable, the most common root cause is poor communication between hospital personnel and either the patient or the outpatient primary care physician.1 In addition, there may be deficits in discharge processes related to assessment and communication of unresolved problems.1 Systematic reviews have shown that discharge communication is an inefficient and error‐prone process.46

One potential solution to poor discharge communication is health information technology.7 An example of technology is discharge software with a computerized physician order entry (CPOE) system. By definition, a CPOE system is a computer‐based system that automates direct entry of orders by physicians and ensures standardized, legible, and complete orders.8 The benefits of CPOE have been tested in other inpatient settings.8, 9 It is logical to consider software applications with CPOE for discharge interventions.7

Several mechanisms explain the potential benefit of discharge software with CPOE.7 Applications with CPOE decrease medication errors.8, 10 Software with decision support could prompt physicians to enter posthospitalization appointment dates and orders for preventive services.11, 12 Discharge software could facilitate medication reconciliation and generate patient instructions and information.4, 1315 The potential benefits of discharge software with CPOE provide a rationale for clinical trials to measure benefits.

Previous studies addressed discharge applications of health information technology. Observational studies recorded outcomes such as physician satisfaction.16, 17 Prior randomized clinical trials measured quality and timeliness of discharge summaries.18 However, these previous trials did not assess clinically relevant outcomes like readmissions, emergency department visits, or adverse events. We performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The rationale for our clustered design complied with recommendations from a systematic review of discharge interventions.5 Our objective was to assess the benefit of discharge software with CPOE when used to discharge patients at high risk for repeat admission. After the intervention, we compared the rates of hospital readmission, emergency department visits, and postdischarge adverse events due to medical management.

Methods

The trial design was a cluster randomized, controlled trial with blinded outcome assessment. Follow‐up occurred until 6 months after discharge from index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. The physician characteristics appear in Table 1. Most of the hospital physicians were interns in the first year of postgraduate training (58.6%; 41/70). We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). The most common reason for hospital physician exclusion applied to resident physicians in their last months of training before graduation or emergency department residents temporarily assigned to internal medicine training. We approached 6,884 patients during their index hospitalization. After excluding 6,253 ineligible patients, we enrolled and followed 631 patients who received the discharge intervention (Figure 1). During 6 months of follow‐up, a small proportion of patients died (3%; 20/631). Hospital records were available for deceased patients and they were included in the analysis. A small proportion (6%; 41/631) of patients withdrew consent or left the trial for other reasons during 6 months. There was no differential dropout between the interventions. Protocol deviations were rare (0.5%; 3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly‐assigned hospital physicians and their patients are in Table 1.

Figure 1

We asked outpatient physicians about their receipt of discharge communication from hospital physicians. The text of the question was, How soon after discharge did you receive any information (in any form) relating to this patient's hospital admission and discharge plans? We mailed the question 10 days after discharge to outpatient physicians designated by patients enrolled in the study. Among patients in the discharge software group, 75.0% (237/316) of their outpatient physicians responded to the question. The response rate was 80.6% (254/315) from physicians who followed patients in the usual care group. Respondents from the discharge software group said within 1‐2 days or within a week for 56.0% (177/316) of patients. Respondents from the usual care group said within 1‐2 days or within a week for 57.1% (180/315) of patients. The difference between the intervention groups, 1.1% (95% CI, 9.2% to 6.9%), was not significant.

The primary, prespecified, outcome of the study was the proportion of patients with at least 1 readmission to the hospital. After intervention with discharge software versus usual care, there was no significant difference in readmission rates (Table 2) or time to first readmission (Figure 2). We screened all baseline variables in Table 1 and sought predictors of readmission to employ in adjusted models. For example, we evaluated physician level of training because we wondered if experience or seniority affected readmission when hospital physicians used the discharge software or usual care discharge. The candidate variable, physician level of training, did not correlate with readmission (rho = 0.066; P = 0.100), so it was dropped from subsequent analyses. After screening all variables in Table 1, we found 4 independent predictors of readmission: previous hospitalizations, previous emergency department visits, heart failure, and physical function. Generalized estimating equations for readmission that adjusted for predictor variables confirmed a negligible parameter estimate for the discharge intervention variable coefficient (Table 2).

Figure 2

We evaluated emergency department visits that were unrelated to readmission as secondary, prespecified, outcomes. The results were similar to readmission results. While the proportion of patients with at least 1 emergency department visit was lower for the discharge software intervention, the difference with usual care was not significant (Table 2). There was no significant difference between interventions for time to first emergency department visit (Figure 3).

Figure 3

Postdischarge adverse events were secondary, prespecified, outcomes. Data for adverse event adjudication were available for 98% (309/316) of discharge software patients and 97% (307/315) of usual care patients. Within 1 month after discharge, 46 patients had adverse events probably or definitely related to medical management. Two patients had 2 events and 1 patient had 3 events. For analysis, we randomly selected 1 event per patient. When comparing patients assigned to discharge software versus usual care, there were no differences in adverse events related to medical management (Table 2). Most of the events were possible adverse drug events (74%; 34/46). The adverse event severity was several days of symptoms or nonpermanent disability for 76% (35/46) of the adverse events. Adjudicators rated 26% (12/46) of the adverse events as preventable and 46% (21/46) as ameliorable. The absolute numbers of events were small. There were no differences between discharge software and usual care patients within adverse event strata defined by type, severity, preventable, or ameliorable (Table 3). For most of the patients with adverse events, the adjudicators could not identify a system problem or preventability category (Table 3). When a deficiency was evident, there was no pattern to suggest a significant difference between discharge software patients versus usual care patients.

When we designed the trial, we assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for readmissions, emergency department visits, and adverse events. For all of these outcomes, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on confidence intervals for intervention coefficients (Table 2).

We performed an exploratory stratified analysis. We evaluated the intervention effect on readmission within subgroups defined by covariates that predicted readmission (Table 4). When the intervention groups were compared within baseline categories of previous hospitalizations, previous emergency department visits, heart failure, and physical functioning, there was a consistent pattern with no differential effect by intervention assignment. None of the intervention coefficients were statistically significant (Table 4).

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software versus usual care handwritten discharge. The discharge software with CPOE implemented elements of high‐quality discharge planning and communication endorsed by the National Quality Forum and systematic reviews.6, 38 Despite theoretical benefits, our discharge software intervention did not reduce readmissions or emergency department visits. What were potential explanations for our results? We assumed an association between postdischarge adverse events and readmissions or emergency department visits.1 Our failure to reduce adverse events might explain the failure to reduce readmissions or emergency department visits. Another potential explanation was related to adverse drug events. Other investigators showed most postdischarge adverse events were adverse drug events and our data confirmed previous studies.1, 2 Medication reconciliation at discharge was a potential mechanism for adverse drug event reduction.14 Medication reconciliation was the standard at the study hospital, so it was unethical to deny reconciliation to patients assigned to either intervention.39 Required medication reconciliation in both groups, by its known effect on preventable adverse drug events, might have reduced the event rates in both groups.14 This possibility is supported by the low rate of adverse events observed in our study compared with other studies.1 We speculate that the low background rate of adverse events at the study hospital may have minimized events in both the discharge software and usual care groups and prevented detection of software benefits, if present.39

One limitation of our study may have been the discharge software. The automated decision support in our software lacked features that might have improved outcomes. For example, the software did not generate a list of diagnostic test results that were pending at the time of discharge. Our software relied on prompts to the physician user that did not specify which tests were pending. The software did not perform error checks on the discharge orders to warn physicians about drug‐drug interactions, therapeutic duplications, or missing items (eg, immunizations, drugs, education). The absence of these software enhancements made our discharge process vulnerable to the lapses and slips of the physician user. Whether or not such enhancements affect clinically relevant outcomes remains a testable hypothesis for future studies.

Another limitation of our study was the outpatient physician response. Discharge software did not increase the proportion of outpatient physicians who said they received communication within 7 days after hospital discharge. Our intervention addressed the sending partner but not the receiving partner in the communication dyad. Our discharge software was not designed to change information flow within the outpatient physician office. We do not know if discharge communication arrived and remained unnoticed until the patient called or visited the outpatient clinic. Future studies of discharge communication should consider a closed loop design to assure receipt and comprehension.

When we designed our study, we expected at least some variance between patient clusters attributable to the physician who performed the discharge. Our analysis of intracluster correlation revealed negligible variance. We speculate the highly‐standardized discharge process implemented by discharge software and usual care at our hospital resulted in minimal variance. Future studies of discharge interventions may consider designs that avoid cluster randomization.

In conclusion, a discharge software application of CPOE did not affect readmissions, emergency department visits, or adverse events after discharge.

Adverse events occur to patients after their discharge from acute care hospitals.1, 2 Most of these injuries are adverse drug events, procedure‐related events, nosocomial infections, or falls.1 Postdischarge adverse events are associated with several days of symptoms, nonpermanent disability, emergency department visits, or hospital readmission.1, 3 When adverse events are preventable or ameliorable, the most common root cause is poor communication between hospital personnel and either the patient or the outpatient primary care physician.1 In addition, there may be deficits in discharge processes related to assessment and communication of unresolved problems.1 Systematic reviews have shown that discharge communication is an inefficient and error‐prone process.46

One potential solution to poor discharge communication is health information technology.7 An example of technology is discharge software with a computerized physician order entry (CPOE) system. By definition, a CPOE system is a computer‐based system that automates direct entry of orders by physicians and ensures standardized, legible, and complete orders.8 The benefits of CPOE have been tested in other inpatient settings.8, 9 It is logical to consider software applications with CPOE for discharge interventions.7

Several mechanisms explain the potential benefit of discharge software with CPOE.7 Applications with CPOE decrease medication errors.8, 10 Software with decision support could prompt physicians to enter posthospitalization appointment dates and orders for preventive services.11, 12 Discharge software could facilitate medication reconciliation and generate patient instructions and information.4, 1315 The potential benefits of discharge software with CPOE provide a rationale for clinical trials to measure benefits.

Previous studies addressed discharge applications of health information technology. Observational studies recorded outcomes such as physician satisfaction.16, 17 Prior randomized clinical trials measured quality and timeliness of discharge summaries.18 However, these previous trials did not assess clinically relevant outcomes like readmissions, emergency department visits, or adverse events. We performed a cluster‐randomized trial to assess the value of a discharge software application of CPOE. The rationale for our clustered design complied with recommendations from a systematic review of discharge interventions.5 Our objective was to assess the benefit of discharge software with CPOE when used to discharge patients at high risk for repeat admission. After the intervention, we compared the rates of hospital readmission, emergency department visits, and postdischarge adverse events due to medical management.

Methods

The trial design was a cluster randomized, controlled trial with blinded outcome assessment. Follow‐up occurred until 6 months after discharge from index hospitalization at a 730‐bed, tertiary care, teaching hospital in central Illinois. The Peoria Institutional Review Board approved the protocol for human research.

Results

We screened 127 physicians who were general internal medicine hospital physicians. Seventy physicians consented and received random allocation to discharge software or usual care. The physician characteristics appear in Table 1. Most of the hospital physicians were interns in the first year of postgraduate training (58.6%; 41/70). We excluded 57 physicians for reasons shown in the trial flow diagram (Figure 1). The most common reason for hospital physician exclusion applied to resident physicians in their last months of training before graduation or emergency department residents temporarily assigned to internal medicine training. We approached 6,884 patients during their index hospitalization. After excluding 6,253 ineligible patients, we enrolled and followed 631 patients who received the discharge intervention (Figure 1). During 6 months of follow‐up, a small proportion of patients died (3%; 20/631). Hospital records were available for deceased patients and they were included in the analysis. A small proportion (6%; 41/631) of patients withdrew consent or left the trial for other reasons during 6 months. There was no differential dropout between the interventions. Protocol deviations were rare (0.5%; 3/631). Three patients erroneously received usual care discharge from physicians assigned to discharge software. All 631 patients were included in the intention‐to‐treat analysis. The baseline characteristics of the randomly‐assigned hospital physicians and their patients are in Table 1.

Figure 1

We asked outpatient physicians about their receipt of discharge communication from hospital physicians. The text of the question was, How soon after discharge did you receive any information (in any form) relating to this patient's hospital admission and discharge plans? We mailed the question 10 days after discharge to outpatient physicians designated by patients enrolled in the study. Among patients in the discharge software group, 75.0% (237/316) of their outpatient physicians responded to the question. The response rate was 80.6% (254/315) from physicians who followed patients in the usual care group. Respondents from the discharge software group said within 1‐2 days or within a week for 56.0% (177/316) of patients. Respondents from the usual care group said within 1‐2 days or within a week for 57.1% (180/315) of patients. The difference between the intervention groups, 1.1% (95% CI, 9.2% to 6.9%), was not significant.

The primary, prespecified, outcome of the study was the proportion of patients with at least 1 readmission to the hospital. After intervention with discharge software versus usual care, there was no significant difference in readmission rates (Table 2) or time to first readmission (Figure 2). We screened all baseline variables in Table 1 and sought predictors of readmission to employ in adjusted models. For example, we evaluated physician level of training because we wondered if experience or seniority affected readmission when hospital physicians used the discharge software or usual care discharge. The candidate variable, physician level of training, did not correlate with readmission (rho = 0.066; P = 0.100), so it was dropped from subsequent analyses. After screening all variables in Table 1, we found 4 independent predictors of readmission: previous hospitalizations, previous emergency department visits, heart failure, and physical function. Generalized estimating equations for readmission that adjusted for predictor variables confirmed a negligible parameter estimate for the discharge intervention variable coefficient (Table 2).

Figure 2

We evaluated emergency department visits that were unrelated to readmission as secondary, prespecified, outcomes. The results were similar to readmission results. While the proportion of patients with at least 1 emergency department visit was lower for the discharge software intervention, the difference with usual care was not significant (Table 2). There was no significant difference between interventions for time to first emergency department visit (Figure 3).

Figure 3

Postdischarge adverse events were secondary, prespecified, outcomes. Data for adverse event adjudication were available for 98% (309/316) of discharge software patients and 97% (307/315) of usual care patients. Within 1 month after discharge, 46 patients had adverse events probably or definitely related to medical management. Two patients had 2 events and 1 patient had 3 events. For analysis, we randomly selected 1 event per patient. When comparing patients assigned to discharge software versus usual care, there were no differences in adverse events related to medical management (Table 2). Most of the events were possible adverse drug events (74%; 34/46). The adverse event severity was several days of symptoms or nonpermanent disability for 76% (35/46) of the adverse events. Adjudicators rated 26% (12/46) of the adverse events as preventable and 46% (21/46) as ameliorable. The absolute numbers of events were small. There were no differences between discharge software and usual care patients within adverse event strata defined by type, severity, preventable, or ameliorable (Table 3). For most of the patients with adverse events, the adjudicators could not identify a system problem or preventability category (Table 3). When a deficiency was evident, there was no pattern to suggest a significant difference between discharge software patients versus usual care patients.

When we designed the trial, we assumed variance in outcomes measured at the patient level. We predicted some variance attributable to clustering by hospital physician. After the trial, we calculated the intracluster correlation coefficients for readmissions, emergency department visits, and adverse events. For all of these outcomes, the intracluster correlation coefficients were negligible. We also evaluated generalized estimating equations with and without correction for hospital physician cluster. We confirmed the negligible cluster effect on confidence intervals for intervention coefficients (Table 2).

We performed an exploratory stratified analysis. We evaluated the intervention effect on readmission within subgroups defined by covariates that predicted readmission (Table 4). When the intervention groups were compared within baseline categories of previous hospitalizations, previous emergency department visits, heart failure, and physical functioning, there was a consistent pattern with no differential effect by intervention assignment. None of the intervention coefficients were statistically significant (Table 4).

Discussion

We performed a cluster‐randomized clinical trial to measure the effects of discharge software versus usual care handwritten discharge. The discharge software with CPOE implemented elements of high‐quality discharge planning and communication endorsed by the National Quality Forum and systematic reviews.6, 38 Despite theoretical benefits, our discharge software intervention did not reduce readmissions or emergency department visits. What were potential explanations for our results? We assumed an association between postdischarge adverse events and readmissions or emergency department visits.1 Our failure to reduce adverse events might explain the failure to reduce readmissions or emergency department visits. Another potential explanation was related to adverse drug events. Other investigators showed most postdischarge adverse events were adverse drug events and our data confirmed previous studies.1, 2 Medication reconciliation at discharge was a potential mechanism for adverse drug event reduction.14 Medication reconciliation was the standard at the study hospital, so it was unethical to deny reconciliation to patients assigned to either intervention.39 Required medication reconciliation in both groups, by its known effect on preventable adverse drug events, might have reduced the event rates in both groups.14 This possibility is supported by the low rate of adverse events observed in our study compared with other studies.1 We speculate that the low background rate of adverse events at the study hospital may have minimized events in both the discharge software and usual care groups and prevented detection of software benefits, if present.39

One limitation of our study may have been the discharge software. The automated decision support in our software lacked features that might have improved outcomes. For example, the software did not generate a list of diagnostic test results that were pending at the time of discharge. Our software relied on prompts to the physician user that did not specify which tests were pending. The software did not perform error checks on the discharge orders to warn physicians about drug‐drug interactions, therapeutic duplications, or missing items (eg, immunizations, drugs, education). The absence of these software enhancements made our discharge process vulnerable to the lapses and slips of the physician user. Whether or not such enhancements affect clinically relevant outcomes remains a testable hypothesis for future studies.

Another limitation of our study was the outpatient physician response. Discharge software did not increase the proportion of outpatient physicians who said they received communication within 7 days after hospital discharge. Our intervention addressed the sending partner but not the receiving partner in the communication dyad. Our discharge software was not designed to change information flow within the outpatient physician office. We do not know if discharge communication arrived and remained unnoticed until the patient called or visited the outpatient clinic. Future studies of discharge communication should consider a closed loop design to assure receipt and comprehension.

When we designed our study, we expected at least some variance between patient clusters attributable to the physician who performed the discharge. Our analysis of intracluster correlation revealed negligible variance. We speculate the highly‐standardized discharge process implemented by discharge software and usual care at our hospital resulted in minimal variance. Future studies of discharge interventions may consider designs that avoid cluster randomization.

In conclusion, a discharge software application of CPOE did not affect readmissions, emergency department visits, or adverse events after discharge.