User login
The Impact of Checklists on Inpatient Safety Outcomes: A Systematic Review of Randomized Controlled Trials
In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.1-4 Numerous strategies to improve patient safety have been proposed,5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.10,11
Checklists have served as a foundation for the standardization and safety of aviation and nuclear power12,13 and are advocated as simple and effective instruments for ensuring safe care.7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.
METHODS
The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.23
Search Strategy
On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.24 The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.
Eligibility Criteria
We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).
Study Selection
Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.25 Reviewers were not blinded to author or journal names.26
Data Extraction
The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.
Quality Assessment
The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.24 For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.27
Analysis
Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.28 Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).29 Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.
RESULTS
The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%
Study Characteristics
Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,21,30,36 4 studies examined hospital length of stay,21,30,33,36 and 2 studies reported user compliance with the checklist.21,31 Five of the studies reported patient outcomes,21,30,33,35,36 and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).31-34,37
Description of Checklists
Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,21,30,33,35-37 whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT30-35 and the 3 remaining studies used modified versions of previously validated checklists.21,36,37 The number of items included in the checklist ranged from 2 to 54.
Impact of the Checklist
Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)38 and IV (life-threatening)38 postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.21 Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).36 Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).30 Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.35 The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).33 However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).37 Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.31 Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).32 In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).34
30-day Mortality
A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I2 = 0.0%, P = 0.573).
Study Quality
Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;32,34,36 while this was unclear in 2 studies.21,30 Three studies reported baseline characteristics.21,30,36 Two studies reported power calculations;33,37 however, one study had a sample size that was less than that required to achieve the target power.37 The Jadad scores ranged from 1to 5.
DISCUSSION
This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.
The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).39 However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).18 The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.40 The translation of evidence into clinical practice is known to be slow;41 however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.
The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?42 The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.5 It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.46 These general principles of human factors engineering46 provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.
Checklists used in aviation are perhaps best known12 and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.12 Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.7 Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).19 Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.18 These recommendations are nicely illustrated by Urbach et al.’s20 and O’Leary et al.’s47 evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.48 This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation49 and realist reviews of published studies50 may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.
There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.40 While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,24 reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.
In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.
Acknowledgments
We would like to thank Diane Lorenzetti for her help with the development of the search strategy.
Disclosure: The authors have no known conflicts of interest to declare.
Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.
Authors’ Contributions
HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.
1. World Health Organization. Patient safety. Available at: http://www.who.int/patientsafety/about/en/. Accessed June 21, 2016.
2. Institute of Medicine. To err is human: Building a safer health system. In: Kohn L, Corrigan J, Donaldson M, eds. Institute of Medicine-Committee on Quality of Health Care in America. Washington DC: National Academy Press; 1999:86-101. PubMed
3. Institute of Medicine Committee on the Quality of Health Care in America. Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academy Press; 2001. PubMed
4. Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “to err is human” report and the patient safety literature. Qual Saf Health Care. 2006; 15(3):174-178. PubMed
5. Winters BD, Gurses AP, Lehmann H, Sexton JB, Rampersad CJ, Pronovost, PJ. Clinical review: Checklists - translating evidence into practice. Crit Care. 2009; 13(6):210. PubMed
6. Ely EW, Bennett PA, Bowton DL, Murphy SM, Florance AM, Haponik EF. Large scale implementation of a respiratory therapist-driven protocol for ventilator weaning. Am J Respir Crit Care Med. 1999; 159(2):439-446. PubMed
7. Gawande A. The checklist manifesto: How to get things right. Great Britain: Profile Books LTD; 2010.
8. Pronovost P, Vohr E. Safe patients, smart hospitals. New York, NY: Hudson Street Press; 2010.
9. Hughes RG. Advances in patient safety: Tools and strategies for quality improvement and patient safety. In: Hughes RG, ed. Patient safety and quality: An evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008. PubMed
10. Henriksen K, Oppenheimer C, Leape LL, et al. Envisioning patient safety in the year 2025: Eight perspectives. In: Henriksen K, Battles JB, Keyes MA, et al., eds. Advances in patient safety: New directions and alternative approaches. Rockville, MD: Agency for Healthcare Research and Quality; 2008. PubMed
11. Gaba DM, Howard SK. Patient safety: Fatigue among clinicians and the safety of patients. N Engl J Med. 2002; 347(16):1249-1255. PubMed
12. Degani A, Wiener EL. Cockpit checklists: Concepts, design, and use. Human Factors: The Journal of the Human Factors and Ergonomics Society 1993; 35(2):345-359.
13. Swain AD, Guttmann HE. Handbook of human reliability analysis with emphasis on nuclear power plant applications: Final report. Washington, DC: U.S. Nuclear Regulatroy Commission; 1983.
14. de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010; 363(20):1928-1937. PubMed
15. Bergs J, Hellings J, Cleemput I, et al. Systematic review and meta-analysis of the effect of the world health organization surgical safety checklist on postoperative complications. Br J Surg. 2014; 101(3):150-158. PubMed
16. Pucher PH, Johnston MJ, Aggarwal R, Arora S, Darzi A. Effectiveness of interventions to improve patient handover in surgery: A systematic review. Surgery. 2015; 158(1):85-95. PubMed
17. Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: A systematic review of the qualitative evidence. BMJ Qual Saf. 2015; 23(12):776-786. PubMed
18. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: A systematic review of impacts and implementation. BMJ Qual Saf. 2014; 23(4):299-318. PubMed
19. Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of using a safety checklist on patient complications after surgery: A systematic review and meta-analysis. Anesthesiology. 2014; 120(6):1380-1389. PubMed
20. Reames BN, Krell RW, Campbell DA Jr, Dimick JB. A checklist-based intervention to improve surgical outcomes in michigan: Evaluation of the keystone surgery program. JAMA Surg. 2015; 150(3):208-215. PubMed
21. Chaudhary N, Varma V, Kapoor S, Mehta N, Kumaran V, Nundy S. Implementation of a surgical safety checklist and postoperative outcomes: A prospective randomized controlled study. J Gastrointest Surg. 2015; 19(5):935-942. PubMed
22. Reames BN, Krell RW, Campbell DA, Jr., Dimick JB. A checklist-based intervention to improve surgical outcomes in Michigan: Evaluation of the Keystone Surgery program. JAMA surgery. 2015; 150(3):208-215. PubMed
23. Liberati A, Altman DG, Tetzlaff J, et al. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann Intern Med. 2009; 151(4):W65-94. PubMed
24. The Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Oxford, UK: The Cochrane Collaboration, 2011.
25. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33(1):159-174. PubMed
26. Berlin JA. Does blinding of readers affect the results of meta-analyses? University of pennsylvania meta-analysis blinding study group. Lancet 1997;350(9072):185-186. PubMed
27. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17(1):1-12. PubMed
28. Popay J, Roberts H, Sowden A, et al. Guidance on the conduct of narrative synthesis in systematic reviews: A product form the esrc methods programme. Available at: https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000.pdf. Accessed June 17, 2016.
29. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177-188. PubMed
30. Basoor A, Doshi NC, Cotant JF, et al. Decreased readmissions and improved quality of care with the use of an inexpensive checklist in heart failure. Congest Heart Fail. 2013; 19(4):200-206. PubMed
31. Ong MS, Magrabi F, Post J, et al. Communication interventions to improve adherence to infection control precautions: A randomised crossover trial. BMC Infect Dis. 2013; 13:72. PubMed
32. Salzwedel C, Bartz HJ, Kuhnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: A randomized controlled trial. Int J Qual Health Care. 2013; 25(2):176-181.
33. Implement Sci. 50. Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: Illustrating the method for implementation research. 2008; 337.BMJ. PubMed
49. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: The new medical research council guidance. 2014; 9(9):e108585.PloS one. PubMed
48. Gagliardi AR, Straus SE, Shojania KG, Urbach DR. Multiple interacting factors influence adherence, and outcomes associated with surgical safety checklists: A qualitative study.
2016; 188(9):E191-E198.CMAJ. PubMed
47. O’Leary JD, Wijeysundera DN, Crawford MW. Effect of surgical safety checklists on pediatric surgical complications in Ontario. Rockville, MD: Agency for Healthcare Research and Quality; 2013.Human factors and ergonomics. Making health care safer ii: An updated critical analysis of the evidence for patient safety practices. PubMed
46. Carayon P, Xie A, Kianfar SH. 2005; 16(1):70-76.Psychol Sci. 2004; 30(4):689-707.
45. Halford GS, Baker R, McCredden JE, Bain JD. How many variables can humans process? J Exp Psychol Hum Percept Perform. PubMed
44. Oberauer K, Kliegl R. Simultaneous cognitive operations in working memory after dual-task practice. 1956; 63(2):81-97.Psychol Rev. PubMed
43. Miller GA. The magical number seven plus or minus two: Some limits on our capacity for processing information. 2008; 20(1):22-30.Int J Qual Health Care. PubMed
42. Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. 2011; 104(12):510-520.J R Soc Med. PubMed
41. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: Understanding time lags in translational research. 2015; 24(5):325-336.BMJ Qual Saf. PubMed
40. Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: A brief overview of possible study types. Washington (DC): National Academies Press (US); 2013.Best care at lower cost: The path to continuously learning health care in america. PubMed
39. Institute of Medicine. Committee on the learning health care system in America. In: Smith M, Saunders R, Stuckhardt L, et al., eds. 2004; 240(2):205-213.Ann Surg. PubMed
38. Dindo D, Demartines N, Clavien P-A. Classification of surgical complications: A new proposal with evaluation in a cohort of 6336 patients and results of a survey. 2013; 66(3):157-162.Can J Hosp Pharm. PubMed
37. Masson SC, Mabasa VH, Malyuk DL, Perrott JL. Validity evidence for fasthug-maidens, a mnemonic for identifying drug-related problems in the intensive care unit. 2015; 261(5):821-828.Ann Surg. PubMed
36. Haugen AS, Softeland E, Almeland SK, et al. Effect of the world health organization checklist on patient outcomes: A stepped wedge cluster randomized controlled trial.
2016; 12(2):199-205.Future Oncol. PubMed
35. Gentili M, Clerico M, Spizzichino M, Fanelli G. Use of a checklist to improve pain control in hospitalized cancer patients: the 38Checkpain project. 2016; 32:170-174.J Crit Care. PubMed
34. Salzwedel C, Mai V, Punke MA, Kluge S, Reuter DA. The effect of a checklist on the quality of patient handover from the operating room to the intensive care unit: A randomized controlled trial. 2016; 315(14):1480-1490.JAMA. PubMed
33. The Writing Group for CHECKLIST-ICU, Cavalcanti AB, Bozza FA, et al. Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial. PubMed
In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.1-4 Numerous strategies to improve patient safety have been proposed,5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.10,11
Checklists have served as a foundation for the standardization and safety of aviation and nuclear power12,13 and are advocated as simple and effective instruments for ensuring safe care.7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.
METHODS
The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.23
Search Strategy
On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.24 The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.
Eligibility Criteria
We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).
Study Selection
Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.25 Reviewers were not blinded to author or journal names.26
Data Extraction
The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.
Quality Assessment
The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.24 For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.27
Analysis
Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.28 Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).29 Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.
RESULTS
The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%
Study Characteristics
Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,21,30,36 4 studies examined hospital length of stay,21,30,33,36 and 2 studies reported user compliance with the checklist.21,31 Five of the studies reported patient outcomes,21,30,33,35,36 and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).31-34,37
Description of Checklists
Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,21,30,33,35-37 whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT30-35 and the 3 remaining studies used modified versions of previously validated checklists.21,36,37 The number of items included in the checklist ranged from 2 to 54.
Impact of the Checklist
Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)38 and IV (life-threatening)38 postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.21 Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).36 Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).30 Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.35 The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).33 However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).37 Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.31 Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).32 In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).34
30-day Mortality
A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I2 = 0.0%, P = 0.573).
Study Quality
Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;32,34,36 while this was unclear in 2 studies.21,30 Three studies reported baseline characteristics.21,30,36 Two studies reported power calculations;33,37 however, one study had a sample size that was less than that required to achieve the target power.37 The Jadad scores ranged from 1to 5.
DISCUSSION
This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.
The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).39 However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).18 The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.40 The translation of evidence into clinical practice is known to be slow;41 however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.
The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?42 The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.5 It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.46 These general principles of human factors engineering46 provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.
Checklists used in aviation are perhaps best known12 and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.12 Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.7 Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).19 Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.18 These recommendations are nicely illustrated by Urbach et al.’s20 and O’Leary et al.’s47 evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.48 This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation49 and realist reviews of published studies50 may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.
There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.40 While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,24 reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.
In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.
Acknowledgments
We would like to thank Diane Lorenzetti for her help with the development of the search strategy.
Disclosure: The authors have no known conflicts of interest to declare.
Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.
Authors’ Contributions
HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.
In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.1-4 Numerous strategies to improve patient safety have been proposed,5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.10,11
Checklists have served as a foundation for the standardization and safety of aviation and nuclear power12,13 and are advocated as simple and effective instruments for ensuring safe care.7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.
METHODS
The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.23
Search Strategy
On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.24 The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.
Eligibility Criteria
We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).
Study Selection
Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.25 Reviewers were not blinded to author or journal names.26
Data Extraction
The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.
Quality Assessment
The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.24 For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.27
Analysis
Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.28 Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).29 Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.
RESULTS
The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%
Study Characteristics
Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,21,30,36 4 studies examined hospital length of stay,21,30,33,36 and 2 studies reported user compliance with the checklist.21,31 Five of the studies reported patient outcomes,21,30,33,35,36 and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).31-34,37
Description of Checklists
Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,21,30,33,35-37 whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT30-35 and the 3 remaining studies used modified versions of previously validated checklists.21,36,37 The number of items included in the checklist ranged from 2 to 54.
Impact of the Checklist
Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)38 and IV (life-threatening)38 postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.21 Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).36 Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).30 Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.35 The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).33 However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).37 Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.31 Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).32 In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).34
30-day Mortality
A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I2 = 0.0%, P = 0.573).
Study Quality
Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;32,34,36 while this was unclear in 2 studies.21,30 Three studies reported baseline characteristics.21,30,36 Two studies reported power calculations;33,37 however, one study had a sample size that was less than that required to achieve the target power.37 The Jadad scores ranged from 1to 5.
DISCUSSION
This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.
The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).39 However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).18 The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.40 The translation of evidence into clinical practice is known to be slow;41 however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.
The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?42 The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.5 It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.46 These general principles of human factors engineering46 provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.
Checklists used in aviation are perhaps best known12 and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.12 Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.7 Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).19 Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.18 These recommendations are nicely illustrated by Urbach et al.’s20 and O’Leary et al.’s47 evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.48 This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation49 and realist reviews of published studies50 may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.
There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.40 While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,24 reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.
In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.
Acknowledgments
We would like to thank Diane Lorenzetti for her help with the development of the search strategy.
Disclosure: The authors have no known conflicts of interest to declare.
Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.
Authors’ Contributions
HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.
1. World Health Organization. Patient safety. Available at: http://www.who.int/patientsafety/about/en/. Accessed June 21, 2016.
2. Institute of Medicine. To err is human: Building a safer health system. In: Kohn L, Corrigan J, Donaldson M, eds. Institute of Medicine-Committee on Quality of Health Care in America. Washington DC: National Academy Press; 1999:86-101. PubMed
3. Institute of Medicine Committee on the Quality of Health Care in America. Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academy Press; 2001. PubMed
4. Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “to err is human” report and the patient safety literature. Qual Saf Health Care. 2006; 15(3):174-178. PubMed
5. Winters BD, Gurses AP, Lehmann H, Sexton JB, Rampersad CJ, Pronovost, PJ. Clinical review: Checklists - translating evidence into practice. Crit Care. 2009; 13(6):210. PubMed
6. Ely EW, Bennett PA, Bowton DL, Murphy SM, Florance AM, Haponik EF. Large scale implementation of a respiratory therapist-driven protocol for ventilator weaning. Am J Respir Crit Care Med. 1999; 159(2):439-446. PubMed
7. Gawande A. The checklist manifesto: How to get things right. Great Britain: Profile Books LTD; 2010.
8. Pronovost P, Vohr E. Safe patients, smart hospitals. New York, NY: Hudson Street Press; 2010.
9. Hughes RG. Advances in patient safety: Tools and strategies for quality improvement and patient safety. In: Hughes RG, ed. Patient safety and quality: An evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008. PubMed
10. Henriksen K, Oppenheimer C, Leape LL, et al. Envisioning patient safety in the year 2025: Eight perspectives. In: Henriksen K, Battles JB, Keyes MA, et al., eds. Advances in patient safety: New directions and alternative approaches. Rockville, MD: Agency for Healthcare Research and Quality; 2008. PubMed
11. Gaba DM, Howard SK. Patient safety: Fatigue among clinicians and the safety of patients. N Engl J Med. 2002; 347(16):1249-1255. PubMed
12. Degani A, Wiener EL. Cockpit checklists: Concepts, design, and use. Human Factors: The Journal of the Human Factors and Ergonomics Society 1993; 35(2):345-359.
13. Swain AD, Guttmann HE. Handbook of human reliability analysis with emphasis on nuclear power plant applications: Final report. Washington, DC: U.S. Nuclear Regulatroy Commission; 1983.
14. de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010; 363(20):1928-1937. PubMed
15. Bergs J, Hellings J, Cleemput I, et al. Systematic review and meta-analysis of the effect of the world health organization surgical safety checklist on postoperative complications. Br J Surg. 2014; 101(3):150-158. PubMed
16. Pucher PH, Johnston MJ, Aggarwal R, Arora S, Darzi A. Effectiveness of interventions to improve patient handover in surgery: A systematic review. Surgery. 2015; 158(1):85-95. PubMed
17. Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: A systematic review of the qualitative evidence. BMJ Qual Saf. 2015; 23(12):776-786. PubMed
18. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: A systematic review of impacts and implementation. BMJ Qual Saf. 2014; 23(4):299-318. PubMed
19. Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of using a safety checklist on patient complications after surgery: A systematic review and meta-analysis. Anesthesiology. 2014; 120(6):1380-1389. PubMed
20. Reames BN, Krell RW, Campbell DA Jr, Dimick JB. A checklist-based intervention to improve surgical outcomes in michigan: Evaluation of the keystone surgery program. JAMA Surg. 2015; 150(3):208-215. PubMed
21. Chaudhary N, Varma V, Kapoor S, Mehta N, Kumaran V, Nundy S. Implementation of a surgical safety checklist and postoperative outcomes: A prospective randomized controlled study. J Gastrointest Surg. 2015; 19(5):935-942. PubMed
22. Reames BN, Krell RW, Campbell DA, Jr., Dimick JB. A checklist-based intervention to improve surgical outcomes in Michigan: Evaluation of the Keystone Surgery program. JAMA surgery. 2015; 150(3):208-215. PubMed
23. Liberati A, Altman DG, Tetzlaff J, et al. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann Intern Med. 2009; 151(4):W65-94. PubMed
24. The Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Oxford, UK: The Cochrane Collaboration, 2011.
25. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33(1):159-174. PubMed
26. Berlin JA. Does blinding of readers affect the results of meta-analyses? University of pennsylvania meta-analysis blinding study group. Lancet 1997;350(9072):185-186. PubMed
27. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17(1):1-12. PubMed
28. Popay J, Roberts H, Sowden A, et al. Guidance on the conduct of narrative synthesis in systematic reviews: A product form the esrc methods programme. Available at: https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000.pdf. Accessed June 17, 2016.
29. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177-188. PubMed
30. Basoor A, Doshi NC, Cotant JF, et al. Decreased readmissions and improved quality of care with the use of an inexpensive checklist in heart failure. Congest Heart Fail. 2013; 19(4):200-206. PubMed
31. Ong MS, Magrabi F, Post J, et al. Communication interventions to improve adherence to infection control precautions: A randomised crossover trial. BMC Infect Dis. 2013; 13:72. PubMed
32. Salzwedel C, Bartz HJ, Kuhnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: A randomized controlled trial. Int J Qual Health Care. 2013; 25(2):176-181.
33. Implement Sci. 50. Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: Illustrating the method for implementation research. 2008; 337.BMJ. PubMed
49. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: The new medical research council guidance. 2014; 9(9):e108585.PloS one. PubMed
48. Gagliardi AR, Straus SE, Shojania KG, Urbach DR. Multiple interacting factors influence adherence, and outcomes associated with surgical safety checklists: A qualitative study.
2016; 188(9):E191-E198.CMAJ. PubMed
47. O’Leary JD, Wijeysundera DN, Crawford MW. Effect of surgical safety checklists on pediatric surgical complications in Ontario. Rockville, MD: Agency for Healthcare Research and Quality; 2013.Human factors and ergonomics. Making health care safer ii: An updated critical analysis of the evidence for patient safety practices. PubMed
46. Carayon P, Xie A, Kianfar SH. 2005; 16(1):70-76.Psychol Sci. 2004; 30(4):689-707.
45. Halford GS, Baker R, McCredden JE, Bain JD. How many variables can humans process? J Exp Psychol Hum Percept Perform. PubMed
44. Oberauer K, Kliegl R. Simultaneous cognitive operations in working memory after dual-task practice. 1956; 63(2):81-97.Psychol Rev. PubMed
43. Miller GA. The magical number seven plus or minus two: Some limits on our capacity for processing information. 2008; 20(1):22-30.Int J Qual Health Care. PubMed
42. Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. 2011; 104(12):510-520.J R Soc Med. PubMed
41. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: Understanding time lags in translational research. 2015; 24(5):325-336.BMJ Qual Saf. PubMed
40. Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: A brief overview of possible study types. Washington (DC): National Academies Press (US); 2013.Best care at lower cost: The path to continuously learning health care in america. PubMed
39. Institute of Medicine. Committee on the learning health care system in America. In: Smith M, Saunders R, Stuckhardt L, et al., eds. 2004; 240(2):205-213.Ann Surg. PubMed
38. Dindo D, Demartines N, Clavien P-A. Classification of surgical complications: A new proposal with evaluation in a cohort of 6336 patients and results of a survey. 2013; 66(3):157-162.Can J Hosp Pharm. PubMed
37. Masson SC, Mabasa VH, Malyuk DL, Perrott JL. Validity evidence for fasthug-maidens, a mnemonic for identifying drug-related problems in the intensive care unit. 2015; 261(5):821-828.Ann Surg. PubMed
36. Haugen AS, Softeland E, Almeland SK, et al. Effect of the world health organization checklist on patient outcomes: A stepped wedge cluster randomized controlled trial.
2016; 12(2):199-205.Future Oncol. PubMed
35. Gentili M, Clerico M, Spizzichino M, Fanelli G. Use of a checklist to improve pain control in hospitalized cancer patients: the 38Checkpain project. 2016; 32:170-174.J Crit Care. PubMed
34. Salzwedel C, Mai V, Punke MA, Kluge S, Reuter DA. The effect of a checklist on the quality of patient handover from the operating room to the intensive care unit: A randomized controlled trial. 2016; 315(14):1480-1490.JAMA. PubMed
33. The Writing Group for CHECKLIST-ICU, Cavalcanti AB, Bozza FA, et al. Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial. PubMed
1. World Health Organization. Patient safety. Available at: http://www.who.int/patientsafety/about/en/. Accessed June 21, 2016.
2. Institute of Medicine. To err is human: Building a safer health system. In: Kohn L, Corrigan J, Donaldson M, eds. Institute of Medicine-Committee on Quality of Health Care in America. Washington DC: National Academy Press; 1999:86-101. PubMed
3. Institute of Medicine Committee on the Quality of Health Care in America. Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academy Press; 2001. PubMed
4. Stelfox HT, Palmisani S, Scurlock C, Orav EJ, Bates DW. The “to err is human” report and the patient safety literature. Qual Saf Health Care. 2006; 15(3):174-178. PubMed
5. Winters BD, Gurses AP, Lehmann H, Sexton JB, Rampersad CJ, Pronovost, PJ. Clinical review: Checklists - translating evidence into practice. Crit Care. 2009; 13(6):210. PubMed
6. Ely EW, Bennett PA, Bowton DL, Murphy SM, Florance AM, Haponik EF. Large scale implementation of a respiratory therapist-driven protocol for ventilator weaning. Am J Respir Crit Care Med. 1999; 159(2):439-446. PubMed
7. Gawande A. The checklist manifesto: How to get things right. Great Britain: Profile Books LTD; 2010.
8. Pronovost P, Vohr E. Safe patients, smart hospitals. New York, NY: Hudson Street Press; 2010.
9. Hughes RG. Advances in patient safety: Tools and strategies for quality improvement and patient safety. In: Hughes RG, ed. Patient safety and quality: An evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008. PubMed
10. Henriksen K, Oppenheimer C, Leape LL, et al. Envisioning patient safety in the year 2025: Eight perspectives. In: Henriksen K, Battles JB, Keyes MA, et al., eds. Advances in patient safety: New directions and alternative approaches. Rockville, MD: Agency for Healthcare Research and Quality; 2008. PubMed
11. Gaba DM, Howard SK. Patient safety: Fatigue among clinicians and the safety of patients. N Engl J Med. 2002; 347(16):1249-1255. PubMed
12. Degani A, Wiener EL. Cockpit checklists: Concepts, design, and use. Human Factors: The Journal of the Human Factors and Ergonomics Society 1993; 35(2):345-359.
13. Swain AD, Guttmann HE. Handbook of human reliability analysis with emphasis on nuclear power plant applications: Final report. Washington, DC: U.S. Nuclear Regulatroy Commission; 1983.
14. de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010; 363(20):1928-1937. PubMed
15. Bergs J, Hellings J, Cleemput I, et al. Systematic review and meta-analysis of the effect of the world health organization surgical safety checklist on postoperative complications. Br J Surg. 2014; 101(3):150-158. PubMed
16. Pucher PH, Johnston MJ, Aggarwal R, Arora S, Darzi A. Effectiveness of interventions to improve patient handover in surgery: A systematic review. Surgery. 2015; 158(1):85-95. PubMed
17. Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: A systematic review of the qualitative evidence. BMJ Qual Saf. 2015; 23(12):776-786. PubMed
18. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: A systematic review of impacts and implementation. BMJ Qual Saf. 2014; 23(4):299-318. PubMed
19. Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of using a safety checklist on patient complications after surgery: A systematic review and meta-analysis. Anesthesiology. 2014; 120(6):1380-1389. PubMed
20. Reames BN, Krell RW, Campbell DA Jr, Dimick JB. A checklist-based intervention to improve surgical outcomes in michigan: Evaluation of the keystone surgery program. JAMA Surg. 2015; 150(3):208-215. PubMed
21. Chaudhary N, Varma V, Kapoor S, Mehta N, Kumaran V, Nundy S. Implementation of a surgical safety checklist and postoperative outcomes: A prospective randomized controlled study. J Gastrointest Surg. 2015; 19(5):935-942. PubMed
22. Reames BN, Krell RW, Campbell DA, Jr., Dimick JB. A checklist-based intervention to improve surgical outcomes in Michigan: Evaluation of the Keystone Surgery program. JAMA surgery. 2015; 150(3):208-215. PubMed
23. Liberati A, Altman DG, Tetzlaff J, et al. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann Intern Med. 2009; 151(4):W65-94. PubMed
24. The Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Oxford, UK: The Cochrane Collaboration, 2011.
25. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33(1):159-174. PubMed
26. Berlin JA. Does blinding of readers affect the results of meta-analyses? University of pennsylvania meta-analysis blinding study group. Lancet 1997;350(9072):185-186. PubMed
27. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17(1):1-12. PubMed
28. Popay J, Roberts H, Sowden A, et al. Guidance on the conduct of narrative synthesis in systematic reviews: A product form the esrc methods programme. Available at: https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000.pdf. Accessed June 17, 2016.
29. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177-188. PubMed
30. Basoor A, Doshi NC, Cotant JF, et al. Decreased readmissions and improved quality of care with the use of an inexpensive checklist in heart failure. Congest Heart Fail. 2013; 19(4):200-206. PubMed
31. Ong MS, Magrabi F, Post J, et al. Communication interventions to improve adherence to infection control precautions: A randomised crossover trial. BMC Infect Dis. 2013; 13:72. PubMed
32. Salzwedel C, Bartz HJ, Kuhnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: A randomized controlled trial. Int J Qual Health Care. 2013; 25(2):176-181.
33. Implement Sci. 50. Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: Illustrating the method for implementation research. 2008; 337.BMJ. PubMed
49. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: The new medical research council guidance. 2014; 9(9):e108585.PloS one. PubMed
48. Gagliardi AR, Straus SE, Shojania KG, Urbach DR. Multiple interacting factors influence adherence, and outcomes associated with surgical safety checklists: A qualitative study.
2016; 188(9):E191-E198.CMAJ. PubMed
47. O’Leary JD, Wijeysundera DN, Crawford MW. Effect of surgical safety checklists on pediatric surgical complications in Ontario. Rockville, MD: Agency for Healthcare Research and Quality; 2013.Human factors and ergonomics. Making health care safer ii: An updated critical analysis of the evidence for patient safety practices. PubMed
46. Carayon P, Xie A, Kianfar SH. 2005; 16(1):70-76.Psychol Sci. 2004; 30(4):689-707.
45. Halford GS, Baker R, McCredden JE, Bain JD. How many variables can humans process? J Exp Psychol Hum Percept Perform. PubMed
44. Oberauer K, Kliegl R. Simultaneous cognitive operations in working memory after dual-task practice. 1956; 63(2):81-97.Psychol Rev. PubMed
43. Miller GA. The magical number seven plus or minus two: Some limits on our capacity for processing information. 2008; 20(1):22-30.Int J Qual Health Care. PubMed
42. Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. 2011; 104(12):510-520.J R Soc Med. PubMed
41. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: Understanding time lags in translational research. 2015; 24(5):325-336.BMJ Qual Saf. PubMed
40. Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: A brief overview of possible study types. Washington (DC): National Academies Press (US); 2013.Best care at lower cost: The path to continuously learning health care in america. PubMed
39. Institute of Medicine. Committee on the learning health care system in America. In: Smith M, Saunders R, Stuckhardt L, et al., eds. 2004; 240(2):205-213.Ann Surg. PubMed
38. Dindo D, Demartines N, Clavien P-A. Classification of surgical complications: A new proposal with evaluation in a cohort of 6336 patients and results of a survey. 2013; 66(3):157-162.Can J Hosp Pharm. PubMed
37. Masson SC, Mabasa VH, Malyuk DL, Perrott JL. Validity evidence for fasthug-maidens, a mnemonic for identifying drug-related problems in the intensive care unit. 2015; 261(5):821-828.Ann Surg. PubMed
36. Haugen AS, Softeland E, Almeland SK, et al. Effect of the world health organization checklist on patient outcomes: A stepped wedge cluster randomized controlled trial.
2016; 12(2):199-205.Future Oncol. PubMed
35. Gentili M, Clerico M, Spizzichino M, Fanelli G. Use of a checklist to improve pain control in hospitalized cancer patients: the 38Checkpain project. 2016; 32:170-174.J Crit Care. PubMed
34. Salzwedel C, Mai V, Punke MA, Kluge S, Reuter DA. The effect of a checklist on the quality of patient handover from the operating room to the intensive care unit: A randomized controlled trial. 2016; 315(14):1480-1490.JAMA. PubMed
33. The Writing Group for CHECKLIST-ICU, Cavalcanti AB, Bozza FA, et al. Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial. PubMed
© 2017 Society of Hospital Medicine
Computerized Physician Handoff Tools
Physician handoff is a common and essential component of daily patient care that includes transfer of important clinical patient information and accountability of patient care. Thus, high‐quality physician handoffs are crucial to ensure patient safety and continuity of patient care, especially with the new resident work hour restriction in North America.[1, 2] As such, healthcare organizations including the World Health Organization[3] have issued specific goals and organizational challenges to improve the effectiveness and coordination of communication among the care/service providers and with the recipients of care/service across the continuum in healthcare.[4, 5]
It has been well‐documented that physician handoffs in hospital settings are often unstructured and not standardized, which leads to medical errors and jeopardizes patient safety.[2, 6, 7, 8, 9, 10, 11, 12] This lack of standardization of physician handoff for hospitalized patients occurs in every major in‐hospital service and affects trainees and staff.[2, 6, 7, 9, 10, 12, 13] It has been demonstrated in healthcare and in other domains that a standardized handoff protocol that involves both verbal communication and written handoff documents is likely to be an effective method of handoff to decrease miscommunication and associated errors.[14, 15, 16, 17] Computerized physician handoff tools (CHTs) have been increasingly deployed to address these challenges and have quickly gained popularity among physicians for documenting patient information during physician handoff for hospitalized patients.[18] CHTs can be an complementary part of electronic medical record (EMR) systems, but not a substitute since their focus is to deliver concise and essential information vital for patient care during interfaces of patient care.
Two recent systematic reviews have examined information technology (IT) systems to promote the handoff process in healthcare.[17, 19] However, to our knowledge, there has not been a systematic review of the potential role of CHT in physician handoff and quality of patient care for hospitalized patients. We therefore conducted a systematic review to examine the current evidence for CHTs in physician handoff for hospitalized patients, focusing specifically on potential effects on continuity of patient care, physician work efficiency, quality of handoffs, and patient outcomes.
METHODS
Criteria for Considering Eligible Studies
We included randomized controlled trials, controlled clinical trial, quasi‐experimental studies, and controlled beforeafter studies that evaluated CHTs during physician handoff of hospitalized patients. Studies needed to report patient outcomes (adverse events, missing patients at rounds, or in‐hospital mortality), physician work efficiency, quality of handoff (accuracy, consistency, or completeness), continuity of care, or physician satisfaction. Articles that met all these inclusion criteria were considered to be eligible for the review. We excluded review articles, commentaries, case reports, and retrospective studies.
Search Strategy
CHTs were defined as computer‐based platforms, designed specifically for the purpose of physician handoff, to allow distributed access and synchronous archiving of patient information via Internet protocols (ie, electronic tool to allow physician data access and data entry for handoff from different computers at multiple locations within the authorized hospitals or clinics). A search strategy was developed based on a MEDLINE search format combined with our inclusion criteria and with this definition of CHTs. We used search terms related to physician communication and information technology, and relevant Medical Subjects Headings, which include handover, handoff, signoff, sign‐over, off‐duty, post‐call, computerized, Web‐based, communication tool. The databases, including MEDLINE, PUBMED, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane database for systematic reviews, and the Cochrane CENTRAL register of controlled trials, were initially searched from 1985 to December 2011 in all languages. The Cochrane Collaboration filter for controlled interventional studies was used to select the above‐mentioned interventional trial designs. In addition, the first 2 authors hand searched the references of included articles and relevant systematic reviews.
Screening for Eligible Studies
All articles identified in the database searches described above were included for screening in 2 stages. First, 2 reviewers (P.L., S.A.) independently reviewed the title and abstracts of the identified articles for eligibility. The articles selected in the first stage of screening were then further assessed by a full‐text review independently by the 2 reviewers. Any discrepancy was resolved by consensus or by involvement of a third reviewer (C.T.).
Data Abstraction and Analysis
Data abstraction from selected studies was conducted independently by 3 authors based on a predefined template. All discrepancies in this stage were resolved by consensus among the 3 authors. For each study, we analyzed study design, data collection, intervention, main outcomes, and components of physician handoffs in the study. Due to heterogeneity of study outcomes, measures used, and results, a meta‐analysis was not performed. Study outcomes, which included adverse events, missing patients at rounds, time spent on rounding patient, accuracy, consistency or completeness of handoff information, and continuity of care, were summarized.
RESULTS
Study Selection
A total of 1026 citations were identified in the initial search, of which 1006 studies did not evaluate CHT and were excluded by title and abstract screening. Of the 20 studies evaluated further by full‐text review, 5 were selected for the final analysis. One additional study was identified by hand searching references. The kappa score of inter‐reviewer agreement on article selection in the first stage of screening was 0.7, and for the second stage of article selection, kappa was 1.0. The reasons for exclusion in the second selection step are presented in Figure 1.
Study Characteristics
Of the 6 studies identified, 1 study was a randomized controlled trial[20] and the other 5 were controlled beforeafter studies.[21, 22, 23, 24, 25] All studies were conducted in teaching hospitals in English‐speaking high‐income countries. All were single‐center studies, except the study by Van Eaton et[20] that involved 2 centers. All the studies investigated physician handoffs conducted by trainees. Two studies included staff physicians.[22, 24] Van Eaton et al's study included general medical, general surgical, and subspecialty surgical services.[20] The other 5 studies assessed physician handoffs in family medicine,[25] internal medicine services,[21, 23] a surgical service,[22] and a neonatal intensive care unit.[24] The study by Van Eaton et al[20] enrolled the largest study population. The intervention or observation phase ranged from 1 month[20] to 6 months[24] (Table 1).
| Study | Design | Setting | Target Services | Intervention Group | Control Group | Data Collection and Validation |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Beforeafter study | 150‐bed urban hospital in USA | Family Medicine Residents (N = 7) | Patient no. not reported 1 mo of intervention No CHT training prior to the intervention reported | Patient no. not reported Traditional handoff note (on index card or previous list) Components of handoff note not reported | Questionnaire No data validation |
| Peterson et al[21] (1998) | Beforeafter study | 720‐bed tertiary care hospital in USA | All Internal Medicine Services Residents (N = 99) | 3747 patients 4 mo of intervention 8 wk of run‐in period | 1874 patients Handwritten handoff Components of handoff note not reported | Self‐report using e‐mail, report card, in person chart review for unreported adverse events in 250 samples |
| Van Eaton et al[20] (2005) | Randomized cross‐over trial | 450‐bed tertiary care hospital and a 368‐bed trauma center in USA | General Medicine, General Surgery, and Subspecialties Trauma Residents (N = 7 teams) | 8018 patients 14 wk of randomized crossover period 6 wk of run‐in period | 7569 patients Individual written lists, cards, a team‐developed computer‐generated spreadsheet Components of handoff note not reported | Telephone interview and anonymous online survey No validation of data |
| Cheah et al[22] (2005) | Beforesfter atudy | A 400‐bed regional teaching hospital in Australia | General Surgery Registrars and Residents (N = 714) | Patient no. not reported 3 mo of observation period (for weekend coverage only) No CHT training prior to the intervention reported | Patient no. not reported No description of pre‐intervention handoff method reported | In‐person interview and survey No validation of data |
| Flanagan et al[23] (2009) | Beforesfter atudy | Tertiary care hospital in USA | Internal Medicine, Medical Intensive Care Unit First‐year Residents (N = 35) | 1264 patient handoff forms 1 mo of observation Orientation session and 1 cross‐over shift of run‐in period | Patient no. not reported No description of pre‐CHT implementation handoff method reported | In‐person interview and survey No validation of data |
| Palma et al[24] (2011) | Beforeafter study | 304‐bed quaternary care women and children hospital in USA | NICU Attendings, Residents, Nursing staffs (N = 4652) | Patient no. not reported 6 mo of intervention of NICU handoff tool Instruction document by e‐mail and informal instructional session | Patient no. not reported A Microsoft‐based standalone handoff tool or EMR integrated Medical/Surgical handoff tool Components of handoff note not reported | Online survey No validation of data |
CHT Characteristics
Three CHTs were standalone applications designed specifically for physician handoffs.[20, 22, 25] The other 3 CHTs were add‐on functions to existing hospital Electric Medical Record (EMR) systems.[21, 23, 24] All CHTs except one[25] interfaced with existing EMR systems, allowing for variable degrees of data transfer depending on CHT design and the functionalities of the EMR systems. CHT users were actively involved in designing and modifying the CHTs in most of the studies.[20, 21, 23, 25] The characteristics of the CHTs were summarized in Table 2.
| Study | CHT Design | EMR Interface | Physician Daily Progress Note | Participants' Role in CHT Design | Components of CHT | Components That Require Manual Input |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Standalone application | No interface | Paper‐based | Designing | Patient demographics Medications Diagnosis Problem lists Comment line | All the information |
| Peterson et al[21] (1998) | A part of existing EMR | Bi‐directional interface | Paper‐based | Designing | Patient demographics Current medication Allergy Code status Recent lab value A problem list A to do list | A problem list A to do list |
| Van Eaton et al[20] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Designing and modifying | Patient demographics Diagnosis Medication Allergy Vital signs Lab and investigation A problem list A to do list | Diagnosis Medication A problem list A to do list |
| Cheah et al[22] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | No | Patient demographics Diagnosis Length of stay Recent investigations Free‐text note (Not standardized) | Free‐text note |
| Flanagan et al[23] (2009) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Evaluating and modifying | Patient demographics Medication Allergy Lab and investigation Physician daily note Free‐text note (not standardized) | Free‐text note (may contain assessment, a problem list, venous access, short‐term concerns and long‐term plan, and follow‐up tasks) |
| Palma et al[24] (2011) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Paper‐based | No | Patient demographics Lab and measurement Free‐text note (not standardized) | Free‐text note (including patient description, active medical issues, ongoing care and a to do list) |
CHT's Impact on Adverse Events
The impact of CHTs on preventable adverse events was evaluated in a single study by Peterson et al.[21] The authors defined an adverse event as an injury due to medical treatment which prolonged hospital stay or produced disability at discharge in the study. Preventability was determined by using a 6‐point scale and assessed independently by 3 reviewers. Fewer adverse events were found after implementation of CHTs (2.38% vs 3.94%, P < 0.001). They also reported nonsignificant reductions in preventable adverse events (1.23% vs 1.72%, P < 0.1) with implementation of the CHT, and preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10). The odds ratio for a patient experiencing a preventable adverse event during cross‐coverage compared to noncross‐coverage time was reduced from 5.2 (95% confidence interval [CI], 1.518.2) to 1.5 (95% CI, 0.29.0) following implementation of the CHT (Table 3).
| Study | Outcomes of Interest | Results | Implication for CHT Design and Use |
|---|---|---|---|
| |||
| Ram and Block[25] (1992) | Physician satisfaction Importance and accessibility of clinical information | Improved physician satisfaction Handoff documentation more legible, more consistent, and more comprehensive Information required to be typed in by residents and not up‐to‐date | The most important data for handoff: a to do list and code status A CHT interfaced with hospital IT system, and in a format that can focus on physician needs |
| Petersen et al[21] (1998) | Adverse event rate Preventable adverse events rate | Fewer adverse events (2.38% vs 3.94%, P < 0.001) Fewer preventable adverse events (1.23% vs 1.72%, P < 0.1) Few preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10) Lower OR of preventable adverse events during cross‐coverage (1.5; 95% CI 0.29.0 vs 5.2; 95% CI 1.58.2) | Active involvement in the design of CHT by house staff likely contributes to high participation and CHT use rate in the study |
| Van Eaton et al[20] (2005) | No. of patients missed on rounds Perception on continuity of care quality and workflow efficiency Daily self‐reported pre‐rounding and rounding times and tasks | Reduced the no. of patients missed on rounds (2.5 patients/team/mo) (P = 0.0001) Spent 40% more time with patients at pre‐rounds Reduced time on team rounds by 1.5 min per patient Reduced time on manual copying at pre‐rounding by 50% Improved handoff quality Improved continuity of care No reduction of overall pre‐rounding time | The largest benefit from CHTs varies between clinical services, from more time assessing patients before rounds in Internal Medicine to reduced backtracking and locating patients in Surgery |
| Cheah et al[22] (2005) | Completeness and usefulness of handoff information Desirability of electronic handoff system | Identified information set for handoff Free text entry in CHT often deficient in particular patient information Concerns of the completeness and consistency of information delivered in CHT | CHT needs to be linked to hospital information system |
| Flanagan et al[23] (2009) | Common data elements of interest extracted during physician handoff Missing data required during handoff Physicians' perception of CHT | Additional important information needed that not included during handoff in 25% cases | Code status, relevant lab data, short‐term concerns, a problem list, and a if‐then list should be included in CHT template A standard form reduces variability of handoff information |
| Palma et al[24] (2011) | Accuracy of handoff information Healthcare provider satisfaction | Improved perceived accuracy of handoff information (91% vs 78%, P <0.01) Improved satisfaction with handoff process (71% vs 35%, P < 0.01) Improved satisfaction with handoff documents (98% vs 91%, P <0.01) More time spent on updating handoff information (1620 min vs 1115 min, P = 0.03) | A discipline‐specific handoff tool results in perceived handoff accuracy and satisfaction A more efficient handoff tool can be achieved by more extensive data transfer from hospital IT system |
CHT's Impact on Physician Work Efficiency
Van Eaton et al's study examined the effect of CHTs on physician work efficiency.[20] Improved physician work efficiency was found following implementation of CHT. Self‐reported time spent on hand‐copying patient information was reduced by 50%, while the portion of time spent on seeing patients during pre‐rounding increased. Similarly, self‐reported time spent on each patient during rounding (routine patient assessment by the primary team) was decreased by 1.5 minutes. Overall, resident physicians subjectively reported an average time saving of 45 minutes daily for junior residents and 30 minutes for senior residents, and 81% of residents reported finishing their work sooner when using CHTs. Although no data were reported in the pre‐CHT period described in the study by Cheah et al, they indicated that work efficiency was felt to be improved because all physicians could locate their patients quickly and were pleased to be able to check patients' lab results in the CHT.[22] Conversely, Palma et al and Ram and Block reported perceived increased work load with CHTs by users due to time spent updating handoff information.[24, 25]
CHT's Impact on Quality of Physician Handoff
Overall quality of physician handoff and completeness of the handoff document was improved in 3 studies.[20, 24, 25] Flanagan et al reported that patient identifiers and medications were extracted most of the time.[23] However, there were concerns regarding consistency,[22] completeness[22, 23] of information provided during physician handoff using CHTs. Palma et al's and Ram and Block's studies[24, 25] commented on the accuracy of patient information communicated during physician handoff. While Ram and Block's study suggested that it may be poorer during the intervention period,[25] Palma et al's study found improved perceived accuracy of handoff information postimplementation of a CHT (98% vs 91%, P < 0.01).[24]
CHT's Impact on Continuity of Patient Care
Using CHTs was associated with a decreased number of patients missed on rounds after handoff (new admitted patients who were not assessed by the primary team in the morning rounds because cross‐covering physicians did not inform the primary team) in Van Eaton et al's study.[20] On the other hand, Cheah et al[22] reported that documented handoffs after physicians returned to duty occurred on 50% of patients who had experienced important clinical events on weekends.
DISCUSSION
Our systematic review identified 6 controlled studies of CHT. Outcome parameters reported in these studies included quality of the handoff (including completeness, accuracy, and consistency), physician time management, continuity of care, adverse events, and missed patients. Our results suggest that while CHT are a promising tool, further evaluation using rigorous study methodologies is needed. These findings are somewhat surprising given increasing popularity of CHTs in daily patient care.[19, 24, 26, 27, 28] This might be due to the fact that IT adoption and use in healthcare is still in a phase of relative infancy,[29] and that the success of adopting IT systems in healthcare depends on various factors.[30]
Roles of CHT in Physician Handoff for Hospitalized Patients
Our study indicates that CHT can potentially improve continuity of patient care by reducing the number of missing patients during rounds following handoff,[20] and similarly improve patient safety by decreasing adverse events and preventable adverse events.[21] Of note, users reported that they were able to spend more time with patients during pre‐rounding[20] which will likely enhance quality and continuity of patient care. However, it is unclear whether these improvements translate into better patient outcomes. Although Peterson et al attempted to minimize the risk of bias by using anonymous reporting and blinding participants to the timing of data collection,[21] adverse events during the intervention period could have been underestimated due to surveillance bias or decreased self‐reporting. Nevertheless, the results suggest that CHTs may have affected quality of patient care in a positive manner from included studies.
The findings from our review also point to a positive impact of CHT on physician work efficiency. Specifically, residents spent less time rounding on patients after handoff and finished their work sooner after introduction of the intervention.[20] Several other published studies on CHT also indicated potential benefits on work efficiency and/or patient safety,[31, 33, 34, 35] although they did not meet the inclusion criteria for our study (prespecified outcomes not reported,[31, 35] or study design[33, 34, 35]). In the studies in which the majority of handoff information was manually typed in the CHT, the work load was perceived to be increased with CHT implementation.[24, 25] On the other hand, the study conducted by Van Eaton et al demonstrated that a CHT that had broad integration with the hospital main IT system, and could automatically transfer important patient information such as medication, medical problems, recent investigation, and vital signs into CHT, quickly gained popularity among residents and staff due to its user‐friendly features.[20] This integration can also potentially reduce miscommunication and associated medical errors during physician handoff. Palma et al's study reported higher perceived workload due to manual entry of patient data.[24] Although the CHT used in their study was developed within their existing EMR system, large amounts of information needed to be manually imputed, and thus increased time spent on updating handoff information. This information included patient demographics, active medical issues, a to do list, and on‐going issues,[24] some of which could be imputed automatically with better CHT design. It is also possible that users spent more time in updating the handoff because they were able to deliver more information using a CHT.[24] However, this may allow cross‐covering physicians to spend less time on looking for patient information from other sources and thus actually decrease workload during cross‐coverage. Although there are numerous factors that could affect physician work efficiency when using a new IT system,[30] it was felt that a well‐designed and easy‐to‐use CHT that is integrated with the hospital information system can improve physician productivity.
The role of CHT in improving quality of handoff is less clear. Three studies[20, 24, 25] found an overall improvement in the quality of handoff after implementation of CHT, such that the handoff information was more complete and more consistent. On the other hand, physicians were concerned about the comprehensiveness of physician handoff after implementation of CHT in 2 studies.[22, 23] In Ram and Block's study,[25] physicians relied heavily on an unstructured free‐text entry system to deliver the majority of patient information that physicians thought to be important. In Flanagan et al's study,[23] resident physicians had to search for alternative sources, such as patient charts and electronic order systems, to obtain vital information in many cases in spite of a structured CHT. As a result, the information available was often not sufficient to help on‐call physicians make patient care decisions.[23]
Implication of CHT Design and Use
It has been demonstrated in many non‐healthcare domains,[15, 36, 37] as well as nursing care,[38] that a standardized handoff protocol is vital to decrease medical errors and improve patient safety. In our review, we found that physicians generally reported being satisfied with the accuracy of handoff information and the overall handoff when using standardized CHTs interfaced with hospital IT systems. This suggests, as recommended by Flanagan et al,[23] Palma et al,[24] and Ram and Block[25] that CHTs be developed with a standardized protocol and wide integration into hospital IT systems.
In order to achieve this goal, key patient information necessary for patient care need to be communicated during physician handoff. As hospitals consist of a wide range of disciplines and specialties with varying cultures and focuses of patient information, it is likely difficult to develop a single panacea CHT template for all the in‐hospital services.[1] This may be even particularly relevant when developing CHTs for different hospital services. However, some patient information appears to be universally important for physician handoff for inpatient care. Key elements, such as patient demographics, diagnosis, outstanding investigation results, code status, a problem list, and a to do list, were noted to be consistently present in the CHTs that were evaluated in our review (Table 2). Other studies have also demonstrated that information items such as a to do list, outstanding investigation results, and patients' code status were regarded as the most important information during physician handoff.[1, 2, 17, 23, 39, 40] Based on these findings, a potential solution for CHT standardization would be to develop a core CHT which includes the universally important components of physician handoff identified in this review, and provides options for adding well‐categorized service‐specific information as needed (eg, type and date of surgical procedures for surgical patients). It also appears that active involvement of physicians in CHT design and modification facilitates successful implementation of CHT, as demonstrated in Van Eaton et al's and Peterson et al's studies.[20, 21]
It is difficult to recommend metrics for CHT evaluation based on the limited literature identified in our review. However, it appears to be reasonable to consider integration into existing IT system, user friendly features, impact on quality of handoff documents, work efficiency, and processes and outcomes of patient care when assessing CHTs.
Limitations
There are several limitations in the studies included in our review. None of the studies were multi‐centered. The majority of the included studies had a beforeafter design.[21, 22, 23, 24, 25] Some studies did not have user training or a run in period to ensure familiarity of CHTs by users.[22, 24, 25] None of the studies described the key components of handoff in the control groups, or used quality control measurements for user familiarity with the CHTs. Furthermore, outcomes reported by the studies were heterogeneous, subjective, based on participant self‐report, and not independently validated.
Our review also has also several limitations. First, in spite of a comprehensive search effort, it is possible that we failed to identify all relevant articles. However, this is unlikely, given that we searched multiple databases and performed hand searches of all references identified from the included articles, as well as content‐related previously published systematic reviews. Second, we were not able to perform a meta‐analysis, given the heterogeneity seen in outcomes assessed across studies, measures applied, and results presented.
CONCLUSIONS AND IMPLICATIONS FOR PRACTICE
Although the current literature suggests that implementation of CHTs is likely to improve physician work efficiency, satisfaction, and quality of patient care during physician handoff for hospitalized patients, the evidence supporting these potential benefits is limited. Furthermore, it is unknown what impacts CHTs may have on clinical outcomes, such as hospital length of stay and mortality. Further studies with larger sample size, multiple center involvement, and more objective patient outcome measurements are therefore needed to evaluate the roles of CHTs in physician handoff and improving the quality of patient care.
In the absence of larger studies evaluating major clinical outcomes, such as length of stay and mortality, hospitals considering innovations in the domain of computerized platforms for physician handoffs will need to consider the pros and cons of immediate system implementation on the basis of the evidence presented here versus waiting until there is more evidence from more definitive studies. In addition, our study suggests that organizations engage physicians during CHT design and develop a standardized CHT protocol that is interfaced with hospital IT systems and includes key components of handoff information, but provides flexibility to meet service‐specific needs. The evidence summarized here, while far from definitive for major outcomes, is nonetheless rather positive for the general benefits of CHTan impetus for careful design, implementation, and modification, whenever and wherever possible. Any such system implementations should, however, incorporate an evaluative component so that the evidence‐base surrounding CHT can be enhanced.
Acknowledgments
Disclosure: Nothing to report.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , , . Lost in translation: challenges and opportunities in physician‐to‐physician communication during patient handoffs. Acad Med. 2005;80(12):1094–1099.
- World Health Organization. Patient safety solution: communication during patient handovers. Available at: http://www.who.int/patientsafety/solutions/patientsafety/PS‐Solution3.pdf Accessed January 20, 2011.
- Accreditation Canada. Required Organizational Practices: Communication. Available at: http://wwwaccreditationca/uploadedFiles/information%20transferpdf?n=1212. Accessed January 20, 2010.
- Joint Commission on Accreditation of Healthcare Organizations National Patient Safety Goals. Available at: http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/06_npsg_cah.html. Accessed January 20, 2010.
- , , . Communicating in the “gray zone”: perceptions about emergency physician hospitalist handoffs and patient safety. Acad Emerg Med. 2007;14(10):884–894.
- . Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005;142(5):352–358.
- , , , . Transfers of patient care between house staff on internal medicine wards: a national survey. Arch Intern Med. 2006;166(11):1173–1177.
- , , , et al. Consequences of inadequate sign‐out for patient care. Arch Intern Med. 2008;168(16):1755–1760.
- , , , et al. What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff. Qual Saf Health Care. 2009;18(4):248–255.
- , , , et al. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Ann Emerg Med. 2009;53(6):701–710.
- , , , . Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med. 2007;167(19):2030–2036.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , . Utility of a standardized sign‐out card for new medical interns. J Gen Intern Med. 1996;11(12):753–755.
- , , , et al. Handoff strategies in settings with high consequences for failure: lessons for health care operations. Int J Qual Health Care. 2004;16(2):125–132.
- , , . Enhancing patient safety: improving the patient handoff process through appreciative inquiry. J Nurs Adm. 2007;37(2):95–104.
- , , , et al. Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign‐out. J Hosp Med. 2006;1(4):257–266.
- , , , . Organizing the transfer of patient care information: the development of a computerized resident sign‐out system. Surgery. 2004;136(1):5–13.
- , , , et al. Hospitalist handoffs: a systematic review and task force recommendations. J Hosp Med. 2009;4(7):433–440.
- , , , et al. A randomized, controlled trial evaluating the impact of a computerized rounding and sign‐out system on continuity of care and resident work hours. J Am Coll Surg. 2005;200(4):538–545.
- , , , et al. Using a computerized sign‐out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improve. 1998;24(2):77–87.
- , , , . Electronic medical handover: towards safer medical care. Med J Aust. 2005;183(7):369–372.
- , , , . Evaluation of a physician informatics tool to improve patient handoffs. J Am Med Inform Assoc. 2009;16(4):509–515.
- , , . Impact of electronic medical record integration of a handoff tool on sign‐out in a newborn intensive care unit. J Perinatol. 2011;31(5):311–317.
- , . Signing out patients for off‐hours coverage: comparison of manual and computer‐aided methods. Proceedings—The Annual Symposium on Computer Applications in Medical Care. 1992;114–118.
- , . MediSign: using a Web‐based SignOut system to improve provider identification. Proc AMIA Symp. 1999:550–554.
- , , , et al. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84(2–3):181–190.
- , . Signing on to sign out, part 2: describing the success of a Web‐based patient sign‐out application and how it will serve as a platform for an electronic discharge summary program. Healthc Q. 2007;10(1):120–124.
- , , , et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs. 2005;24(5):1103–1117.
- , , , et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev. 2009;Jan21(1):CD006093.
- , , . Improving physician communication through an automated, integrated sign‐out system. J Healthc Inf Manag. 2005;19(4):68–74.
- , , , et al. SynopSIS: integrating physician sign‐out with the electronic medical record. J Hosp Med. 2007;2(5):336–342.
- , , , . Improved physician work flow after integrating sign‐out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010;36(2):72–78.
- , , , et al. Electronic inpatient whiteboards: improving multidisciplinary communication and coordination of care. Int J Med Inform. 2009;78(4):239–247.
- , , , et al. Systematically improving physician assignment during in‐hospital transitions of care by enhancing a preexisting hospital electronic health record. J Hosp Med. 2009;4(5):308–312.
- . On error management: lessons from aviation. BMJ. 2000;320(7237):781–785.
- , , , . There is more to monitoring a nuclear power plant than meets the eye. Hum Factors 2000;42(1):36–55.
- , , . Handoffs in care—can we make them safer?Pediatr Clin North Am. 2006;53(6):1185–1195.
- , , , et al. The top 10 list for a safe and effective sign‐out. Arch Surg. 2008;143(10):1008–1010.
- , , . Systematic review of handoff mnemonics literature. Am J Med Qual. 2009;24(3):196–204.
Physician handoff is a common and essential component of daily patient care that includes transfer of important clinical patient information and accountability of patient care. Thus, high‐quality physician handoffs are crucial to ensure patient safety and continuity of patient care, especially with the new resident work hour restriction in North America.[1, 2] As such, healthcare organizations including the World Health Organization[3] have issued specific goals and organizational challenges to improve the effectiveness and coordination of communication among the care/service providers and with the recipients of care/service across the continuum in healthcare.[4, 5]
It has been well‐documented that physician handoffs in hospital settings are often unstructured and not standardized, which leads to medical errors and jeopardizes patient safety.[2, 6, 7, 8, 9, 10, 11, 12] This lack of standardization of physician handoff for hospitalized patients occurs in every major in‐hospital service and affects trainees and staff.[2, 6, 7, 9, 10, 12, 13] It has been demonstrated in healthcare and in other domains that a standardized handoff protocol that involves both verbal communication and written handoff documents is likely to be an effective method of handoff to decrease miscommunication and associated errors.[14, 15, 16, 17] Computerized physician handoff tools (CHTs) have been increasingly deployed to address these challenges and have quickly gained popularity among physicians for documenting patient information during physician handoff for hospitalized patients.[18] CHTs can be an complementary part of electronic medical record (EMR) systems, but not a substitute since their focus is to deliver concise and essential information vital for patient care during interfaces of patient care.
Two recent systematic reviews have examined information technology (IT) systems to promote the handoff process in healthcare.[17, 19] However, to our knowledge, there has not been a systematic review of the potential role of CHT in physician handoff and quality of patient care for hospitalized patients. We therefore conducted a systematic review to examine the current evidence for CHTs in physician handoff for hospitalized patients, focusing specifically on potential effects on continuity of patient care, physician work efficiency, quality of handoffs, and patient outcomes.
METHODS
Criteria for Considering Eligible Studies
We included randomized controlled trials, controlled clinical trial, quasi‐experimental studies, and controlled beforeafter studies that evaluated CHTs during physician handoff of hospitalized patients. Studies needed to report patient outcomes (adverse events, missing patients at rounds, or in‐hospital mortality), physician work efficiency, quality of handoff (accuracy, consistency, or completeness), continuity of care, or physician satisfaction. Articles that met all these inclusion criteria were considered to be eligible for the review. We excluded review articles, commentaries, case reports, and retrospective studies.
Search Strategy
CHTs were defined as computer‐based platforms, designed specifically for the purpose of physician handoff, to allow distributed access and synchronous archiving of patient information via Internet protocols (ie, electronic tool to allow physician data access and data entry for handoff from different computers at multiple locations within the authorized hospitals or clinics). A search strategy was developed based on a MEDLINE search format combined with our inclusion criteria and with this definition of CHTs. We used search terms related to physician communication and information technology, and relevant Medical Subjects Headings, which include handover, handoff, signoff, sign‐over, off‐duty, post‐call, computerized, Web‐based, communication tool. The databases, including MEDLINE, PUBMED, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane database for systematic reviews, and the Cochrane CENTRAL register of controlled trials, were initially searched from 1985 to December 2011 in all languages. The Cochrane Collaboration filter for controlled interventional studies was used to select the above‐mentioned interventional trial designs. In addition, the first 2 authors hand searched the references of included articles and relevant systematic reviews.
Screening for Eligible Studies
All articles identified in the database searches described above were included for screening in 2 stages. First, 2 reviewers (P.L., S.A.) independently reviewed the title and abstracts of the identified articles for eligibility. The articles selected in the first stage of screening were then further assessed by a full‐text review independently by the 2 reviewers. Any discrepancy was resolved by consensus or by involvement of a third reviewer (C.T.).
Data Abstraction and Analysis
Data abstraction from selected studies was conducted independently by 3 authors based on a predefined template. All discrepancies in this stage were resolved by consensus among the 3 authors. For each study, we analyzed study design, data collection, intervention, main outcomes, and components of physician handoffs in the study. Due to heterogeneity of study outcomes, measures used, and results, a meta‐analysis was not performed. Study outcomes, which included adverse events, missing patients at rounds, time spent on rounding patient, accuracy, consistency or completeness of handoff information, and continuity of care, were summarized.
RESULTS
Study Selection
A total of 1026 citations were identified in the initial search, of which 1006 studies did not evaluate CHT and were excluded by title and abstract screening. Of the 20 studies evaluated further by full‐text review, 5 were selected for the final analysis. One additional study was identified by hand searching references. The kappa score of inter‐reviewer agreement on article selection in the first stage of screening was 0.7, and for the second stage of article selection, kappa was 1.0. The reasons for exclusion in the second selection step are presented in Figure 1.
Study Characteristics
Of the 6 studies identified, 1 study was a randomized controlled trial[20] and the other 5 were controlled beforeafter studies.[21, 22, 23, 24, 25] All studies were conducted in teaching hospitals in English‐speaking high‐income countries. All were single‐center studies, except the study by Van Eaton et[20] that involved 2 centers. All the studies investigated physician handoffs conducted by trainees. Two studies included staff physicians.[22, 24] Van Eaton et al's study included general medical, general surgical, and subspecialty surgical services.[20] The other 5 studies assessed physician handoffs in family medicine,[25] internal medicine services,[21, 23] a surgical service,[22] and a neonatal intensive care unit.[24] The study by Van Eaton et al[20] enrolled the largest study population. The intervention or observation phase ranged from 1 month[20] to 6 months[24] (Table 1).
| Study | Design | Setting | Target Services | Intervention Group | Control Group | Data Collection and Validation |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Beforeafter study | 150‐bed urban hospital in USA | Family Medicine Residents (N = 7) | Patient no. not reported 1 mo of intervention No CHT training prior to the intervention reported | Patient no. not reported Traditional handoff note (on index card or previous list) Components of handoff note not reported | Questionnaire No data validation |
| Peterson et al[21] (1998) | Beforeafter study | 720‐bed tertiary care hospital in USA | All Internal Medicine Services Residents (N = 99) | 3747 patients 4 mo of intervention 8 wk of run‐in period | 1874 patients Handwritten handoff Components of handoff note not reported | Self‐report using e‐mail, report card, in person chart review for unreported adverse events in 250 samples |
| Van Eaton et al[20] (2005) | Randomized cross‐over trial | 450‐bed tertiary care hospital and a 368‐bed trauma center in USA | General Medicine, General Surgery, and Subspecialties Trauma Residents (N = 7 teams) | 8018 patients 14 wk of randomized crossover period 6 wk of run‐in period | 7569 patients Individual written lists, cards, a team‐developed computer‐generated spreadsheet Components of handoff note not reported | Telephone interview and anonymous online survey No validation of data |
| Cheah et al[22] (2005) | Beforesfter atudy | A 400‐bed regional teaching hospital in Australia | General Surgery Registrars and Residents (N = 714) | Patient no. not reported 3 mo of observation period (for weekend coverage only) No CHT training prior to the intervention reported | Patient no. not reported No description of pre‐intervention handoff method reported | In‐person interview and survey No validation of data |
| Flanagan et al[23] (2009) | Beforesfter atudy | Tertiary care hospital in USA | Internal Medicine, Medical Intensive Care Unit First‐year Residents (N = 35) | 1264 patient handoff forms 1 mo of observation Orientation session and 1 cross‐over shift of run‐in period | Patient no. not reported No description of pre‐CHT implementation handoff method reported | In‐person interview and survey No validation of data |
| Palma et al[24] (2011) | Beforeafter study | 304‐bed quaternary care women and children hospital in USA | NICU Attendings, Residents, Nursing staffs (N = 4652) | Patient no. not reported 6 mo of intervention of NICU handoff tool Instruction document by e‐mail and informal instructional session | Patient no. not reported A Microsoft‐based standalone handoff tool or EMR integrated Medical/Surgical handoff tool Components of handoff note not reported | Online survey No validation of data |
CHT Characteristics
Three CHTs were standalone applications designed specifically for physician handoffs.[20, 22, 25] The other 3 CHTs were add‐on functions to existing hospital Electric Medical Record (EMR) systems.[21, 23, 24] All CHTs except one[25] interfaced with existing EMR systems, allowing for variable degrees of data transfer depending on CHT design and the functionalities of the EMR systems. CHT users were actively involved in designing and modifying the CHTs in most of the studies.[20, 21, 23, 25] The characteristics of the CHTs were summarized in Table 2.
| Study | CHT Design | EMR Interface | Physician Daily Progress Note | Participants' Role in CHT Design | Components of CHT | Components That Require Manual Input |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Standalone application | No interface | Paper‐based | Designing | Patient demographics Medications Diagnosis Problem lists Comment line | All the information |
| Peterson et al[21] (1998) | A part of existing EMR | Bi‐directional interface | Paper‐based | Designing | Patient demographics Current medication Allergy Code status Recent lab value A problem list A to do list | A problem list A to do list |
| Van Eaton et al[20] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Designing and modifying | Patient demographics Diagnosis Medication Allergy Vital signs Lab and investigation A problem list A to do list | Diagnosis Medication A problem list A to do list |
| Cheah et al[22] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | No | Patient demographics Diagnosis Length of stay Recent investigations Free‐text note (Not standardized) | Free‐text note |
| Flanagan et al[23] (2009) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Evaluating and modifying | Patient demographics Medication Allergy Lab and investigation Physician daily note Free‐text note (not standardized) | Free‐text note (may contain assessment, a problem list, venous access, short‐term concerns and long‐term plan, and follow‐up tasks) |
| Palma et al[24] (2011) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Paper‐based | No | Patient demographics Lab and measurement Free‐text note (not standardized) | Free‐text note (including patient description, active medical issues, ongoing care and a to do list) |
CHT's Impact on Adverse Events
The impact of CHTs on preventable adverse events was evaluated in a single study by Peterson et al.[21] The authors defined an adverse event as an injury due to medical treatment which prolonged hospital stay or produced disability at discharge in the study. Preventability was determined by using a 6‐point scale and assessed independently by 3 reviewers. Fewer adverse events were found after implementation of CHTs (2.38% vs 3.94%, P < 0.001). They also reported nonsignificant reductions in preventable adverse events (1.23% vs 1.72%, P < 0.1) with implementation of the CHT, and preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10). The odds ratio for a patient experiencing a preventable adverse event during cross‐coverage compared to noncross‐coverage time was reduced from 5.2 (95% confidence interval [CI], 1.518.2) to 1.5 (95% CI, 0.29.0) following implementation of the CHT (Table 3).
| Study | Outcomes of Interest | Results | Implication for CHT Design and Use |
|---|---|---|---|
| |||
| Ram and Block[25] (1992) | Physician satisfaction Importance and accessibility of clinical information | Improved physician satisfaction Handoff documentation more legible, more consistent, and more comprehensive Information required to be typed in by residents and not up‐to‐date | The most important data for handoff: a to do list and code status A CHT interfaced with hospital IT system, and in a format that can focus on physician needs |
| Petersen et al[21] (1998) | Adverse event rate Preventable adverse events rate | Fewer adverse events (2.38% vs 3.94%, P < 0.001) Fewer preventable adverse events (1.23% vs 1.72%, P < 0.1) Few preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10) Lower OR of preventable adverse events during cross‐coverage (1.5; 95% CI 0.29.0 vs 5.2; 95% CI 1.58.2) | Active involvement in the design of CHT by house staff likely contributes to high participation and CHT use rate in the study |
| Van Eaton et al[20] (2005) | No. of patients missed on rounds Perception on continuity of care quality and workflow efficiency Daily self‐reported pre‐rounding and rounding times and tasks | Reduced the no. of patients missed on rounds (2.5 patients/team/mo) (P = 0.0001) Spent 40% more time with patients at pre‐rounds Reduced time on team rounds by 1.5 min per patient Reduced time on manual copying at pre‐rounding by 50% Improved handoff quality Improved continuity of care No reduction of overall pre‐rounding time | The largest benefit from CHTs varies between clinical services, from more time assessing patients before rounds in Internal Medicine to reduced backtracking and locating patients in Surgery |
| Cheah et al[22] (2005) | Completeness and usefulness of handoff information Desirability of electronic handoff system | Identified information set for handoff Free text entry in CHT often deficient in particular patient information Concerns of the completeness and consistency of information delivered in CHT | CHT needs to be linked to hospital information system |
| Flanagan et al[23] (2009) | Common data elements of interest extracted during physician handoff Missing data required during handoff Physicians' perception of CHT | Additional important information needed that not included during handoff in 25% cases | Code status, relevant lab data, short‐term concerns, a problem list, and a if‐then list should be included in CHT template A standard form reduces variability of handoff information |
| Palma et al[24] (2011) | Accuracy of handoff information Healthcare provider satisfaction | Improved perceived accuracy of handoff information (91% vs 78%, P <0.01) Improved satisfaction with handoff process (71% vs 35%, P < 0.01) Improved satisfaction with handoff documents (98% vs 91%, P <0.01) More time spent on updating handoff information (1620 min vs 1115 min, P = 0.03) | A discipline‐specific handoff tool results in perceived handoff accuracy and satisfaction A more efficient handoff tool can be achieved by more extensive data transfer from hospital IT system |
CHT's Impact on Physician Work Efficiency
Van Eaton et al's study examined the effect of CHTs on physician work efficiency.[20] Improved physician work efficiency was found following implementation of CHT. Self‐reported time spent on hand‐copying patient information was reduced by 50%, while the portion of time spent on seeing patients during pre‐rounding increased. Similarly, self‐reported time spent on each patient during rounding (routine patient assessment by the primary team) was decreased by 1.5 minutes. Overall, resident physicians subjectively reported an average time saving of 45 minutes daily for junior residents and 30 minutes for senior residents, and 81% of residents reported finishing their work sooner when using CHTs. Although no data were reported in the pre‐CHT period described in the study by Cheah et al, they indicated that work efficiency was felt to be improved because all physicians could locate their patients quickly and were pleased to be able to check patients' lab results in the CHT.[22] Conversely, Palma et al and Ram and Block reported perceived increased work load with CHTs by users due to time spent updating handoff information.[24, 25]
CHT's Impact on Quality of Physician Handoff
Overall quality of physician handoff and completeness of the handoff document was improved in 3 studies.[20, 24, 25] Flanagan et al reported that patient identifiers and medications were extracted most of the time.[23] However, there were concerns regarding consistency,[22] completeness[22, 23] of information provided during physician handoff using CHTs. Palma et al's and Ram and Block's studies[24, 25] commented on the accuracy of patient information communicated during physician handoff. While Ram and Block's study suggested that it may be poorer during the intervention period,[25] Palma et al's study found improved perceived accuracy of handoff information postimplementation of a CHT (98% vs 91%, P < 0.01).[24]
CHT's Impact on Continuity of Patient Care
Using CHTs was associated with a decreased number of patients missed on rounds after handoff (new admitted patients who were not assessed by the primary team in the morning rounds because cross‐covering physicians did not inform the primary team) in Van Eaton et al's study.[20] On the other hand, Cheah et al[22] reported that documented handoffs after physicians returned to duty occurred on 50% of patients who had experienced important clinical events on weekends.
DISCUSSION
Our systematic review identified 6 controlled studies of CHT. Outcome parameters reported in these studies included quality of the handoff (including completeness, accuracy, and consistency), physician time management, continuity of care, adverse events, and missed patients. Our results suggest that while CHT are a promising tool, further evaluation using rigorous study methodologies is needed. These findings are somewhat surprising given increasing popularity of CHTs in daily patient care.[19, 24, 26, 27, 28] This might be due to the fact that IT adoption and use in healthcare is still in a phase of relative infancy,[29] and that the success of adopting IT systems in healthcare depends on various factors.[30]
Roles of CHT in Physician Handoff for Hospitalized Patients
Our study indicates that CHT can potentially improve continuity of patient care by reducing the number of missing patients during rounds following handoff,[20] and similarly improve patient safety by decreasing adverse events and preventable adverse events.[21] Of note, users reported that they were able to spend more time with patients during pre‐rounding[20] which will likely enhance quality and continuity of patient care. However, it is unclear whether these improvements translate into better patient outcomes. Although Peterson et al attempted to minimize the risk of bias by using anonymous reporting and blinding participants to the timing of data collection,[21] adverse events during the intervention period could have been underestimated due to surveillance bias or decreased self‐reporting. Nevertheless, the results suggest that CHTs may have affected quality of patient care in a positive manner from included studies.
The findings from our review also point to a positive impact of CHT on physician work efficiency. Specifically, residents spent less time rounding on patients after handoff and finished their work sooner after introduction of the intervention.[20] Several other published studies on CHT also indicated potential benefits on work efficiency and/or patient safety,[31, 33, 34, 35] although they did not meet the inclusion criteria for our study (prespecified outcomes not reported,[31, 35] or study design[33, 34, 35]). In the studies in which the majority of handoff information was manually typed in the CHT, the work load was perceived to be increased with CHT implementation.[24, 25] On the other hand, the study conducted by Van Eaton et al demonstrated that a CHT that had broad integration with the hospital main IT system, and could automatically transfer important patient information such as medication, medical problems, recent investigation, and vital signs into CHT, quickly gained popularity among residents and staff due to its user‐friendly features.[20] This integration can also potentially reduce miscommunication and associated medical errors during physician handoff. Palma et al's study reported higher perceived workload due to manual entry of patient data.[24] Although the CHT used in their study was developed within their existing EMR system, large amounts of information needed to be manually imputed, and thus increased time spent on updating handoff information. This information included patient demographics, active medical issues, a to do list, and on‐going issues,[24] some of which could be imputed automatically with better CHT design. It is also possible that users spent more time in updating the handoff because they were able to deliver more information using a CHT.[24] However, this may allow cross‐covering physicians to spend less time on looking for patient information from other sources and thus actually decrease workload during cross‐coverage. Although there are numerous factors that could affect physician work efficiency when using a new IT system,[30] it was felt that a well‐designed and easy‐to‐use CHT that is integrated with the hospital information system can improve physician productivity.
The role of CHT in improving quality of handoff is less clear. Three studies[20, 24, 25] found an overall improvement in the quality of handoff after implementation of CHT, such that the handoff information was more complete and more consistent. On the other hand, physicians were concerned about the comprehensiveness of physician handoff after implementation of CHT in 2 studies.[22, 23] In Ram and Block's study,[25] physicians relied heavily on an unstructured free‐text entry system to deliver the majority of patient information that physicians thought to be important. In Flanagan et al's study,[23] resident physicians had to search for alternative sources, such as patient charts and electronic order systems, to obtain vital information in many cases in spite of a structured CHT. As a result, the information available was often not sufficient to help on‐call physicians make patient care decisions.[23]
Implication of CHT Design and Use
It has been demonstrated in many non‐healthcare domains,[15, 36, 37] as well as nursing care,[38] that a standardized handoff protocol is vital to decrease medical errors and improve patient safety. In our review, we found that physicians generally reported being satisfied with the accuracy of handoff information and the overall handoff when using standardized CHTs interfaced with hospital IT systems. This suggests, as recommended by Flanagan et al,[23] Palma et al,[24] and Ram and Block[25] that CHTs be developed with a standardized protocol and wide integration into hospital IT systems.
In order to achieve this goal, key patient information necessary for patient care need to be communicated during physician handoff. As hospitals consist of a wide range of disciplines and specialties with varying cultures and focuses of patient information, it is likely difficult to develop a single panacea CHT template for all the in‐hospital services.[1] This may be even particularly relevant when developing CHTs for different hospital services. However, some patient information appears to be universally important for physician handoff for inpatient care. Key elements, such as patient demographics, diagnosis, outstanding investigation results, code status, a problem list, and a to do list, were noted to be consistently present in the CHTs that were evaluated in our review (Table 2). Other studies have also demonstrated that information items such as a to do list, outstanding investigation results, and patients' code status were regarded as the most important information during physician handoff.[1, 2, 17, 23, 39, 40] Based on these findings, a potential solution for CHT standardization would be to develop a core CHT which includes the universally important components of physician handoff identified in this review, and provides options for adding well‐categorized service‐specific information as needed (eg, type and date of surgical procedures for surgical patients). It also appears that active involvement of physicians in CHT design and modification facilitates successful implementation of CHT, as demonstrated in Van Eaton et al's and Peterson et al's studies.[20, 21]
It is difficult to recommend metrics for CHT evaluation based on the limited literature identified in our review. However, it appears to be reasonable to consider integration into existing IT system, user friendly features, impact on quality of handoff documents, work efficiency, and processes and outcomes of patient care when assessing CHTs.
Limitations
There are several limitations in the studies included in our review. None of the studies were multi‐centered. The majority of the included studies had a beforeafter design.[21, 22, 23, 24, 25] Some studies did not have user training or a run in period to ensure familiarity of CHTs by users.[22, 24, 25] None of the studies described the key components of handoff in the control groups, or used quality control measurements for user familiarity with the CHTs. Furthermore, outcomes reported by the studies were heterogeneous, subjective, based on participant self‐report, and not independently validated.
Our review also has also several limitations. First, in spite of a comprehensive search effort, it is possible that we failed to identify all relevant articles. However, this is unlikely, given that we searched multiple databases and performed hand searches of all references identified from the included articles, as well as content‐related previously published systematic reviews. Second, we were not able to perform a meta‐analysis, given the heterogeneity seen in outcomes assessed across studies, measures applied, and results presented.
CONCLUSIONS AND IMPLICATIONS FOR PRACTICE
Although the current literature suggests that implementation of CHTs is likely to improve physician work efficiency, satisfaction, and quality of patient care during physician handoff for hospitalized patients, the evidence supporting these potential benefits is limited. Furthermore, it is unknown what impacts CHTs may have on clinical outcomes, such as hospital length of stay and mortality. Further studies with larger sample size, multiple center involvement, and more objective patient outcome measurements are therefore needed to evaluate the roles of CHTs in physician handoff and improving the quality of patient care.
In the absence of larger studies evaluating major clinical outcomes, such as length of stay and mortality, hospitals considering innovations in the domain of computerized platforms for physician handoffs will need to consider the pros and cons of immediate system implementation on the basis of the evidence presented here versus waiting until there is more evidence from more definitive studies. In addition, our study suggests that organizations engage physicians during CHT design and develop a standardized CHT protocol that is interfaced with hospital IT systems and includes key components of handoff information, but provides flexibility to meet service‐specific needs. The evidence summarized here, while far from definitive for major outcomes, is nonetheless rather positive for the general benefits of CHTan impetus for careful design, implementation, and modification, whenever and wherever possible. Any such system implementations should, however, incorporate an evaluative component so that the evidence‐base surrounding CHT can be enhanced.
Acknowledgments
Disclosure: Nothing to report.
Physician handoff is a common and essential component of daily patient care that includes transfer of important clinical patient information and accountability of patient care. Thus, high‐quality physician handoffs are crucial to ensure patient safety and continuity of patient care, especially with the new resident work hour restriction in North America.[1, 2] As such, healthcare organizations including the World Health Organization[3] have issued specific goals and organizational challenges to improve the effectiveness and coordination of communication among the care/service providers and with the recipients of care/service across the continuum in healthcare.[4, 5]
It has been well‐documented that physician handoffs in hospital settings are often unstructured and not standardized, which leads to medical errors and jeopardizes patient safety.[2, 6, 7, 8, 9, 10, 11, 12] This lack of standardization of physician handoff for hospitalized patients occurs in every major in‐hospital service and affects trainees and staff.[2, 6, 7, 9, 10, 12, 13] It has been demonstrated in healthcare and in other domains that a standardized handoff protocol that involves both verbal communication and written handoff documents is likely to be an effective method of handoff to decrease miscommunication and associated errors.[14, 15, 16, 17] Computerized physician handoff tools (CHTs) have been increasingly deployed to address these challenges and have quickly gained popularity among physicians for documenting patient information during physician handoff for hospitalized patients.[18] CHTs can be an complementary part of electronic medical record (EMR) systems, but not a substitute since their focus is to deliver concise and essential information vital for patient care during interfaces of patient care.
Two recent systematic reviews have examined information technology (IT) systems to promote the handoff process in healthcare.[17, 19] However, to our knowledge, there has not been a systematic review of the potential role of CHT in physician handoff and quality of patient care for hospitalized patients. We therefore conducted a systematic review to examine the current evidence for CHTs in physician handoff for hospitalized patients, focusing specifically on potential effects on continuity of patient care, physician work efficiency, quality of handoffs, and patient outcomes.
METHODS
Criteria for Considering Eligible Studies
We included randomized controlled trials, controlled clinical trial, quasi‐experimental studies, and controlled beforeafter studies that evaluated CHTs during physician handoff of hospitalized patients. Studies needed to report patient outcomes (adverse events, missing patients at rounds, or in‐hospital mortality), physician work efficiency, quality of handoff (accuracy, consistency, or completeness), continuity of care, or physician satisfaction. Articles that met all these inclusion criteria were considered to be eligible for the review. We excluded review articles, commentaries, case reports, and retrospective studies.
Search Strategy
CHTs were defined as computer‐based platforms, designed specifically for the purpose of physician handoff, to allow distributed access and synchronous archiving of patient information via Internet protocols (ie, electronic tool to allow physician data access and data entry for handoff from different computers at multiple locations within the authorized hospitals or clinics). A search strategy was developed based on a MEDLINE search format combined with our inclusion criteria and with this definition of CHTs. We used search terms related to physician communication and information technology, and relevant Medical Subjects Headings, which include handover, handoff, signoff, sign‐over, off‐duty, post‐call, computerized, Web‐based, communication tool. The databases, including MEDLINE, PUBMED, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane database for systematic reviews, and the Cochrane CENTRAL register of controlled trials, were initially searched from 1985 to December 2011 in all languages. The Cochrane Collaboration filter for controlled interventional studies was used to select the above‐mentioned interventional trial designs. In addition, the first 2 authors hand searched the references of included articles and relevant systematic reviews.
Screening for Eligible Studies
All articles identified in the database searches described above were included for screening in 2 stages. First, 2 reviewers (P.L., S.A.) independently reviewed the title and abstracts of the identified articles for eligibility. The articles selected in the first stage of screening were then further assessed by a full‐text review independently by the 2 reviewers. Any discrepancy was resolved by consensus or by involvement of a third reviewer (C.T.).
Data Abstraction and Analysis
Data abstraction from selected studies was conducted independently by 3 authors based on a predefined template. All discrepancies in this stage were resolved by consensus among the 3 authors. For each study, we analyzed study design, data collection, intervention, main outcomes, and components of physician handoffs in the study. Due to heterogeneity of study outcomes, measures used, and results, a meta‐analysis was not performed. Study outcomes, which included adverse events, missing patients at rounds, time spent on rounding patient, accuracy, consistency or completeness of handoff information, and continuity of care, were summarized.
RESULTS
Study Selection
A total of 1026 citations were identified in the initial search, of which 1006 studies did not evaluate CHT and were excluded by title and abstract screening. Of the 20 studies evaluated further by full‐text review, 5 were selected for the final analysis. One additional study was identified by hand searching references. The kappa score of inter‐reviewer agreement on article selection in the first stage of screening was 0.7, and for the second stage of article selection, kappa was 1.0. The reasons for exclusion in the second selection step are presented in Figure 1.
Study Characteristics
Of the 6 studies identified, 1 study was a randomized controlled trial[20] and the other 5 were controlled beforeafter studies.[21, 22, 23, 24, 25] All studies were conducted in teaching hospitals in English‐speaking high‐income countries. All were single‐center studies, except the study by Van Eaton et[20] that involved 2 centers. All the studies investigated physician handoffs conducted by trainees. Two studies included staff physicians.[22, 24] Van Eaton et al's study included general medical, general surgical, and subspecialty surgical services.[20] The other 5 studies assessed physician handoffs in family medicine,[25] internal medicine services,[21, 23] a surgical service,[22] and a neonatal intensive care unit.[24] The study by Van Eaton et al[20] enrolled the largest study population. The intervention or observation phase ranged from 1 month[20] to 6 months[24] (Table 1).
| Study | Design | Setting | Target Services | Intervention Group | Control Group | Data Collection and Validation |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Beforeafter study | 150‐bed urban hospital in USA | Family Medicine Residents (N = 7) | Patient no. not reported 1 mo of intervention No CHT training prior to the intervention reported | Patient no. not reported Traditional handoff note (on index card or previous list) Components of handoff note not reported | Questionnaire No data validation |
| Peterson et al[21] (1998) | Beforeafter study | 720‐bed tertiary care hospital in USA | All Internal Medicine Services Residents (N = 99) | 3747 patients 4 mo of intervention 8 wk of run‐in period | 1874 patients Handwritten handoff Components of handoff note not reported | Self‐report using e‐mail, report card, in person chart review for unreported adverse events in 250 samples |
| Van Eaton et al[20] (2005) | Randomized cross‐over trial | 450‐bed tertiary care hospital and a 368‐bed trauma center in USA | General Medicine, General Surgery, and Subspecialties Trauma Residents (N = 7 teams) | 8018 patients 14 wk of randomized crossover period 6 wk of run‐in period | 7569 patients Individual written lists, cards, a team‐developed computer‐generated spreadsheet Components of handoff note not reported | Telephone interview and anonymous online survey No validation of data |
| Cheah et al[22] (2005) | Beforesfter atudy | A 400‐bed regional teaching hospital in Australia | General Surgery Registrars and Residents (N = 714) | Patient no. not reported 3 mo of observation period (for weekend coverage only) No CHT training prior to the intervention reported | Patient no. not reported No description of pre‐intervention handoff method reported | In‐person interview and survey No validation of data |
| Flanagan et al[23] (2009) | Beforesfter atudy | Tertiary care hospital in USA | Internal Medicine, Medical Intensive Care Unit First‐year Residents (N = 35) | 1264 patient handoff forms 1 mo of observation Orientation session and 1 cross‐over shift of run‐in period | Patient no. not reported No description of pre‐CHT implementation handoff method reported | In‐person interview and survey No validation of data |
| Palma et al[24] (2011) | Beforeafter study | 304‐bed quaternary care women and children hospital in USA | NICU Attendings, Residents, Nursing staffs (N = 4652) | Patient no. not reported 6 mo of intervention of NICU handoff tool Instruction document by e‐mail and informal instructional session | Patient no. not reported A Microsoft‐based standalone handoff tool or EMR integrated Medical/Surgical handoff tool Components of handoff note not reported | Online survey No validation of data |
CHT Characteristics
Three CHTs were standalone applications designed specifically for physician handoffs.[20, 22, 25] The other 3 CHTs were add‐on functions to existing hospital Electric Medical Record (EMR) systems.[21, 23, 24] All CHTs except one[25] interfaced with existing EMR systems, allowing for variable degrees of data transfer depending on CHT design and the functionalities of the EMR systems. CHT users were actively involved in designing and modifying the CHTs in most of the studies.[20, 21, 23, 25] The characteristics of the CHTs were summarized in Table 2.
| Study | CHT Design | EMR Interface | Physician Daily Progress Note | Participants' Role in CHT Design | Components of CHT | Components That Require Manual Input |
|---|---|---|---|---|---|---|
| ||||||
| Ram and Block[25] (1992) | Standalone application | No interface | Paper‐based | Designing | Patient demographics Medications Diagnosis Problem lists Comment line | All the information |
| Peterson et al[21] (1998) | A part of existing EMR | Bi‐directional interface | Paper‐based | Designing | Patient demographics Current medication Allergy Code status Recent lab value A problem list A to do list | A problem list A to do list |
| Van Eaton et al[20] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Designing and modifying | Patient demographics Diagnosis Medication Allergy Vital signs Lab and investigation A problem list A to do list | Diagnosis Medication A problem list A to do list |
| Cheah et al[22] (2005) | Standalone application | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | No | Patient demographics Diagnosis Length of stay Recent investigations Free‐text note (Not standardized) | Free‐text note |
| Flanagan et al[23] (2009) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Electronic‐based | Evaluating and modifying | Patient demographics Medication Allergy Lab and investigation Physician daily note Free‐text note (not standardized) | Free‐text note (may contain assessment, a problem list, venous access, short‐term concerns and long‐term plan, and follow‐up tasks) |
| Palma et al[24] (2011) | A part of existing EMR | Uni‐directional interface (data input from hospital IT system) | Paper‐based | No | Patient demographics Lab and measurement Free‐text note (not standardized) | Free‐text note (including patient description, active medical issues, ongoing care and a to do list) |
CHT's Impact on Adverse Events
The impact of CHTs on preventable adverse events was evaluated in a single study by Peterson et al.[21] The authors defined an adverse event as an injury due to medical treatment which prolonged hospital stay or produced disability at discharge in the study. Preventability was determined by using a 6‐point scale and assessed independently by 3 reviewers. Fewer adverse events were found after implementation of CHTs (2.38% vs 3.94%, P < 0.001). They also reported nonsignificant reductions in preventable adverse events (1.23% vs 1.72%, P < 0.1) with implementation of the CHT, and preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10). The odds ratio for a patient experiencing a preventable adverse event during cross‐coverage compared to noncross‐coverage time was reduced from 5.2 (95% confidence interval [CI], 1.518.2) to 1.5 (95% CI, 0.29.0) following implementation of the CHT (Table 3).
| Study | Outcomes of Interest | Results | Implication for CHT Design and Use |
|---|---|---|---|
| |||
| Ram and Block[25] (1992) | Physician satisfaction Importance and accessibility of clinical information | Improved physician satisfaction Handoff documentation more legible, more consistent, and more comprehensive Information required to be typed in by residents and not up‐to‐date | The most important data for handoff: a to do list and code status A CHT interfaced with hospital IT system, and in a format that can focus on physician needs |
| Petersen et al[21] (1998) | Adverse event rate Preventable adverse events rate | Fewer adverse events (2.38% vs 3.94%, P < 0.001) Fewer preventable adverse events (1.23% vs 1.72%, P < 0.1) Few preventable adverse events during cross‐coverage (0.24% vs 0.38%, P > 0.10) Lower OR of preventable adverse events during cross‐coverage (1.5; 95% CI 0.29.0 vs 5.2; 95% CI 1.58.2) | Active involvement in the design of CHT by house staff likely contributes to high participation and CHT use rate in the study |
| Van Eaton et al[20] (2005) | No. of patients missed on rounds Perception on continuity of care quality and workflow efficiency Daily self‐reported pre‐rounding and rounding times and tasks | Reduced the no. of patients missed on rounds (2.5 patients/team/mo) (P = 0.0001) Spent 40% more time with patients at pre‐rounds Reduced time on team rounds by 1.5 min per patient Reduced time on manual copying at pre‐rounding by 50% Improved handoff quality Improved continuity of care No reduction of overall pre‐rounding time | The largest benefit from CHTs varies between clinical services, from more time assessing patients before rounds in Internal Medicine to reduced backtracking and locating patients in Surgery |
| Cheah et al[22] (2005) | Completeness and usefulness of handoff information Desirability of electronic handoff system | Identified information set for handoff Free text entry in CHT often deficient in particular patient information Concerns of the completeness and consistency of information delivered in CHT | CHT needs to be linked to hospital information system |
| Flanagan et al[23] (2009) | Common data elements of interest extracted during physician handoff Missing data required during handoff Physicians' perception of CHT | Additional important information needed that not included during handoff in 25% cases | Code status, relevant lab data, short‐term concerns, a problem list, and a if‐then list should be included in CHT template A standard form reduces variability of handoff information |
| Palma et al[24] (2011) | Accuracy of handoff information Healthcare provider satisfaction | Improved perceived accuracy of handoff information (91% vs 78%, P <0.01) Improved satisfaction with handoff process (71% vs 35%, P < 0.01) Improved satisfaction with handoff documents (98% vs 91%, P <0.01) More time spent on updating handoff information (1620 min vs 1115 min, P = 0.03) | A discipline‐specific handoff tool results in perceived handoff accuracy and satisfaction A more efficient handoff tool can be achieved by more extensive data transfer from hospital IT system |
CHT's Impact on Physician Work Efficiency
Van Eaton et al's study examined the effect of CHTs on physician work efficiency.[20] Improved physician work efficiency was found following implementation of CHT. Self‐reported time spent on hand‐copying patient information was reduced by 50%, while the portion of time spent on seeing patients during pre‐rounding increased. Similarly, self‐reported time spent on each patient during rounding (routine patient assessment by the primary team) was decreased by 1.5 minutes. Overall, resident physicians subjectively reported an average time saving of 45 minutes daily for junior residents and 30 minutes for senior residents, and 81% of residents reported finishing their work sooner when using CHTs. Although no data were reported in the pre‐CHT period described in the study by Cheah et al, they indicated that work efficiency was felt to be improved because all physicians could locate their patients quickly and were pleased to be able to check patients' lab results in the CHT.[22] Conversely, Palma et al and Ram and Block reported perceived increased work load with CHTs by users due to time spent updating handoff information.[24, 25]
CHT's Impact on Quality of Physician Handoff
Overall quality of physician handoff and completeness of the handoff document was improved in 3 studies.[20, 24, 25] Flanagan et al reported that patient identifiers and medications were extracted most of the time.[23] However, there were concerns regarding consistency,[22] completeness[22, 23] of information provided during physician handoff using CHTs. Palma et al's and Ram and Block's studies[24, 25] commented on the accuracy of patient information communicated during physician handoff. While Ram and Block's study suggested that it may be poorer during the intervention period,[25] Palma et al's study found improved perceived accuracy of handoff information postimplementation of a CHT (98% vs 91%, P < 0.01).[24]
CHT's Impact on Continuity of Patient Care
Using CHTs was associated with a decreased number of patients missed on rounds after handoff (new admitted patients who were not assessed by the primary team in the morning rounds because cross‐covering physicians did not inform the primary team) in Van Eaton et al's study.[20] On the other hand, Cheah et al[22] reported that documented handoffs after physicians returned to duty occurred on 50% of patients who had experienced important clinical events on weekends.
DISCUSSION
Our systematic review identified 6 controlled studies of CHT. Outcome parameters reported in these studies included quality of the handoff (including completeness, accuracy, and consistency), physician time management, continuity of care, adverse events, and missed patients. Our results suggest that while CHT are a promising tool, further evaluation using rigorous study methodologies is needed. These findings are somewhat surprising given increasing popularity of CHTs in daily patient care.[19, 24, 26, 27, 28] This might be due to the fact that IT adoption and use in healthcare is still in a phase of relative infancy,[29] and that the success of adopting IT systems in healthcare depends on various factors.[30]
Roles of CHT in Physician Handoff for Hospitalized Patients
Our study indicates that CHT can potentially improve continuity of patient care by reducing the number of missing patients during rounds following handoff,[20] and similarly improve patient safety by decreasing adverse events and preventable adverse events.[21] Of note, users reported that they were able to spend more time with patients during pre‐rounding[20] which will likely enhance quality and continuity of patient care. However, it is unclear whether these improvements translate into better patient outcomes. Although Peterson et al attempted to minimize the risk of bias by using anonymous reporting and blinding participants to the timing of data collection,[21] adverse events during the intervention period could have been underestimated due to surveillance bias or decreased self‐reporting. Nevertheless, the results suggest that CHTs may have affected quality of patient care in a positive manner from included studies.
The findings from our review also point to a positive impact of CHT on physician work efficiency. Specifically, residents spent less time rounding on patients after handoff and finished their work sooner after introduction of the intervention.[20] Several other published studies on CHT also indicated potential benefits on work efficiency and/or patient safety,[31, 33, 34, 35] although they did not meet the inclusion criteria for our study (prespecified outcomes not reported,[31, 35] or study design[33, 34, 35]). In the studies in which the majority of handoff information was manually typed in the CHT, the work load was perceived to be increased with CHT implementation.[24, 25] On the other hand, the study conducted by Van Eaton et al demonstrated that a CHT that had broad integration with the hospital main IT system, and could automatically transfer important patient information such as medication, medical problems, recent investigation, and vital signs into CHT, quickly gained popularity among residents and staff due to its user‐friendly features.[20] This integration can also potentially reduce miscommunication and associated medical errors during physician handoff. Palma et al's study reported higher perceived workload due to manual entry of patient data.[24] Although the CHT used in their study was developed within their existing EMR system, large amounts of information needed to be manually imputed, and thus increased time spent on updating handoff information. This information included patient demographics, active medical issues, a to do list, and on‐going issues,[24] some of which could be imputed automatically with better CHT design. It is also possible that users spent more time in updating the handoff because they were able to deliver more information using a CHT.[24] However, this may allow cross‐covering physicians to spend less time on looking for patient information from other sources and thus actually decrease workload during cross‐coverage. Although there are numerous factors that could affect physician work efficiency when using a new IT system,[30] it was felt that a well‐designed and easy‐to‐use CHT that is integrated with the hospital information system can improve physician productivity.
The role of CHT in improving quality of handoff is less clear. Three studies[20, 24, 25] found an overall improvement in the quality of handoff after implementation of CHT, such that the handoff information was more complete and more consistent. On the other hand, physicians were concerned about the comprehensiveness of physician handoff after implementation of CHT in 2 studies.[22, 23] In Ram and Block's study,[25] physicians relied heavily on an unstructured free‐text entry system to deliver the majority of patient information that physicians thought to be important. In Flanagan et al's study,[23] resident physicians had to search for alternative sources, such as patient charts and electronic order systems, to obtain vital information in many cases in spite of a structured CHT. As a result, the information available was often not sufficient to help on‐call physicians make patient care decisions.[23]
Implication of CHT Design and Use
It has been demonstrated in many non‐healthcare domains,[15, 36, 37] as well as nursing care,[38] that a standardized handoff protocol is vital to decrease medical errors and improve patient safety. In our review, we found that physicians generally reported being satisfied with the accuracy of handoff information and the overall handoff when using standardized CHTs interfaced with hospital IT systems. This suggests, as recommended by Flanagan et al,[23] Palma et al,[24] and Ram and Block[25] that CHTs be developed with a standardized protocol and wide integration into hospital IT systems.
In order to achieve this goal, key patient information necessary for patient care need to be communicated during physician handoff. As hospitals consist of a wide range of disciplines and specialties with varying cultures and focuses of patient information, it is likely difficult to develop a single panacea CHT template for all the in‐hospital services.[1] This may be even particularly relevant when developing CHTs for different hospital services. However, some patient information appears to be universally important for physician handoff for inpatient care. Key elements, such as patient demographics, diagnosis, outstanding investigation results, code status, a problem list, and a to do list, were noted to be consistently present in the CHTs that were evaluated in our review (Table 2). Other studies have also demonstrated that information items such as a to do list, outstanding investigation results, and patients' code status were regarded as the most important information during physician handoff.[1, 2, 17, 23, 39, 40] Based on these findings, a potential solution for CHT standardization would be to develop a core CHT which includes the universally important components of physician handoff identified in this review, and provides options for adding well‐categorized service‐specific information as needed (eg, type and date of surgical procedures for surgical patients). It also appears that active involvement of physicians in CHT design and modification facilitates successful implementation of CHT, as demonstrated in Van Eaton et al's and Peterson et al's studies.[20, 21]
It is difficult to recommend metrics for CHT evaluation based on the limited literature identified in our review. However, it appears to be reasonable to consider integration into existing IT system, user friendly features, impact on quality of handoff documents, work efficiency, and processes and outcomes of patient care when assessing CHTs.
Limitations
There are several limitations in the studies included in our review. None of the studies were multi‐centered. The majority of the included studies had a beforeafter design.[21, 22, 23, 24, 25] Some studies did not have user training or a run in period to ensure familiarity of CHTs by users.[22, 24, 25] None of the studies described the key components of handoff in the control groups, or used quality control measurements for user familiarity with the CHTs. Furthermore, outcomes reported by the studies were heterogeneous, subjective, based on participant self‐report, and not independently validated.
Our review also has also several limitations. First, in spite of a comprehensive search effort, it is possible that we failed to identify all relevant articles. However, this is unlikely, given that we searched multiple databases and performed hand searches of all references identified from the included articles, as well as content‐related previously published systematic reviews. Second, we were not able to perform a meta‐analysis, given the heterogeneity seen in outcomes assessed across studies, measures applied, and results presented.
CONCLUSIONS AND IMPLICATIONS FOR PRACTICE
Although the current literature suggests that implementation of CHTs is likely to improve physician work efficiency, satisfaction, and quality of patient care during physician handoff for hospitalized patients, the evidence supporting these potential benefits is limited. Furthermore, it is unknown what impacts CHTs may have on clinical outcomes, such as hospital length of stay and mortality. Further studies with larger sample size, multiple center involvement, and more objective patient outcome measurements are therefore needed to evaluate the roles of CHTs in physician handoff and improving the quality of patient care.
In the absence of larger studies evaluating major clinical outcomes, such as length of stay and mortality, hospitals considering innovations in the domain of computerized platforms for physician handoffs will need to consider the pros and cons of immediate system implementation on the basis of the evidence presented here versus waiting until there is more evidence from more definitive studies. In addition, our study suggests that organizations engage physicians during CHT design and develop a standardized CHT protocol that is interfaced with hospital IT systems and includes key components of handoff information, but provides flexibility to meet service‐specific needs. The evidence summarized here, while far from definitive for major outcomes, is nonetheless rather positive for the general benefits of CHTan impetus for careful design, implementation, and modification, whenever and wherever possible. Any such system implementations should, however, incorporate an evaluative component so that the evidence‐base surrounding CHT can be enhanced.
Acknowledgments
Disclosure: Nothing to report.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , , . Lost in translation: challenges and opportunities in physician‐to‐physician communication during patient handoffs. Acad Med. 2005;80(12):1094–1099.
- World Health Organization. Patient safety solution: communication during patient handovers. Available at: http://www.who.int/patientsafety/solutions/patientsafety/PS‐Solution3.pdf Accessed January 20, 2011.
- Accreditation Canada. Required Organizational Practices: Communication. Available at: http://wwwaccreditationca/uploadedFiles/information%20transferpdf?n=1212. Accessed January 20, 2010.
- Joint Commission on Accreditation of Healthcare Organizations National Patient Safety Goals. Available at: http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/06_npsg_cah.html. Accessed January 20, 2010.
- , , . Communicating in the “gray zone”: perceptions about emergency physician hospitalist handoffs and patient safety. Acad Emerg Med. 2007;14(10):884–894.
- . Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005;142(5):352–358.
- , , , . Transfers of patient care between house staff on internal medicine wards: a national survey. Arch Intern Med. 2006;166(11):1173–1177.
- , , , et al. Consequences of inadequate sign‐out for patient care. Arch Intern Med. 2008;168(16):1755–1760.
- , , , et al. What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff. Qual Saf Health Care. 2009;18(4):248–255.
- , , , et al. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Ann Emerg Med. 2009;53(6):701–710.
- , , , . Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med. 2007;167(19):2030–2036.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , . Utility of a standardized sign‐out card for new medical interns. J Gen Intern Med. 1996;11(12):753–755.
- , , , et al. Handoff strategies in settings with high consequences for failure: lessons for health care operations. Int J Qual Health Care. 2004;16(2):125–132.
- , , . Enhancing patient safety: improving the patient handoff process through appreciative inquiry. J Nurs Adm. 2007;37(2):95–104.
- , , , et al. Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign‐out. J Hosp Med. 2006;1(4):257–266.
- , , , . Organizing the transfer of patient care information: the development of a computerized resident sign‐out system. Surgery. 2004;136(1):5–13.
- , , , et al. Hospitalist handoffs: a systematic review and task force recommendations. J Hosp Med. 2009;4(7):433–440.
- , , , et al. A randomized, controlled trial evaluating the impact of a computerized rounding and sign‐out system on continuity of care and resident work hours. J Am Coll Surg. 2005;200(4):538–545.
- , , , et al. Using a computerized sign‐out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improve. 1998;24(2):77–87.
- , , , . Electronic medical handover: towards safer medical care. Med J Aust. 2005;183(7):369–372.
- , , , . Evaluation of a physician informatics tool to improve patient handoffs. J Am Med Inform Assoc. 2009;16(4):509–515.
- , , . Impact of electronic medical record integration of a handoff tool on sign‐out in a newborn intensive care unit. J Perinatol. 2011;31(5):311–317.
- , . Signing out patients for off‐hours coverage: comparison of manual and computer‐aided methods. Proceedings—The Annual Symposium on Computer Applications in Medical Care. 1992;114–118.
- , . MediSign: using a Web‐based SignOut system to improve provider identification. Proc AMIA Symp. 1999:550–554.
- , , , et al. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84(2–3):181–190.
- , . Signing on to sign out, part 2: describing the success of a Web‐based patient sign‐out application and how it will serve as a platform for an electronic discharge summary program. Healthc Q. 2007;10(1):120–124.
- , , , et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs. 2005;24(5):1103–1117.
- , , , et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev. 2009;Jan21(1):CD006093.
- , , . Improving physician communication through an automated, integrated sign‐out system. J Healthc Inf Manag. 2005;19(4):68–74.
- , , , et al. SynopSIS: integrating physician sign‐out with the electronic medical record. J Hosp Med. 2007;2(5):336–342.
- , , , . Improved physician work flow after integrating sign‐out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010;36(2):72–78.
- , , , et al. Electronic inpatient whiteboards: improving multidisciplinary communication and coordination of care. Int J Med Inform. 2009;78(4):239–247.
- , , , et al. Systematically improving physician assignment during in‐hospital transitions of care by enhancing a preexisting hospital electronic health record. J Hosp Med. 2009;4(5):308–312.
- . On error management: lessons from aviation. BMJ. 2000;320(7237):781–785.
- , , , . There is more to monitoring a nuclear power plant than meets the eye. Hum Factors 2000;42(1):36–55.
- , , . Handoffs in care—can we make them safer?Pediatr Clin North Am. 2006;53(6):1185–1195.
- , , , et al. The top 10 list for a safe and effective sign‐out. Arch Surg. 2008;143(10):1008–1010.
- , , . Systematic review of handoff mnemonics literature. Am J Med Qual. 2009;24(3):196–204.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , , . Lost in translation: challenges and opportunities in physician‐to‐physician communication during patient handoffs. Acad Med. 2005;80(12):1094–1099.
- World Health Organization. Patient safety solution: communication during patient handovers. Available at: http://www.who.int/patientsafety/solutions/patientsafety/PS‐Solution3.pdf Accessed January 20, 2011.
- Accreditation Canada. Required Organizational Practices: Communication. Available at: http://wwwaccreditationca/uploadedFiles/information%20transferpdf?n=1212. Accessed January 20, 2010.
- Joint Commission on Accreditation of Healthcare Organizations National Patient Safety Goals. Available at: http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/06_npsg_cah.html. Accessed January 20, 2010.
- , , . Communicating in the “gray zone”: perceptions about emergency physician hospitalist handoffs and patient safety. Acad Emerg Med. 2007;14(10):884–894.
- . Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005;142(5):352–358.
- , , , . Transfers of patient care between house staff on internal medicine wards: a national survey. Arch Intern Med. 2006;166(11):1173–1177.
- , , , et al. Consequences of inadequate sign‐out for patient care. Arch Intern Med. 2008;168(16):1755–1760.
- , , , et al. What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff. Qual Saf Health Care. 2009;18(4):248–255.
- , , , et al. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Ann Emerg Med. 2009;53(6):701–710.
- , , , . Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med. 2007;167(19):2030–2036.
- , , , et al. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.
- , , . Utility of a standardized sign‐out card for new medical interns. J Gen Intern Med. 1996;11(12):753–755.
- , , , et al. Handoff strategies in settings with high consequences for failure: lessons for health care operations. Int J Qual Health Care. 2004;16(2):125–132.
- , , . Enhancing patient safety: improving the patient handoff process through appreciative inquiry. J Nurs Adm. 2007;37(2):95–104.
- , , , et al. Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign‐out. J Hosp Med. 2006;1(4):257–266.
- , , , . Organizing the transfer of patient care information: the development of a computerized resident sign‐out system. Surgery. 2004;136(1):5–13.
- , , , et al. Hospitalist handoffs: a systematic review and task force recommendations. J Hosp Med. 2009;4(7):433–440.
- , , , et al. A randomized, controlled trial evaluating the impact of a computerized rounding and sign‐out system on continuity of care and resident work hours. J Am Coll Surg. 2005;200(4):538–545.
- , , , et al. Using a computerized sign‐out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improve. 1998;24(2):77–87.
- , , , . Electronic medical handover: towards safer medical care. Med J Aust. 2005;183(7):369–372.
- , , , . Evaluation of a physician informatics tool to improve patient handoffs. J Am Med Inform Assoc. 2009;16(4):509–515.
- , , . Impact of electronic medical record integration of a handoff tool on sign‐out in a newborn intensive care unit. J Perinatol. 2011;31(5):311–317.
- , . Signing out patients for off‐hours coverage: comparison of manual and computer‐aided methods. Proceedings—The Annual Symposium on Computer Applications in Medical Care. 1992;114–118.
- , . MediSign: using a Web‐based SignOut system to improve provider identification. Proc AMIA Symp. 1999:550–554.
- , , , et al. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84(2–3):181–190.
- , . Signing on to sign out, part 2: describing the success of a Web‐based patient sign‐out application and how it will serve as a platform for an electronic discharge summary program. Healthc Q. 2007;10(1):120–124.
- , , , et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs. 2005;24(5):1103–1117.
- , , , et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev. 2009;Jan21(1):CD006093.
- , , . Improving physician communication through an automated, integrated sign‐out system. J Healthc Inf Manag. 2005;19(4):68–74.
- , , , et al. SynopSIS: integrating physician sign‐out with the electronic medical record. J Hosp Med. 2007;2(5):336–342.
- , , , . Improved physician work flow after integrating sign‐out notes into the electronic medical record. Jt Comm J Qual Patient Saf. 2010;36(2):72–78.
- , , , et al. Electronic inpatient whiteboards: improving multidisciplinary communication and coordination of care. Int J Med Inform. 2009;78(4):239–247.
- , , , et al. Systematically improving physician assignment during in‐hospital transitions of care by enhancing a preexisting hospital electronic health record. J Hosp Med. 2009;4(5):308–312.
- . On error management: lessons from aviation. BMJ. 2000;320(7237):781–785.
- , , , . There is more to monitoring a nuclear power plant than meets the eye. Hum Factors 2000;42(1):36–55.
- , , . Handoffs in care—can we make them safer?Pediatr Clin North Am. 2006;53(6):1185–1195.
- , , , et al. The top 10 list for a safe and effective sign‐out. Arch Surg. 2008;143(10):1008–1010.
- , , . Systematic review of handoff mnemonics literature. Am J Med Qual. 2009;24(3):196–204.
