User login
Interventions to Reduce the Overuse of Imaging for Pulmonary Embolism: A Systematic Review
The last 2 decades have seen a dramatic rise in the use of medical imaging in general,1,2 as well as in the diagnostic workup of pulmonary embolism (PE) more specifically, since the introduction of multidetector row computed tomography pulmonary angiography (CTPA) in 1998.3 From 1999 to 2010, the proportions of emergency department (ED) visits associated with a diagnosis of PE and admissions for PE have increased markedly in the United States, where the situation has been well documented.4,5 A 14-fold increase in the use of CTPA was observed in health maintenance organizations from 2001 to 2008.3 A significant increase in the probability of having a diagnosis of PE in the ED was reported, likely because of increased access to CTPA, from 2001 to 2010.4 With a prevalence of 2% or less in the ED, diagnostic yields as low as 5% suggest a significant problem of overuse.6,7
Strategies have been proposed to improve the appropriateness of imaging in the detection of PE, and these rely on the use of a validated clinical decision rule (CDR) to assess the pretest probability of the diagnosis. The purpose of this systematic review is to summarize the evidence associated with interventions aimed at reducing the overuse of imaging in the diagnostic workup of PE in the ED and hospital wards. Specifically, the types of interventions, their clinical effectiveness, as well as possible harms will be assessed. A secondary objective is to appraise the impact of these interventions on healthcare costs as well as the facilitators and barriers to their implementation.
METHODS
Inclusion Criteria
Targeted settings were EDs and inpatient services of adult tertiary and quaternary care hospitals. The search addressed interventions aimed at reducing the overuse of imaging in the diagnostic workup for PE. The comparators were usual care or another type of related intervention. The main outcomes considered were the use of imaging, diagnostic yield, radiation dose, adherence to guidelines to a quality measure, safety, and costs; both experimental and observational studies were included.
Literature Search
A systematic literature search in the following electronic databases was performed: PubMed, MEDLINE, Embase, and EBM Reviews (Cochrane, ACP Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Cochrane Health Technology Assessment, and the NHS Economic Evaluation Database). The reference period was from 1998 to March 28, 2017, and publications in English and French were searched. The detailed search strategy, adapted to each of the databases, appears in supplemental Appendix 1.
Study Selection and Data Extraction
One author (SD) reviewed the titles of the selected articles and excluded those that obviously did not satisfy the inclusion criteria. Then, 2 authors (SD and LL) independently reviewed the titles and abstracts of the remaining articles. They reviewed the full manuscript of potentially relevant articles for inclusion. Disagreements that could not be resolved by discussion would have been arbitrated by a third author (CCL); however, no such disagreement occurred.
Quality and Risk of Bias Assessment
For experimental or quasiexperimental studies that involved an intervention group and a control group, the criteria proposed by the Cochrane collaborative for the evaluation of bias were used.8 For studies using a before and after design, the following main biases associated with such designs were assessed: history effect, maturation bias, testing bias, regression to the mean, and conditioning bias.9
Data Extraction and Synthesis
Data pertaining to efficacy, safety, costs, and facilitators and barriers to the implementation of interventions were extracted from the studies. The research process adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 checklist.10 In view of the heterogeneity of the studies, a narrative synthesis was produced in accordance with the methodology proposed by Popay et al.11 The review protocol was registered in PROSPERO (this registry can be consulted at the following URL address: http://www.crd.york.ac.uk/PROSPERO/).
RESULTS
The search screened 2814 records after the removal of duplicates and studies published before 1998. The figure illustrates the literature selection process.12 Seventeen studies were included in the review following appraisal. Most of the studies (15/17) evaluated interventions in the ED,7,13-26 while the remaining studies (2/17) were conducted in clinical wards of acute care hospitals.27,28 Thirteen studies were conducted in the United States, 3 in Australia, and 1 in Europe. Four types of interventions were identified in the selected studies: electronic clinical decision support (CDS) (8/17), educational interventions (7/17), performance feedback reports (PFRs) (1/17), and an institutional clinical pretest policy (1/17). In 10 of the studies, the proposed intervention was mandatory.
One systematic review and meta-analysis pertaining to the impact of CDRs on CTPA use and yield was identified.29 Five of the studies it included were also included in the present review.13,16,21-23 However, its focus is different than the present one, which aims at assessing the evidence associated with the interventions being implemented to promote the use of the CDRs.29
The list of included studies appears in supplemental Appendix 2. The list of potentially relevant studies that were finally excluded is provided in supplemental Appendix 3.
Most studies (14/17) presented a before-after design, with data collection corresponding to periods preceding and following a specific intervention. Most of them are retrospective and assessed the efficacy and safety results. They were deemed of generally poor quality and were subject to many of the biases mentioned above as well as to an interaction between the intervention and its implementation context. The remaining 3 studies were experimental in design with a comparative control group.13,14,27 In 2 of these studies, a comparison was made with traditional clinical practice (no intervention).13,27 In the third, the intervention was compared with CDS only.14 The control group studies were of intermediate to very good quality and were subject to biases of performance, detection, selection, and attrition.
Table 1 summarizes the study characteristics of the included studies. The detailed methodological quality appraisal of the control group studies appears in supplemental Appendix 4.
Efficacy
CDS and PFRs
Eight of the studies appraised CDS interventions.13,16,17,19,21,22,24,28 They consisted of computer-based applications imbedded into the computerized physician order entry of the setting (ED or clinical ward of an acute care hospital), which are prompted when a physician orders an imaging exam or D-dimer test.
Educational Interventions and Policy
Seven of the interventions assessed in the included studies were educational in their essence, involving training sessions aimed at strengthening physician use of CDRs for the diagnosis of PE.15,18,20,23,25-27 Three studies observed a statistically significant impact on the
The impact of a policy fostering the use of a CDR and D-dimer was appraised in 1 study.7 This intervention translated into a significant reduction of CTPA use and a significant increase of CTPA diagnostic yield. However, only 4% of patient charts reported a clinical probability of PE, and in most cases, the type of CDR used was not mentioned.7
Safety
A minority of studies evaluated the safety of the interventions.13,18,19,23,25,27 Only 2 of these
The 2 studies involving a control group did not find significant differences between the intervention and the control groups with respect to mortality, complications because of thromboembolic and bleeding events, or any other adverse event during the 3-months’ follow-up.13,27
Jiménez et al.19 reported less than 1% mortality following the implementation of a CDS (0.7%; 95% CI, 0.2%-1.1%). In their study assessing the impact of an educational intervention, Kline et al.23 (2004) observed that none of the patients discharged with a fully negative Charlotte rule died suddenly and unexpectedly at 90-day follow-up. However, another educational intervention aimed at reducing ED patients’ radiation exposure observed a significant increase in the 90-day all-cause mortality of patients with negative CTPA, which was associated with a decline in the 90-day mortality of patients with negative ventilation/perfusion (V/Q) scanning.25
Jiménez et al.19 observed an absolute decrease of 2.5% in the incidence of symptomatic VTE events after the intervention (95% CI, 0.9%-4.6%; P < .01). The occurrence of VTE events, including PE, reached 1% in Goergen et al.18 and 3.9% in Kline et al.23 (2004) during follow-up.
Economic Aspects
Kline et al.13 (2014) found a significant decrease in charges and estimated costs for medical care within 90 days of initial ED presentation in the patients who were investigated with CTPA in the intervention group. The median costs of medical care within 30 days of the initial ED presentation were US $1274 in the control group and US $934 in the intervention group (P = .018).13 The median charges of medical care within 30 days of the initial ED presentation were US $7595 in the control group and US $6281 in the intervention group (P = .004).13
Facilitators and Barriers
Only 1 study appraised the reasons given by emergency physicians for not adhering to CDS recommendations.16 The reason most often given was the time needed to access and use the application, which was perceived as having a negative impact on productivity as well as a preference for intuitive clinical judgment.16 Though not the result of specific evaluation or data collection, some authors commented on the factors that may facilitate or impede the implementation of interventions to diminish the inappropriate use.14,20 Kanaan et al.20 proposed that factors other than the knowledge of current clinical guidelines may explain CTPA use. Booker and Johnson26 suggested that the demand for rapid turnover in the ED may lead to “so-called ‘blanket ordering’, which attempts to reach diagnosis as quickly as possible despite cost and patient safety.” Raja et al.14 (2015) suggested that the unambiguous representation of guidelines based on validated, high-quality evidence in the CDS may have improved physician adoption in their study.
DISCUSSION
Efficacy
Baseline values for the use of imaging and diagnostic yield show important variation, especially when compared with the study performed in Europe.19 In general, only a modest impact is measured with regard to a decrease in the use of imaging, an increase in diagnostic use, and adherence to validated CDRs.
Among the interventions appraised, CDS was evaluated in the largest number of included studies, and its
The impact of CDS on diagnostic yield was mixed because 3 studies observed an increase in diagnostic yield postintervention,16,21,22 and 3 others monitored no significant impact.19,24,28 Adherence to guidelines or a quality measure was assessed in 2 studies, which reported a significant increase in appropriate ordering.17,24 Raja et al.24 (2014) observed an 18.7% increase in appropriate ordering after the implementation of a CDS from 56.9% to 75.6% (P < .01). Geeting et al.17 observed a similar increase, with appropriate ordering increasing from 58% to 76% over the duration of the intervention. However, this increase in appropriate use was not associated with a variation in CTPA use or diagnostic yield, which leads the investigators to posit that the physicians gradually inflated the Wells score they keyed into the CDS despite that no threshold Wells score was required to perform a CTPA.17
Raja et al.14 (2015) demonstrated that the implementation of performance feedback reporting, in addition to a CDS, can significantly increase adherence to CDR for the evaluation of PE in the ED. Additional studies would help to better understand the potential impact of such reports on CTPA use in the diagnostic workup of PE. However, it suggests that a combination of interventions, including the implementation of a CDS, performance feedback reporting, and well-designed and specific educational interventions, may have a more significant impact than any of these types of interventions taken separately.
The impact of the educational interventions appraised in this review on the expected results is mixed, though it is difficult to compare the observed results and draw conclusive remarks, as the characteristics of the interventions and study designs are different from each other.
Safety
There is limited evidence on the safety of appraised interventions. Only 6 studies appraised venous thrombolic events or mortality.13,18,19,23,25,27 However, no adverse events were noted in those studies evaluating possible complications or missed diagnoses. Additional research is needed to confirm the safety of the interventions appraised in this systematic review.
Facilitators and Barriers
There are significant limitations with respect to the analysis of the factors that favor or impede the implementation of the interventions appraised in this review. However, 2 studies that did not meet the inclusion criteria appraised physicians’ perceptions and attitudes toward prescribing imaging tests in the diagnostic workup of PE.31,32 One is Swiss31 and the other is Canadian.32 Both were conducted in the ED of academic hospitals. Rohacek et al.31 observed that defensive behaviors, such as “fear of missing PE,” were frequent and associated with a lower probability of a positive CTPA (OR = 0.36; 95% CI, 0.14-0.92). Ahn et al.32 concluded that, although ED physicians who participated in their survey possessed limited knowledge of radiation doses of CTPA and V/Q scans, they opted for V/Q scans that emit lower radiation doses in younger patients, especially females, which may reflect efforts done in the study setting to reduce patients’ radiation exposure.
There is not enough data to conclude on safety and the impact on healthcare costs.
Implications for Future Research
Future controlled studies of high methodological quality would help to better understand the effects associated with the implementation of the interventions aimed at reducing the inappropriate use of imaging in the diagnostic workup of PE. Efficacy results show that the success of the implementation of the various types of interventions is variable. This variation may be at least partly attributable to contextual factors, such as the external environment, the organizational leadership and culture, or the microsystem, such as differences in care patterns.33-35 The impact of context factors on the effectiveness of the interventions should be assessed further with appropriate tools.33,34,36
CONCLUSION
The joint use of CDS and PFRs appears more effective than the other types of intervention in reducing the inappropriate use of CTPA. However, an approach combining these with well-designed educational interventions as well as policies may be even more effective.
Future studies of high methodological quality would strengthen the evidence concerning the relative efficacy and safety of the interventions appraised, especially when various types are combined. Future research should also aim at bringing answers to the knowledge gaps related to the factors of success and barriers associated with the implementation of the interventions.
Disclosure
The authors report no conflict of interest.
1. Smith-Bindman R, Miglioretti DL, Johnson E, et al. Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. JAMA. 2012;307(22):2400-2409. PubMed
2. Canadian Institute for Health Information (CIHI). Medical Imaging in Canada 2012. https://www.cihi.ca/en/mit_summary_2012_en.pdf. Accessed December 14, 2016.
3. Wiener RS, Schwartz LM, Woloshin S. When a test is too good: how CT pulmonary angiograms find pulmonary emboli that do not need to be found. BMJ. 2013;347:f3368. doi:10.1136/bmj.f3368. PubMed
4. Schissler AJ, Rozenshtein A, Schluger NW, Einstein AJ. National trends in emergency room diagnosis of pulmonary embolism, 2001-2010: a cross-sectional study. Respir Res. 2015;16:44-50. PubMed
5. Minges KE, Bikdeli B, Wang Y, et al. National Trends in Pulmonary Embolism Hospitalization Rates and Outcomes for Adults Aged >/=65 Years in the United States (1999 to 2010). Am J Cardiol. 2015;116(9):1436-1442. PubMed
6. Duriseti RS, Brandeau ML. Cost-effectiveness of strategies for diagnosing pulmonary embolism among emergency department patients presenting with undifferentiated symptoms. Ann Emerg Med. 2010;56(4):321-332.e310. PubMed
7. Char S, Yoon HC. Improving appropriate use of pulmonary computed tomography angiography by increasing the serum D-dimer threshold and assessing clinical probability. Perm J. 2014;18(4):10-15. PubMed
8. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMed
9. Champagne F, Brousselle A, Contendriopoulos AP, Hartz Z. L’analyse des effets. In: Brousselle A, Champagne F, Contandriopoulos AP, Hartz Z, editors. L’évaluation: Concepts et Méthodes 2e Edition. Montréal: Les Presses de l’Université de Montréal; 2011: 173-198.
10. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-1012. PubMed
11. Popay J, Roberts H, Sowden A, et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Manchester, UK: ESRC Methods Programme; 2006.
12. Velasco M, Perleth M, Drummond M, et al. Best practice in undertaking and reporting health technology assessments. Working group 4 report. Int J Technol Assess Health Care. 2002;18(2):361-422. PubMed
13. Kline JA, Jones AE, Shapiro NI, et al. Multicenter, randomized trial of quantitative pretest probability to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Circ Cardiovasc Imaging. 2014;7(1):66-73. PubMed
14. Raja AS, Ip IK, Dunne RM, Schuur JD, Mills AM, Khorasani R. Effects of Performance Feedback Reports on Adherence to Evidence-Based Guidelines in Use of CT for Evaluation of Pulmonary Embolism in the Emergency Department: A Randomized Trial. AJR Am J Roentgenol. 2015;205(5):936-940. PubMed
15. Agarwal A, Persaud J, Grabinski R, Rabinowitz D, Bremner A, Mendelson R. Pulmonary embolism: are we there yet? J Med Imaging Radiat Oncol. 2012;56(3):270-281. PubMed
16. Drescher FS, Chandrika S, Weir ID, et al. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med. 2011;57(6):613-621. PubMed
17. Geeting GK, Beck M, Bruno MA, et al. Mandatory Assignment of Modified Wells Score Before CT Angiography for Pulmonary Embolism Fails to Improve Utilization or Percentage of Positive Cases. AJR Am J Roentgenol. 2016;207(2):442-449. PubMed
18. Goergen SK, Chan T, de Campo JF, et al. Reducing the use of diagnostic imaging in patients with suspected pulmonary embolism: validation of a risk assessment strategy. Emerg Med Australas. 2005;17(1):16-23. PubMed
19. Jiménez D, Resano S, Otero R, et al. Computerised clinical decision support for suspected PE. Thorax. 2015;70(9):909-911. PubMed
20. Kanaan Y, Knoepp UD, Kelly AM. The influence of education on appropriateness rates for CT pulmonary angiography in emergency department patients. Acad Radiol. 2013;20(9):1107-1114. PubMed
21. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
22. Raja AS, Ip IK, Prevedello LM, et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012;262(2):468-474. PubMed
23. Kline JA, Webb WB, Jones AE, Hernandez-Nino J. Impact of a rapid rule-out protocol for pulmonary embolism on the rate of screening, missed cases, and pulmonary vascular imaging in an urban US emergency department. Ann Emerg Med. 2004;44(5):490-502. PubMed
24. Raja AS, Gupta A, Ip IK, Mills AM, Khorasani R. The use of decision support to measure documented adherence to a national imaging quality measure. Acad Radiol. 2014;21(3):378-383. PubMed
25. Stein EG, Haramati LB, Chamarthy M, Sprayregen S, Davitt MM, Freeman LM. Success of a safe and simple algorithm to reduce use of CT pulmonary angiography in the emergency department. AJR Am J Roentgenol. 2010;194(2):392-397. PubMed
26. Booker MT, Johnson JO. Optimizing CT Pulmonary Angiogram Utilization in a Community Emergency Department: A Pre- and Postintervention Study. J Am Coll Radiol. 2017;14(1):65-71. PubMed
27. Goldstein NM, Kollef MH, Ward S, Gage BF. The impact of the introduction of a rapid D-dimer assay on the diagnostic evaluation of suspected pulmonary embolism. Arch Intern Med. 2001;161(4):567-571. PubMed
28. Dunne RM, Ip IK, Abbett S, et al. Effect of Evidence-based Clinical Decision Support on the Use and Yield of CT Pulmonary Angiographic Imaging in Hospitalized Patients. Radiology. 2015;276(1):167-174. PubMed
29. Wang RC, Bent S, Weber E, Neilson J, Smith-Bindman R, Fahimi J. The Impact of Clinical Decision Rules on Computed Tomography Use and Yield for Pulmonary Embolism: A Systematic Review and Meta-analysis. Ann Emerg Med. 2016;67(6):693-701. PubMed
30. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
31. Rohacek M, Buatsi J, Szucs-Farkas Z, et al. Ordering CT pulmonary angiography to exclude pulmonary embolism: defense versus evidence in the emergency room. Intensive Care Med. 2012;38(8):1345-1351. PubMed
32. Ahn JS, Edmonds ML, McLeod SL, Dreyer JF. Familiarity with radiation exposure dose from diagnostic imaging for acute pulmonary embolism and current patterns of practice. CJEM. 2014;16(5):393-404. PubMed
33. Kringos DS, Sunol R, Wagner C, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res. 2015;15(277):015-0906. PubMed
34. Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q. 2010;88(4):500-559. PubMed
35. Pernod G, Caterino J, Maignan M, Tissier C, Kassis J, Lazarchick J. D-dimer use and pulmonary embolism diagnosis in emergency units: Why is there such a difference in pulmonary embolism prevalence between the United States of America and countries outside USA? PLoS ONE. 2017;12(1):e0169268. doi:10.1371/journal.pone.0169268 PubMed
36. Saillour-Glenisson F, Domecq S, Kret M, Sibe M, Dumond JP, Michel P. Design and validation of a questionnaire to assess organizational culture in French hospital wards. BMC Health Serv Res. 2016;16:491-503. PubMed
37. Kline JA, Nelson RD, Jackson RE, Courtney DM. Criteria for the safe use of D-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med. 2002;39(2):144-152. PubMed
38. Stein PD, Fowler SE, Goodman LR, et al. Multidetector computed tomography for acute pulmonary embolism. New Engl J Med. 2006;354(22):2317-2327. PubMed
39. Stein PD, Woodard PK, Weg JG, et al. Diagnostic pathways in acute pulmonary embolism: recommendations of the PIOPED II investigators. Am J Med. 2006;119(12):1048-1055. PubMed
40. Torbicki A, Perrier A, Konstantinides S, et al. Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J. 2008;29(18):2276-2315. PubMed
The last 2 decades have seen a dramatic rise in the use of medical imaging in general,1,2 as well as in the diagnostic workup of pulmonary embolism (PE) more specifically, since the introduction of multidetector row computed tomography pulmonary angiography (CTPA) in 1998.3 From 1999 to 2010, the proportions of emergency department (ED) visits associated with a diagnosis of PE and admissions for PE have increased markedly in the United States, where the situation has been well documented.4,5 A 14-fold increase in the use of CTPA was observed in health maintenance organizations from 2001 to 2008.3 A significant increase in the probability of having a diagnosis of PE in the ED was reported, likely because of increased access to CTPA, from 2001 to 2010.4 With a prevalence of 2% or less in the ED, diagnostic yields as low as 5% suggest a significant problem of overuse.6,7
Strategies have been proposed to improve the appropriateness of imaging in the detection of PE, and these rely on the use of a validated clinical decision rule (CDR) to assess the pretest probability of the diagnosis. The purpose of this systematic review is to summarize the evidence associated with interventions aimed at reducing the overuse of imaging in the diagnostic workup of PE in the ED and hospital wards. Specifically, the types of interventions, their clinical effectiveness, as well as possible harms will be assessed. A secondary objective is to appraise the impact of these interventions on healthcare costs as well as the facilitators and barriers to their implementation.
METHODS
Inclusion Criteria
Targeted settings were EDs and inpatient services of adult tertiary and quaternary care hospitals. The search addressed interventions aimed at reducing the overuse of imaging in the diagnostic workup for PE. The comparators were usual care or another type of related intervention. The main outcomes considered were the use of imaging, diagnostic yield, radiation dose, adherence to guidelines to a quality measure, safety, and costs; both experimental and observational studies were included.
Literature Search
A systematic literature search in the following electronic databases was performed: PubMed, MEDLINE, Embase, and EBM Reviews (Cochrane, ACP Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Cochrane Health Technology Assessment, and the NHS Economic Evaluation Database). The reference period was from 1998 to March 28, 2017, and publications in English and French were searched. The detailed search strategy, adapted to each of the databases, appears in supplemental Appendix 1.
Study Selection and Data Extraction
One author (SD) reviewed the titles of the selected articles and excluded those that obviously did not satisfy the inclusion criteria. Then, 2 authors (SD and LL) independently reviewed the titles and abstracts of the remaining articles. They reviewed the full manuscript of potentially relevant articles for inclusion. Disagreements that could not be resolved by discussion would have been arbitrated by a third author (CCL); however, no such disagreement occurred.
Quality and Risk of Bias Assessment
For experimental or quasiexperimental studies that involved an intervention group and a control group, the criteria proposed by the Cochrane collaborative for the evaluation of bias were used.8 For studies using a before and after design, the following main biases associated with such designs were assessed: history effect, maturation bias, testing bias, regression to the mean, and conditioning bias.9
Data Extraction and Synthesis
Data pertaining to efficacy, safety, costs, and facilitators and barriers to the implementation of interventions were extracted from the studies. The research process adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 checklist.10 In view of the heterogeneity of the studies, a narrative synthesis was produced in accordance with the methodology proposed by Popay et al.11 The review protocol was registered in PROSPERO (this registry can be consulted at the following URL address: http://www.crd.york.ac.uk/PROSPERO/).
RESULTS
The search screened 2814 records after the removal of duplicates and studies published before 1998. The figure illustrates the literature selection process.12 Seventeen studies were included in the review following appraisal. Most of the studies (15/17) evaluated interventions in the ED,7,13-26 while the remaining studies (2/17) were conducted in clinical wards of acute care hospitals.27,28 Thirteen studies were conducted in the United States, 3 in Australia, and 1 in Europe. Four types of interventions were identified in the selected studies: electronic clinical decision support (CDS) (8/17), educational interventions (7/17), performance feedback reports (PFRs) (1/17), and an institutional clinical pretest policy (1/17). In 10 of the studies, the proposed intervention was mandatory.
One systematic review and meta-analysis pertaining to the impact of CDRs on CTPA use and yield was identified.29 Five of the studies it included were also included in the present review.13,16,21-23 However, its focus is different than the present one, which aims at assessing the evidence associated with the interventions being implemented to promote the use of the CDRs.29
The list of included studies appears in supplemental Appendix 2. The list of potentially relevant studies that were finally excluded is provided in supplemental Appendix 3.
Most studies (14/17) presented a before-after design, with data collection corresponding to periods preceding and following a specific intervention. Most of them are retrospective and assessed the efficacy and safety results. They were deemed of generally poor quality and were subject to many of the biases mentioned above as well as to an interaction between the intervention and its implementation context. The remaining 3 studies were experimental in design with a comparative control group.13,14,27 In 2 of these studies, a comparison was made with traditional clinical practice (no intervention).13,27 In the third, the intervention was compared with CDS only.14 The control group studies were of intermediate to very good quality and were subject to biases of performance, detection, selection, and attrition.
Table 1 summarizes the study characteristics of the included studies. The detailed methodological quality appraisal of the control group studies appears in supplemental Appendix 4.
Efficacy
CDS and PFRs
Eight of the studies appraised CDS interventions.13,16,17,19,21,22,24,28 They consisted of computer-based applications imbedded into the computerized physician order entry of the setting (ED or clinical ward of an acute care hospital), which are prompted when a physician orders an imaging exam or D-dimer test.
Educational Interventions and Policy
Seven of the interventions assessed in the included studies were educational in their essence, involving training sessions aimed at strengthening physician use of CDRs for the diagnosis of PE.15,18,20,23,25-27 Three studies observed a statistically significant impact on the
The impact of a policy fostering the use of a CDR and D-dimer was appraised in 1 study.7 This intervention translated into a significant reduction of CTPA use and a significant increase of CTPA diagnostic yield. However, only 4% of patient charts reported a clinical probability of PE, and in most cases, the type of CDR used was not mentioned.7
Safety
A minority of studies evaluated the safety of the interventions.13,18,19,23,25,27 Only 2 of these
The 2 studies involving a control group did not find significant differences between the intervention and the control groups with respect to mortality, complications because of thromboembolic and bleeding events, or any other adverse event during the 3-months’ follow-up.13,27
Jiménez et al.19 reported less than 1% mortality following the implementation of a CDS (0.7%; 95% CI, 0.2%-1.1%). In their study assessing the impact of an educational intervention, Kline et al.23 (2004) observed that none of the patients discharged with a fully negative Charlotte rule died suddenly and unexpectedly at 90-day follow-up. However, another educational intervention aimed at reducing ED patients’ radiation exposure observed a significant increase in the 90-day all-cause mortality of patients with negative CTPA, which was associated with a decline in the 90-day mortality of patients with negative ventilation/perfusion (V/Q) scanning.25
Jiménez et al.19 observed an absolute decrease of 2.5% in the incidence of symptomatic VTE events after the intervention (95% CI, 0.9%-4.6%; P < .01). The occurrence of VTE events, including PE, reached 1% in Goergen et al.18 and 3.9% in Kline et al.23 (2004) during follow-up.
Economic Aspects
Kline et al.13 (2014) found a significant decrease in charges and estimated costs for medical care within 90 days of initial ED presentation in the patients who were investigated with CTPA in the intervention group. The median costs of medical care within 30 days of the initial ED presentation were US $1274 in the control group and US $934 in the intervention group (P = .018).13 The median charges of medical care within 30 days of the initial ED presentation were US $7595 in the control group and US $6281 in the intervention group (P = .004).13
Facilitators and Barriers
Only 1 study appraised the reasons given by emergency physicians for not adhering to CDS recommendations.16 The reason most often given was the time needed to access and use the application, which was perceived as having a negative impact on productivity as well as a preference for intuitive clinical judgment.16 Though not the result of specific evaluation or data collection, some authors commented on the factors that may facilitate or impede the implementation of interventions to diminish the inappropriate use.14,20 Kanaan et al.20 proposed that factors other than the knowledge of current clinical guidelines may explain CTPA use. Booker and Johnson26 suggested that the demand for rapid turnover in the ED may lead to “so-called ‘blanket ordering’, which attempts to reach diagnosis as quickly as possible despite cost and patient safety.” Raja et al.14 (2015) suggested that the unambiguous representation of guidelines based on validated, high-quality evidence in the CDS may have improved physician adoption in their study.
DISCUSSION
Efficacy
Baseline values for the use of imaging and diagnostic yield show important variation, especially when compared with the study performed in Europe.19 In general, only a modest impact is measured with regard to a decrease in the use of imaging, an increase in diagnostic use, and adherence to validated CDRs.
Among the interventions appraised, CDS was evaluated in the largest number of included studies, and its
The impact of CDS on diagnostic yield was mixed because 3 studies observed an increase in diagnostic yield postintervention,16,21,22 and 3 others monitored no significant impact.19,24,28 Adherence to guidelines or a quality measure was assessed in 2 studies, which reported a significant increase in appropriate ordering.17,24 Raja et al.24 (2014) observed an 18.7% increase in appropriate ordering after the implementation of a CDS from 56.9% to 75.6% (P < .01). Geeting et al.17 observed a similar increase, with appropriate ordering increasing from 58% to 76% over the duration of the intervention. However, this increase in appropriate use was not associated with a variation in CTPA use or diagnostic yield, which leads the investigators to posit that the physicians gradually inflated the Wells score they keyed into the CDS despite that no threshold Wells score was required to perform a CTPA.17
Raja et al.14 (2015) demonstrated that the implementation of performance feedback reporting, in addition to a CDS, can significantly increase adherence to CDR for the evaluation of PE in the ED. Additional studies would help to better understand the potential impact of such reports on CTPA use in the diagnostic workup of PE. However, it suggests that a combination of interventions, including the implementation of a CDS, performance feedback reporting, and well-designed and specific educational interventions, may have a more significant impact than any of these types of interventions taken separately.
The impact of the educational interventions appraised in this review on the expected results is mixed, though it is difficult to compare the observed results and draw conclusive remarks, as the characteristics of the interventions and study designs are different from each other.
Safety
There is limited evidence on the safety of appraised interventions. Only 6 studies appraised venous thrombolic events or mortality.13,18,19,23,25,27 However, no adverse events were noted in those studies evaluating possible complications or missed diagnoses. Additional research is needed to confirm the safety of the interventions appraised in this systematic review.
Facilitators and Barriers
There are significant limitations with respect to the analysis of the factors that favor or impede the implementation of the interventions appraised in this review. However, 2 studies that did not meet the inclusion criteria appraised physicians’ perceptions and attitudes toward prescribing imaging tests in the diagnostic workup of PE.31,32 One is Swiss31 and the other is Canadian.32 Both were conducted in the ED of academic hospitals. Rohacek et al.31 observed that defensive behaviors, such as “fear of missing PE,” were frequent and associated with a lower probability of a positive CTPA (OR = 0.36; 95% CI, 0.14-0.92). Ahn et al.32 concluded that, although ED physicians who participated in their survey possessed limited knowledge of radiation doses of CTPA and V/Q scans, they opted for V/Q scans that emit lower radiation doses in younger patients, especially females, which may reflect efforts done in the study setting to reduce patients’ radiation exposure.
There is not enough data to conclude on safety and the impact on healthcare costs.
Implications for Future Research
Future controlled studies of high methodological quality would help to better understand the effects associated with the implementation of the interventions aimed at reducing the inappropriate use of imaging in the diagnostic workup of PE. Efficacy results show that the success of the implementation of the various types of interventions is variable. This variation may be at least partly attributable to contextual factors, such as the external environment, the organizational leadership and culture, or the microsystem, such as differences in care patterns.33-35 The impact of context factors on the effectiveness of the interventions should be assessed further with appropriate tools.33,34,36
CONCLUSION
The joint use of CDS and PFRs appears more effective than the other types of intervention in reducing the inappropriate use of CTPA. However, an approach combining these with well-designed educational interventions as well as policies may be even more effective.
Future studies of high methodological quality would strengthen the evidence concerning the relative efficacy and safety of the interventions appraised, especially when various types are combined. Future research should also aim at bringing answers to the knowledge gaps related to the factors of success and barriers associated with the implementation of the interventions.
Disclosure
The authors report no conflict of interest.
The last 2 decades have seen a dramatic rise in the use of medical imaging in general,1,2 as well as in the diagnostic workup of pulmonary embolism (PE) more specifically, since the introduction of multidetector row computed tomography pulmonary angiography (CTPA) in 1998.3 From 1999 to 2010, the proportions of emergency department (ED) visits associated with a diagnosis of PE and admissions for PE have increased markedly in the United States, where the situation has been well documented.4,5 A 14-fold increase in the use of CTPA was observed in health maintenance organizations from 2001 to 2008.3 A significant increase in the probability of having a diagnosis of PE in the ED was reported, likely because of increased access to CTPA, from 2001 to 2010.4 With a prevalence of 2% or less in the ED, diagnostic yields as low as 5% suggest a significant problem of overuse.6,7
Strategies have been proposed to improve the appropriateness of imaging in the detection of PE, and these rely on the use of a validated clinical decision rule (CDR) to assess the pretest probability of the diagnosis. The purpose of this systematic review is to summarize the evidence associated with interventions aimed at reducing the overuse of imaging in the diagnostic workup of PE in the ED and hospital wards. Specifically, the types of interventions, their clinical effectiveness, as well as possible harms will be assessed. A secondary objective is to appraise the impact of these interventions on healthcare costs as well as the facilitators and barriers to their implementation.
METHODS
Inclusion Criteria
Targeted settings were EDs and inpatient services of adult tertiary and quaternary care hospitals. The search addressed interventions aimed at reducing the overuse of imaging in the diagnostic workup for PE. The comparators were usual care or another type of related intervention. The main outcomes considered were the use of imaging, diagnostic yield, radiation dose, adherence to guidelines to a quality measure, safety, and costs; both experimental and observational studies were included.
Literature Search
A systematic literature search in the following electronic databases was performed: PubMed, MEDLINE, Embase, and EBM Reviews (Cochrane, ACP Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Cochrane Health Technology Assessment, and the NHS Economic Evaluation Database). The reference period was from 1998 to March 28, 2017, and publications in English and French were searched. The detailed search strategy, adapted to each of the databases, appears in supplemental Appendix 1.
Study Selection and Data Extraction
One author (SD) reviewed the titles of the selected articles and excluded those that obviously did not satisfy the inclusion criteria. Then, 2 authors (SD and LL) independently reviewed the titles and abstracts of the remaining articles. They reviewed the full manuscript of potentially relevant articles for inclusion. Disagreements that could not be resolved by discussion would have been arbitrated by a third author (CCL); however, no such disagreement occurred.
Quality and Risk of Bias Assessment
For experimental or quasiexperimental studies that involved an intervention group and a control group, the criteria proposed by the Cochrane collaborative for the evaluation of bias were used.8 For studies using a before and after design, the following main biases associated with such designs were assessed: history effect, maturation bias, testing bias, regression to the mean, and conditioning bias.9
Data Extraction and Synthesis
Data pertaining to efficacy, safety, costs, and facilitators and barriers to the implementation of interventions were extracted from the studies. The research process adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 checklist.10 In view of the heterogeneity of the studies, a narrative synthesis was produced in accordance with the methodology proposed by Popay et al.11 The review protocol was registered in PROSPERO (this registry can be consulted at the following URL address: http://www.crd.york.ac.uk/PROSPERO/).
RESULTS
The search screened 2814 records after the removal of duplicates and studies published before 1998. The figure illustrates the literature selection process.12 Seventeen studies were included in the review following appraisal. Most of the studies (15/17) evaluated interventions in the ED,7,13-26 while the remaining studies (2/17) were conducted in clinical wards of acute care hospitals.27,28 Thirteen studies were conducted in the United States, 3 in Australia, and 1 in Europe. Four types of interventions were identified in the selected studies: electronic clinical decision support (CDS) (8/17), educational interventions (7/17), performance feedback reports (PFRs) (1/17), and an institutional clinical pretest policy (1/17). In 10 of the studies, the proposed intervention was mandatory.
One systematic review and meta-analysis pertaining to the impact of CDRs on CTPA use and yield was identified.29 Five of the studies it included were also included in the present review.13,16,21-23 However, its focus is different than the present one, which aims at assessing the evidence associated with the interventions being implemented to promote the use of the CDRs.29
The list of included studies appears in supplemental Appendix 2. The list of potentially relevant studies that were finally excluded is provided in supplemental Appendix 3.
Most studies (14/17) presented a before-after design, with data collection corresponding to periods preceding and following a specific intervention. Most of them are retrospective and assessed the efficacy and safety results. They were deemed of generally poor quality and were subject to many of the biases mentioned above as well as to an interaction between the intervention and its implementation context. The remaining 3 studies were experimental in design with a comparative control group.13,14,27 In 2 of these studies, a comparison was made with traditional clinical practice (no intervention).13,27 In the third, the intervention was compared with CDS only.14 The control group studies were of intermediate to very good quality and were subject to biases of performance, detection, selection, and attrition.
Table 1 summarizes the study characteristics of the included studies. The detailed methodological quality appraisal of the control group studies appears in supplemental Appendix 4.
Efficacy
CDS and PFRs
Eight of the studies appraised CDS interventions.13,16,17,19,21,22,24,28 They consisted of computer-based applications imbedded into the computerized physician order entry of the setting (ED or clinical ward of an acute care hospital), which are prompted when a physician orders an imaging exam or D-dimer test.
Educational Interventions and Policy
Seven of the interventions assessed in the included studies were educational in their essence, involving training sessions aimed at strengthening physician use of CDRs for the diagnosis of PE.15,18,20,23,25-27 Three studies observed a statistically significant impact on the
The impact of a policy fostering the use of a CDR and D-dimer was appraised in 1 study.7 This intervention translated into a significant reduction of CTPA use and a significant increase of CTPA diagnostic yield. However, only 4% of patient charts reported a clinical probability of PE, and in most cases, the type of CDR used was not mentioned.7
Safety
A minority of studies evaluated the safety of the interventions.13,18,19,23,25,27 Only 2 of these
The 2 studies involving a control group did not find significant differences between the intervention and the control groups with respect to mortality, complications because of thromboembolic and bleeding events, or any other adverse event during the 3-months’ follow-up.13,27
Jiménez et al.19 reported less than 1% mortality following the implementation of a CDS (0.7%; 95% CI, 0.2%-1.1%). In their study assessing the impact of an educational intervention, Kline et al.23 (2004) observed that none of the patients discharged with a fully negative Charlotte rule died suddenly and unexpectedly at 90-day follow-up. However, another educational intervention aimed at reducing ED patients’ radiation exposure observed a significant increase in the 90-day all-cause mortality of patients with negative CTPA, which was associated with a decline in the 90-day mortality of patients with negative ventilation/perfusion (V/Q) scanning.25
Jiménez et al.19 observed an absolute decrease of 2.5% in the incidence of symptomatic VTE events after the intervention (95% CI, 0.9%-4.6%; P < .01). The occurrence of VTE events, including PE, reached 1% in Goergen et al.18 and 3.9% in Kline et al.23 (2004) during follow-up.
Economic Aspects
Kline et al.13 (2014) found a significant decrease in charges and estimated costs for medical care within 90 days of initial ED presentation in the patients who were investigated with CTPA in the intervention group. The median costs of medical care within 30 days of the initial ED presentation were US $1274 in the control group and US $934 in the intervention group (P = .018).13 The median charges of medical care within 30 days of the initial ED presentation were US $7595 in the control group and US $6281 in the intervention group (P = .004).13
Facilitators and Barriers
Only 1 study appraised the reasons given by emergency physicians for not adhering to CDS recommendations.16 The reason most often given was the time needed to access and use the application, which was perceived as having a negative impact on productivity as well as a preference for intuitive clinical judgment.16 Though not the result of specific evaluation or data collection, some authors commented on the factors that may facilitate or impede the implementation of interventions to diminish the inappropriate use.14,20 Kanaan et al.20 proposed that factors other than the knowledge of current clinical guidelines may explain CTPA use. Booker and Johnson26 suggested that the demand for rapid turnover in the ED may lead to “so-called ‘blanket ordering’, which attempts to reach diagnosis as quickly as possible despite cost and patient safety.” Raja et al.14 (2015) suggested that the unambiguous representation of guidelines based on validated, high-quality evidence in the CDS may have improved physician adoption in their study.
DISCUSSION
Efficacy
Baseline values for the use of imaging and diagnostic yield show important variation, especially when compared with the study performed in Europe.19 In general, only a modest impact is measured with regard to a decrease in the use of imaging, an increase in diagnostic use, and adherence to validated CDRs.
Among the interventions appraised, CDS was evaluated in the largest number of included studies, and its
The impact of CDS on diagnostic yield was mixed because 3 studies observed an increase in diagnostic yield postintervention,16,21,22 and 3 others monitored no significant impact.19,24,28 Adherence to guidelines or a quality measure was assessed in 2 studies, which reported a significant increase in appropriate ordering.17,24 Raja et al.24 (2014) observed an 18.7% increase in appropriate ordering after the implementation of a CDS from 56.9% to 75.6% (P < .01). Geeting et al.17 observed a similar increase, with appropriate ordering increasing from 58% to 76% over the duration of the intervention. However, this increase in appropriate use was not associated with a variation in CTPA use or diagnostic yield, which leads the investigators to posit that the physicians gradually inflated the Wells score they keyed into the CDS despite that no threshold Wells score was required to perform a CTPA.17
Raja et al.14 (2015) demonstrated that the implementation of performance feedback reporting, in addition to a CDS, can significantly increase adherence to CDR for the evaluation of PE in the ED. Additional studies would help to better understand the potential impact of such reports on CTPA use in the diagnostic workup of PE. However, it suggests that a combination of interventions, including the implementation of a CDS, performance feedback reporting, and well-designed and specific educational interventions, may have a more significant impact than any of these types of interventions taken separately.
The impact of the educational interventions appraised in this review on the expected results is mixed, though it is difficult to compare the observed results and draw conclusive remarks, as the characteristics of the interventions and study designs are different from each other.
Safety
There is limited evidence on the safety of appraised interventions. Only 6 studies appraised venous thrombolic events or mortality.13,18,19,23,25,27 However, no adverse events were noted in those studies evaluating possible complications or missed diagnoses. Additional research is needed to confirm the safety of the interventions appraised in this systematic review.
Facilitators and Barriers
There are significant limitations with respect to the analysis of the factors that favor or impede the implementation of the interventions appraised in this review. However, 2 studies that did not meet the inclusion criteria appraised physicians’ perceptions and attitudes toward prescribing imaging tests in the diagnostic workup of PE.31,32 One is Swiss31 and the other is Canadian.32 Both were conducted in the ED of academic hospitals. Rohacek et al.31 observed that defensive behaviors, such as “fear of missing PE,” were frequent and associated with a lower probability of a positive CTPA (OR = 0.36; 95% CI, 0.14-0.92). Ahn et al.32 concluded that, although ED physicians who participated in their survey possessed limited knowledge of radiation doses of CTPA and V/Q scans, they opted for V/Q scans that emit lower radiation doses in younger patients, especially females, which may reflect efforts done in the study setting to reduce patients’ radiation exposure.
There is not enough data to conclude on safety and the impact on healthcare costs.
Implications for Future Research
Future controlled studies of high methodological quality would help to better understand the effects associated with the implementation of the interventions aimed at reducing the inappropriate use of imaging in the diagnostic workup of PE. Efficacy results show that the success of the implementation of the various types of interventions is variable. This variation may be at least partly attributable to contextual factors, such as the external environment, the organizational leadership and culture, or the microsystem, such as differences in care patterns.33-35 The impact of context factors on the effectiveness of the interventions should be assessed further with appropriate tools.33,34,36
CONCLUSION
The joint use of CDS and PFRs appears more effective than the other types of intervention in reducing the inappropriate use of CTPA. However, an approach combining these with well-designed educational interventions as well as policies may be even more effective.
Future studies of high methodological quality would strengthen the evidence concerning the relative efficacy and safety of the interventions appraised, especially when various types are combined. Future research should also aim at bringing answers to the knowledge gaps related to the factors of success and barriers associated with the implementation of the interventions.
Disclosure
The authors report no conflict of interest.
1. Smith-Bindman R, Miglioretti DL, Johnson E, et al. Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. JAMA. 2012;307(22):2400-2409. PubMed
2. Canadian Institute for Health Information (CIHI). Medical Imaging in Canada 2012. https://www.cihi.ca/en/mit_summary_2012_en.pdf. Accessed December 14, 2016.
3. Wiener RS, Schwartz LM, Woloshin S. When a test is too good: how CT pulmonary angiograms find pulmonary emboli that do not need to be found. BMJ. 2013;347:f3368. doi:10.1136/bmj.f3368. PubMed
4. Schissler AJ, Rozenshtein A, Schluger NW, Einstein AJ. National trends in emergency room diagnosis of pulmonary embolism, 2001-2010: a cross-sectional study. Respir Res. 2015;16:44-50. PubMed
5. Minges KE, Bikdeli B, Wang Y, et al. National Trends in Pulmonary Embolism Hospitalization Rates and Outcomes for Adults Aged >/=65 Years in the United States (1999 to 2010). Am J Cardiol. 2015;116(9):1436-1442. PubMed
6. Duriseti RS, Brandeau ML. Cost-effectiveness of strategies for diagnosing pulmonary embolism among emergency department patients presenting with undifferentiated symptoms. Ann Emerg Med. 2010;56(4):321-332.e310. PubMed
7. Char S, Yoon HC. Improving appropriate use of pulmonary computed tomography angiography by increasing the serum D-dimer threshold and assessing clinical probability. Perm J. 2014;18(4):10-15. PubMed
8. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMed
9. Champagne F, Brousselle A, Contendriopoulos AP, Hartz Z. L’analyse des effets. In: Brousselle A, Champagne F, Contandriopoulos AP, Hartz Z, editors. L’évaluation: Concepts et Méthodes 2e Edition. Montréal: Les Presses de l’Université de Montréal; 2011: 173-198.
10. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-1012. PubMed
11. Popay J, Roberts H, Sowden A, et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Manchester, UK: ESRC Methods Programme; 2006.
12. Velasco M, Perleth M, Drummond M, et al. Best practice in undertaking and reporting health technology assessments. Working group 4 report. Int J Technol Assess Health Care. 2002;18(2):361-422. PubMed
13. Kline JA, Jones AE, Shapiro NI, et al. Multicenter, randomized trial of quantitative pretest probability to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Circ Cardiovasc Imaging. 2014;7(1):66-73. PubMed
14. Raja AS, Ip IK, Dunne RM, Schuur JD, Mills AM, Khorasani R. Effects of Performance Feedback Reports on Adherence to Evidence-Based Guidelines in Use of CT for Evaluation of Pulmonary Embolism in the Emergency Department: A Randomized Trial. AJR Am J Roentgenol. 2015;205(5):936-940. PubMed
15. Agarwal A, Persaud J, Grabinski R, Rabinowitz D, Bremner A, Mendelson R. Pulmonary embolism: are we there yet? J Med Imaging Radiat Oncol. 2012;56(3):270-281. PubMed
16. Drescher FS, Chandrika S, Weir ID, et al. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med. 2011;57(6):613-621. PubMed
17. Geeting GK, Beck M, Bruno MA, et al. Mandatory Assignment of Modified Wells Score Before CT Angiography for Pulmonary Embolism Fails to Improve Utilization or Percentage of Positive Cases. AJR Am J Roentgenol. 2016;207(2):442-449. PubMed
18. Goergen SK, Chan T, de Campo JF, et al. Reducing the use of diagnostic imaging in patients with suspected pulmonary embolism: validation of a risk assessment strategy. Emerg Med Australas. 2005;17(1):16-23. PubMed
19. Jiménez D, Resano S, Otero R, et al. Computerised clinical decision support for suspected PE. Thorax. 2015;70(9):909-911. PubMed
20. Kanaan Y, Knoepp UD, Kelly AM. The influence of education on appropriateness rates for CT pulmonary angiography in emergency department patients. Acad Radiol. 2013;20(9):1107-1114. PubMed
21. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
22. Raja AS, Ip IK, Prevedello LM, et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012;262(2):468-474. PubMed
23. Kline JA, Webb WB, Jones AE, Hernandez-Nino J. Impact of a rapid rule-out protocol for pulmonary embolism on the rate of screening, missed cases, and pulmonary vascular imaging in an urban US emergency department. Ann Emerg Med. 2004;44(5):490-502. PubMed
24. Raja AS, Gupta A, Ip IK, Mills AM, Khorasani R. The use of decision support to measure documented adherence to a national imaging quality measure. Acad Radiol. 2014;21(3):378-383. PubMed
25. Stein EG, Haramati LB, Chamarthy M, Sprayregen S, Davitt MM, Freeman LM. Success of a safe and simple algorithm to reduce use of CT pulmonary angiography in the emergency department. AJR Am J Roentgenol. 2010;194(2):392-397. PubMed
26. Booker MT, Johnson JO. Optimizing CT Pulmonary Angiogram Utilization in a Community Emergency Department: A Pre- and Postintervention Study. J Am Coll Radiol. 2017;14(1):65-71. PubMed
27. Goldstein NM, Kollef MH, Ward S, Gage BF. The impact of the introduction of a rapid D-dimer assay on the diagnostic evaluation of suspected pulmonary embolism. Arch Intern Med. 2001;161(4):567-571. PubMed
28. Dunne RM, Ip IK, Abbett S, et al. Effect of Evidence-based Clinical Decision Support on the Use and Yield of CT Pulmonary Angiographic Imaging in Hospitalized Patients. Radiology. 2015;276(1):167-174. PubMed
29. Wang RC, Bent S, Weber E, Neilson J, Smith-Bindman R, Fahimi J. The Impact of Clinical Decision Rules on Computed Tomography Use and Yield for Pulmonary Embolism: A Systematic Review and Meta-analysis. Ann Emerg Med. 2016;67(6):693-701. PubMed
30. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
31. Rohacek M, Buatsi J, Szucs-Farkas Z, et al. Ordering CT pulmonary angiography to exclude pulmonary embolism: defense versus evidence in the emergency room. Intensive Care Med. 2012;38(8):1345-1351. PubMed
32. Ahn JS, Edmonds ML, McLeod SL, Dreyer JF. Familiarity with radiation exposure dose from diagnostic imaging for acute pulmonary embolism and current patterns of practice. CJEM. 2014;16(5):393-404. PubMed
33. Kringos DS, Sunol R, Wagner C, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res. 2015;15(277):015-0906. PubMed
34. Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q. 2010;88(4):500-559. PubMed
35. Pernod G, Caterino J, Maignan M, Tissier C, Kassis J, Lazarchick J. D-dimer use and pulmonary embolism diagnosis in emergency units: Why is there such a difference in pulmonary embolism prevalence between the United States of America and countries outside USA? PLoS ONE. 2017;12(1):e0169268. doi:10.1371/journal.pone.0169268 PubMed
36. Saillour-Glenisson F, Domecq S, Kret M, Sibe M, Dumond JP, Michel P. Design and validation of a questionnaire to assess organizational culture in French hospital wards. BMC Health Serv Res. 2016;16:491-503. PubMed
37. Kline JA, Nelson RD, Jackson RE, Courtney DM. Criteria for the safe use of D-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med. 2002;39(2):144-152. PubMed
38. Stein PD, Fowler SE, Goodman LR, et al. Multidetector computed tomography for acute pulmonary embolism. New Engl J Med. 2006;354(22):2317-2327. PubMed
39. Stein PD, Woodard PK, Weg JG, et al. Diagnostic pathways in acute pulmonary embolism: recommendations of the PIOPED II investigators. Am J Med. 2006;119(12):1048-1055. PubMed
40. Torbicki A, Perrier A, Konstantinides S, et al. Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J. 2008;29(18):2276-2315. PubMed
1. Smith-Bindman R, Miglioretti DL, Johnson E, et al. Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. JAMA. 2012;307(22):2400-2409. PubMed
2. Canadian Institute for Health Information (CIHI). Medical Imaging in Canada 2012. https://www.cihi.ca/en/mit_summary_2012_en.pdf. Accessed December 14, 2016.
3. Wiener RS, Schwartz LM, Woloshin S. When a test is too good: how CT pulmonary angiograms find pulmonary emboli that do not need to be found. BMJ. 2013;347:f3368. doi:10.1136/bmj.f3368. PubMed
4. Schissler AJ, Rozenshtein A, Schluger NW, Einstein AJ. National trends in emergency room diagnosis of pulmonary embolism, 2001-2010: a cross-sectional study. Respir Res. 2015;16:44-50. PubMed
5. Minges KE, Bikdeli B, Wang Y, et al. National Trends in Pulmonary Embolism Hospitalization Rates and Outcomes for Adults Aged >/=65 Years in the United States (1999 to 2010). Am J Cardiol. 2015;116(9):1436-1442. PubMed
6. Duriseti RS, Brandeau ML. Cost-effectiveness of strategies for diagnosing pulmonary embolism among emergency department patients presenting with undifferentiated symptoms. Ann Emerg Med. 2010;56(4):321-332.e310. PubMed
7. Char S, Yoon HC. Improving appropriate use of pulmonary computed tomography angiography by increasing the serum D-dimer threshold and assessing clinical probability. Perm J. 2014;18(4):10-15. PubMed
8. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMed
9. Champagne F, Brousselle A, Contendriopoulos AP, Hartz Z. L’analyse des effets. In: Brousselle A, Champagne F, Contandriopoulos AP, Hartz Z, editors. L’évaluation: Concepts et Méthodes 2e Edition. Montréal: Les Presses de l’Université de Montréal; 2011: 173-198.
10. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006-1012. PubMed
11. Popay J, Roberts H, Sowden A, et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Manchester, UK: ESRC Methods Programme; 2006.
12. Velasco M, Perleth M, Drummond M, et al. Best practice in undertaking and reporting health technology assessments. Working group 4 report. Int J Technol Assess Health Care. 2002;18(2):361-422. PubMed
13. Kline JA, Jones AE, Shapiro NI, et al. Multicenter, randomized trial of quantitative pretest probability to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Circ Cardiovasc Imaging. 2014;7(1):66-73. PubMed
14. Raja AS, Ip IK, Dunne RM, Schuur JD, Mills AM, Khorasani R. Effects of Performance Feedback Reports on Adherence to Evidence-Based Guidelines in Use of CT for Evaluation of Pulmonary Embolism in the Emergency Department: A Randomized Trial. AJR Am J Roentgenol. 2015;205(5):936-940. PubMed
15. Agarwal A, Persaud J, Grabinski R, Rabinowitz D, Bremner A, Mendelson R. Pulmonary embolism: are we there yet? J Med Imaging Radiat Oncol. 2012;56(3):270-281. PubMed
16. Drescher FS, Chandrika S, Weir ID, et al. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med. 2011;57(6):613-621. PubMed
17. Geeting GK, Beck M, Bruno MA, et al. Mandatory Assignment of Modified Wells Score Before CT Angiography for Pulmonary Embolism Fails to Improve Utilization or Percentage of Positive Cases. AJR Am J Roentgenol. 2016;207(2):442-449. PubMed
18. Goergen SK, Chan T, de Campo JF, et al. Reducing the use of diagnostic imaging in patients with suspected pulmonary embolism: validation of a risk assessment strategy. Emerg Med Australas. 2005;17(1):16-23. PubMed
19. Jiménez D, Resano S, Otero R, et al. Computerised clinical decision support for suspected PE. Thorax. 2015;70(9):909-911. PubMed
20. Kanaan Y, Knoepp UD, Kelly AM. The influence of education on appropriateness rates for CT pulmonary angiography in emergency department patients. Acad Radiol. 2013;20(9):1107-1114. PubMed
21. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
22. Raja AS, Ip IK, Prevedello LM, et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012;262(2):468-474. PubMed
23. Kline JA, Webb WB, Jones AE, Hernandez-Nino J. Impact of a rapid rule-out protocol for pulmonary embolism on the rate of screening, missed cases, and pulmonary vascular imaging in an urban US emergency department. Ann Emerg Med. 2004;44(5):490-502. PubMed
24. Raja AS, Gupta A, Ip IK, Mills AM, Khorasani R. The use of decision support to measure documented adherence to a national imaging quality measure. Acad Radiol. 2014;21(3):378-383. PubMed
25. Stein EG, Haramati LB, Chamarthy M, Sprayregen S, Davitt MM, Freeman LM. Success of a safe and simple algorithm to reduce use of CT pulmonary angiography in the emergency department. AJR Am J Roentgenol. 2010;194(2):392-397. PubMed
26. Booker MT, Johnson JO. Optimizing CT Pulmonary Angiogram Utilization in a Community Emergency Department: A Pre- and Postintervention Study. J Am Coll Radiol. 2017;14(1):65-71. PubMed
27. Goldstein NM, Kollef MH, Ward S, Gage BF. The impact of the introduction of a rapid D-dimer assay on the diagnostic evaluation of suspected pulmonary embolism. Arch Intern Med. 2001;161(4):567-571. PubMed
28. Dunne RM, Ip IK, Abbett S, et al. Effect of Evidence-based Clinical Decision Support on the Use and Yield of CT Pulmonary Angiographic Imaging in Hospitalized Patients. Radiology. 2015;276(1):167-174. PubMed
29. Wang RC, Bent S, Weber E, Neilson J, Smith-Bindman R, Fahimi J. The Impact of Clinical Decision Rules on Computed Tomography Use and Yield for Pulmonary Embolism: A Systematic Review and Meta-analysis. Ann Emerg Med. 2016;67(6):693-701. PubMed
30. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975-981. PubMed
31. Rohacek M, Buatsi J, Szucs-Farkas Z, et al. Ordering CT pulmonary angiography to exclude pulmonary embolism: defense versus evidence in the emergency room. Intensive Care Med. 2012;38(8):1345-1351. PubMed
32. Ahn JS, Edmonds ML, McLeod SL, Dreyer JF. Familiarity with radiation exposure dose from diagnostic imaging for acute pulmonary embolism and current patterns of practice. CJEM. 2014;16(5):393-404. PubMed
33. Kringos DS, Sunol R, Wagner C, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res. 2015;15(277):015-0906. PubMed
34. Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q. 2010;88(4):500-559. PubMed
35. Pernod G, Caterino J, Maignan M, Tissier C, Kassis J, Lazarchick J. D-dimer use and pulmonary embolism diagnosis in emergency units: Why is there such a difference in pulmonary embolism prevalence between the United States of America and countries outside USA? PLoS ONE. 2017;12(1):e0169268. doi:10.1371/journal.pone.0169268 PubMed
36. Saillour-Glenisson F, Domecq S, Kret M, Sibe M, Dumond JP, Michel P. Design and validation of a questionnaire to assess organizational culture in French hospital wards. BMC Health Serv Res. 2016;16:491-503. PubMed
37. Kline JA, Nelson RD, Jackson RE, Courtney DM. Criteria for the safe use of D-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med. 2002;39(2):144-152. PubMed
38. Stein PD, Fowler SE, Goodman LR, et al. Multidetector computed tomography for acute pulmonary embolism. New Engl J Med. 2006;354(22):2317-2327. PubMed
39. Stein PD, Woodard PK, Weg JG, et al. Diagnostic pathways in acute pulmonary embolism: recommendations of the PIOPED II investigators. Am J Med. 2006;119(12):1048-1055. PubMed
40. Torbicki A, Perrier A, Konstantinides S, et al. Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J. 2008;29(18):2276-2315. PubMed
© 2018 Society of Hospital Medicine