Medication errors are an often preventable consequence of the medication use process. Multiple reviews, including the recent Preventing Medication Errors by the Institute of Medicine,1 have emphasized the need to curtail process‐related deficiencies in medication use. Among inpatients, medication errors account for about 20% of medical errors.2 Medication errors can occur at the point of prescribing, transcribing, dispensing, administration, and monitoring. Prescribing errors are the most common and account for 39% to 49% of medication errors among hospitalized patients.3, 4

The Joint Commission has mandated that healthcare institutions track and intervene within the medication use process to reduce errors.5 A number of complex and costly interventions have been forwarded with significant evidence bases, including computerized physician order entry (CPOE), clinical decision‐support systems, and pharmacist participation on rounding medical teams.1 However, little has been published on the effectiveness of providing education and feedback to institutional clinicians to reduce prescribing‐related errors.68

Providers often are already aware of classic medication order errors, but at most institutions, specific examples of prescribing errors are not regularly communicated from the pharmacy dispensary to prescribers. One exception occurs when a dispensary pharmacist contacts a prescriber to clarify an order. However, this usually takes place while that prescriber is focused on the care of another patient, and it is not a good educational setting to reduce future medication order errors.

We delivered a series of short educational sessions to internal medicine (IM) residents, providing repeated feedback on prescribing errors using examples specific to our institution. The sessions followed the effective pharmaceutical industry paradigms of detailing and repeatedly exposing physicians to an educational message to maintain its salience. This innovation report documents the effect of this pharmacist‐led program on prescribing errors made by IM residents.

METHODS

This study was deemed exempt by the institutional review board of the University of Toledo.

Educational Intervention

In July 2006, the Department of Internal Medicine required all IM residents to participate in a weekly 3‐hour didactic training seminar series. The educational intervention occurred longitudinally during a portion of that weekly seminar series. Attendance was mandatory, but the level of participation varied across individual residents. No formal assessment or quiz was used for each resident during these discussions.

The intervention had 2 phases and was designed and executed by the primary author within his roles as an IM clinical pharmacist and faculty member. Phase 1 was an initial hour‐long didactic lecture on prescribing errors at the beginning of November 2006. This lecture focused on definitions and categories of medical errors and medication errors, Institute of Medicine reports, Joint Commission medication management requirements, and institutional medication order policies.

Phase 2 included a number of short, biweekly follow‐up discussion sessions in November and December 2006 and thereafter was modified to monthly discussions from January to May 2007. Discussions specifically addressed prescribing errors identified by the medication safety officer and primary author during the previous month. Sessions were approximately 15 minutes long and followed a handout that highlighted specific and commonly seen prescribing errors within the facility (Figure 1). Within these discussions, the error subtype was identified, and suggestions for properly writing the order were given.

Figure 1

RESULTS

Forty‐two IM residents participated in this study. Prior to the educational intervention, prescribing errors affected 2.25% (861/38,275) of the institution's medication orders (Figure 2). Following phase 1 and early into phase 2 of the prescribing error education, the frequency dropped to 1.51% (P < 0.001); that is, there was a 33% decline from the baseline. During the remainder of the intervention period, the frequency of prescribing errors fluctuated but remained lower than that observed pre‐intervention (P < 0.001 for each pairwise comparison to the baseline). Post‐intervention, the frequency of prescribing errors rose to 2.33% and was similar to that observed at the baseline (P = 0.49).

Figure 2

DISCUSSION

An educational intervention that highlighted institution‐specific prescribing errors reduced such errors by 33% within the first month and resulted in a mean 26% reduction during the 6‐month intervention period. Without ongoing education, however, the frequency of prescribing errors returned to preintervention period levels.

Our findings compare favorably to results obtained by other more complex and costly methods used to reduce medication errors, namely, CPOE, clinical decision‐support software, and clinical pharmacists on medical rounds.1 For example, in 1 study, prescribing errors were reduced by 19% following the implementation of CPOE alone.9 In another report, CPOE with clinical decision‐support software led to a more dramatic reduction of 81%.10 Additionally, pharmacist involvement on medical rounds has reduced adverse drug events by 78%.11

The frequency of prescribing errors found in this study was similar to that found in previous literature, although the variation in the definitions limits this comparison somewhat.12 Interestingly, the frequency of errors increased as the overall number of medication orders for the facility increased (see the ratios in Figure 2). This suggests that errors may be more likely during busier time periods, which are defined by higher total order volumes. Others have made similar observations.13 On most occasions, the individual prescribing errors seemed obvious and most likely due to a physician's haste.

This study had some limitations, including its interrupted time series design, which limits the ability to define a causal relationship. However, a causal effect is suggested by the differences before and during intervention as well as the return to the preintervention error frequency after the intervention had concluded. Second, the reported frequencies represent all medication orders in the studied clinical areas, not only those orders written by medical residents who participated in the intervention, although they do account for a large portion of the prescribing at the study hospital. Third, we did not assess specific resident errors or compare changes in the types of errors over time. Fourth, generalizability is limited to IM residents at an academic institution. As trainees, the IM residents may have been both keener to participate in and more accessible for educational opportunities such as this study. Fifth, as noted previously, the IM residents in this study not only practiced in the inpatient areas but had outpatient clinic rotations as well. It is conceivable that the most error‐prone residents rotated on the inpatient units before and after the intervention period but not during it. This is not very likely but cannot be excluded.

CONCLUSIONS

Adverse drug events have an impact on patient safety and can commonly occur following prescribing errors. Therefore, reducing prescribing errors is extremely important. The longitudinal education of residents using a periodic educational intervention provides a successful and economically feasible prescribing error prevention strategy, although the effects are quickly reversed following cessation of the educational component. Therefore, supporting an ongoing commitment to trainee education and communication between pharmacy and prescribers about institution‐specific medication errors appears warranted.

Medication errors are an often preventable consequence of the medication use process. Multiple reviews, including the recent Preventing Medication Errors by the Institute of Medicine,1 have emphasized the need to curtail process‐related deficiencies in medication use. Among inpatients, medication errors account for about 20% of medical errors.2 Medication errors can occur at the point of prescribing, transcribing, dispensing, administration, and monitoring. Prescribing errors are the most common and account for 39% to 49% of medication errors among hospitalized patients.3, 4

The Joint Commission has mandated that healthcare institutions track and intervene within the medication use process to reduce errors.5 A number of complex and costly interventions have been forwarded with significant evidence bases, including computerized physician order entry (CPOE), clinical decision‐support systems, and pharmacist participation on rounding medical teams.1 However, little has been published on the effectiveness of providing education and feedback to institutional clinicians to reduce prescribing‐related errors.68

Providers often are already aware of classic medication order errors, but at most institutions, specific examples of prescribing errors are not regularly communicated from the pharmacy dispensary to prescribers. One exception occurs when a dispensary pharmacist contacts a prescriber to clarify an order. However, this usually takes place while that prescriber is focused on the care of another patient, and it is not a good educational setting to reduce future medication order errors.

We delivered a series of short educational sessions to internal medicine (IM) residents, providing repeated feedback on prescribing errors using examples specific to our institution. The sessions followed the effective pharmaceutical industry paradigms of detailing and repeatedly exposing physicians to an educational message to maintain its salience. This innovation report documents the effect of this pharmacist‐led program on prescribing errors made by IM residents.

METHODS

This study was deemed exempt by the institutional review board of the University of Toledo.

Educational Intervention

In July 2006, the Department of Internal Medicine required all IM residents to participate in a weekly 3‐hour didactic training seminar series. The educational intervention occurred longitudinally during a portion of that weekly seminar series. Attendance was mandatory, but the level of participation varied across individual residents. No formal assessment or quiz was used for each resident during these discussions.

The intervention had 2 phases and was designed and executed by the primary author within his roles as an IM clinical pharmacist and faculty member. Phase 1 was an initial hour‐long didactic lecture on prescribing errors at the beginning of November 2006. This lecture focused on definitions and categories of medical errors and medication errors, Institute of Medicine reports, Joint Commission medication management requirements, and institutional medication order policies.

Phase 2 included a number of short, biweekly follow‐up discussion sessions in November and December 2006 and thereafter was modified to monthly discussions from January to May 2007. Discussions specifically addressed prescribing errors identified by the medication safety officer and primary author during the previous month. Sessions were approximately 15 minutes long and followed a handout that highlighted specific and commonly seen prescribing errors within the facility (Figure 1). Within these discussions, the error subtype was identified, and suggestions for properly writing the order were given.

Figure 1

RESULTS

Forty‐two IM residents participated in this study. Prior to the educational intervention, prescribing errors affected 2.25% (861/38,275) of the institution's medication orders (Figure 2). Following phase 1 and early into phase 2 of the prescribing error education, the frequency dropped to 1.51% (P < 0.001); that is, there was a 33% decline from the baseline. During the remainder of the intervention period, the frequency of prescribing errors fluctuated but remained lower than that observed pre‐intervention (P < 0.001 for each pairwise comparison to the baseline). Post‐intervention, the frequency of prescribing errors rose to 2.33% and was similar to that observed at the baseline (P = 0.49).

Figure 2

DISCUSSION

An educational intervention that highlighted institution‐specific prescribing errors reduced such errors by 33% within the first month and resulted in a mean 26% reduction during the 6‐month intervention period. Without ongoing education, however, the frequency of prescribing errors returned to preintervention period levels.

Our findings compare favorably to results obtained by other more complex and costly methods used to reduce medication errors, namely, CPOE, clinical decision‐support software, and clinical pharmacists on medical rounds.1 For example, in 1 study, prescribing errors were reduced by 19% following the implementation of CPOE alone.9 In another report, CPOE with clinical decision‐support software led to a more dramatic reduction of 81%.10 Additionally, pharmacist involvement on medical rounds has reduced adverse drug events by 78%.11

The frequency of prescribing errors found in this study was similar to that found in previous literature, although the variation in the definitions limits this comparison somewhat.12 Interestingly, the frequency of errors increased as the overall number of medication orders for the facility increased (see the ratios in Figure 2). This suggests that errors may be more likely during busier time periods, which are defined by higher total order volumes. Others have made similar observations.13 On most occasions, the individual prescribing errors seemed obvious and most likely due to a physician's haste.

This study had some limitations, including its interrupted time series design, which limits the ability to define a causal relationship. However, a causal effect is suggested by the differences before and during intervention as well as the return to the preintervention error frequency after the intervention had concluded. Second, the reported frequencies represent all medication orders in the studied clinical areas, not only those orders written by medical residents who participated in the intervention, although they do account for a large portion of the prescribing at the study hospital. Third, we did not assess specific resident errors or compare changes in the types of errors over time. Fourth, generalizability is limited to IM residents at an academic institution. As trainees, the IM residents may have been both keener to participate in and more accessible for educational opportunities such as this study. Fifth, as noted previously, the IM residents in this study not only practiced in the inpatient areas but had outpatient clinic rotations as well. It is conceivable that the most error‐prone residents rotated on the inpatient units before and after the intervention period but not during it. This is not very likely but cannot be excluded.

CONCLUSIONS

Adverse drug events have an impact on patient safety and can commonly occur following prescribing errors. Therefore, reducing prescribing errors is extremely important. The longitudinal education of residents using a periodic educational intervention provides a successful and economically feasible prescribing error prevention strategy, although the effects are quickly reversed following cessation of the educational component. Therefore, supporting an ongoing commitment to trainee education and communication between pharmacy and prescribers about institution‐specific medication errors appears warranted.

Medication errors are an often preventable consequence of the medication use process. Multiple reviews, including the recent Preventing Medication Errors by the Institute of Medicine,1 have emphasized the need to curtail process‐related deficiencies in medication use. Among inpatients, medication errors account for about 20% of medical errors.2 Medication errors can occur at the point of prescribing, transcribing, dispensing, administration, and monitoring. Prescribing errors are the most common and account for 39% to 49% of medication errors among hospitalized patients.3, 4

The Joint Commission has mandated that healthcare institutions track and intervene within the medication use process to reduce errors.5 A number of complex and costly interventions have been forwarded with significant evidence bases, including computerized physician order entry (CPOE), clinical decision‐support systems, and pharmacist participation on rounding medical teams.1 However, little has been published on the effectiveness of providing education and feedback to institutional clinicians to reduce prescribing‐related errors.68

Providers often are already aware of classic medication order errors, but at most institutions, specific examples of prescribing errors are not regularly communicated from the pharmacy dispensary to prescribers. One exception occurs when a dispensary pharmacist contacts a prescriber to clarify an order. However, this usually takes place while that prescriber is focused on the care of another patient, and it is not a good educational setting to reduce future medication order errors.

We delivered a series of short educational sessions to internal medicine (IM) residents, providing repeated feedback on prescribing errors using examples specific to our institution. The sessions followed the effective pharmaceutical industry paradigms of detailing and repeatedly exposing physicians to an educational message to maintain its salience. This innovation report documents the effect of this pharmacist‐led program on prescribing errors made by IM residents.

METHODS

This study was deemed exempt by the institutional review board of the University of Toledo.

Educational Intervention

In July 2006, the Department of Internal Medicine required all IM residents to participate in a weekly 3‐hour didactic training seminar series. The educational intervention occurred longitudinally during a portion of that weekly seminar series. Attendance was mandatory, but the level of participation varied across individual residents. No formal assessment or quiz was used for each resident during these discussions.

The intervention had 2 phases and was designed and executed by the primary author within his roles as an IM clinical pharmacist and faculty member. Phase 1 was an initial hour‐long didactic lecture on prescribing errors at the beginning of November 2006. This lecture focused on definitions and categories of medical errors and medication errors, Institute of Medicine reports, Joint Commission medication management requirements, and institutional medication order policies.

Phase 2 included a number of short, biweekly follow‐up discussion sessions in November and December 2006 and thereafter was modified to monthly discussions from January to May 2007. Discussions specifically addressed prescribing errors identified by the medication safety officer and primary author during the previous month. Sessions were approximately 15 minutes long and followed a handout that highlighted specific and commonly seen prescribing errors within the facility (Figure 1). Within these discussions, the error subtype was identified, and suggestions for properly writing the order were given.

Figure 1

RESULTS

Forty‐two IM residents participated in this study. Prior to the educational intervention, prescribing errors affected 2.25% (861/38,275) of the institution's medication orders (Figure 2). Following phase 1 and early into phase 2 of the prescribing error education, the frequency dropped to 1.51% (P < 0.001); that is, there was a 33% decline from the baseline. During the remainder of the intervention period, the frequency of prescribing errors fluctuated but remained lower than that observed pre‐intervention (P < 0.001 for each pairwise comparison to the baseline). Post‐intervention, the frequency of prescribing errors rose to 2.33% and was similar to that observed at the baseline (P = 0.49).

Figure 2

DISCUSSION

An educational intervention that highlighted institution‐specific prescribing errors reduced such errors by 33% within the first month and resulted in a mean 26% reduction during the 6‐month intervention period. Without ongoing education, however, the frequency of prescribing errors returned to preintervention period levels.

Our findings compare favorably to results obtained by other more complex and costly methods used to reduce medication errors, namely, CPOE, clinical decision‐support software, and clinical pharmacists on medical rounds.1 For example, in 1 study, prescribing errors were reduced by 19% following the implementation of CPOE alone.9 In another report, CPOE with clinical decision‐support software led to a more dramatic reduction of 81%.10 Additionally, pharmacist involvement on medical rounds has reduced adverse drug events by 78%.11

The frequency of prescribing errors found in this study was similar to that found in previous literature, although the variation in the definitions limits this comparison somewhat.12 Interestingly, the frequency of errors increased as the overall number of medication orders for the facility increased (see the ratios in Figure 2). This suggests that errors may be more likely during busier time periods, which are defined by higher total order volumes. Others have made similar observations.13 On most occasions, the individual prescribing errors seemed obvious and most likely due to a physician's haste.

This study had some limitations, including its interrupted time series design, which limits the ability to define a causal relationship. However, a causal effect is suggested by the differences before and during intervention as well as the return to the preintervention error frequency after the intervention had concluded. Second, the reported frequencies represent all medication orders in the studied clinical areas, not only those orders written by medical residents who participated in the intervention, although they do account for a large portion of the prescribing at the study hospital. Third, we did not assess specific resident errors or compare changes in the types of errors over time. Fourth, generalizability is limited to IM residents at an academic institution. As trainees, the IM residents may have been both keener to participate in and more accessible for educational opportunities such as this study. Fifth, as noted previously, the IM residents in this study not only practiced in the inpatient areas but had outpatient clinic rotations as well. It is conceivable that the most error‐prone residents rotated on the inpatient units before and after the intervention period but not during it. This is not very likely but cannot be excluded.

CONCLUSIONS

Adverse drug events have an impact on patient safety and can commonly occur following prescribing errors. Therefore, reducing prescribing errors is extremely important. The longitudinal education of residents using a periodic educational intervention provides a successful and economically feasible prescribing error prevention strategy, although the effects are quickly reversed following cessation of the educational component. Therefore, supporting an ongoing commitment to trainee education and communication between pharmacy and prescribers about institution‐specific medication errors appears warranted.

In this issue of JHM, O'Connor et al.1 examine the impact of paper‐based admission order sets on several quality measures relevant to medical inpatients in a large community medical center, focusing the most attention on the use of venous thromboembolism (VTE) prophylaxis. Randomly selected medical admissions from 4 time periods were examined by chart review for use of the order set, and for the use of VTE prophylaxis (defined as either unfractionated heparin [UFH] 5,000 units subcutaneous [sc] twice daily [BID] or compression stockings). VTE prophylaxis was ordered in an abysmally low 10.9% of inpatients in the baseline period. In spite of the limitations inherent in a before and after study design and a failure to assess the appropriateness of VTE prophylaxis, VTE rates, or side effects, the authors present convincing evidence that improvement in VTE prophylaxis did occur. However, it was a very limited and suboptimal improvement. By the fourteenth and fifteenth month after order set introduction, only about one‐half of admissions used the order set, and even when the order set was used, only 44% had VTE prophylaxis ordered. The percent of patient‐days with pharmacologic VTE prophylaxis in medical inpatients improved after order set implementation, but remained very low, at 26%. Therefore, the key lessons to be learned from this study are likely derived from what went wrong, rather than what went right.

Why did VTE prophylaxis rates stay so low in the face of a multiyear effort? An examination of more successful efforts,25 recent reviews in the VTE and quality improvement literature,610 and the Society of Hospital Medicine VTE Prevention Collaborative experience reveals several principles for effective improvement that were not followed in this study.

A VTE PREVENTION ORDER SET SHOULD PROVIDE DECISION SUPPORT (NOT JUST A PROMPT)

A simple prompt for mechanical prophylaxis or for UFH 5,000 units sc BID was embedded into a voluntary order set in this effort. Mechanical prophylaxis, pharmacologic prophylaxis, and no prophylaxis were treated as equal options, even though most medical inpatients have significant VTE risk factors,11, 12 and in spite of strong evidence‐based recommendations12 relegating mechanical prophylaxis to an adjunctive role for pharmacologic prophylaxis (unless there are contraindications to pharmacologic prophylaxis). The authors point out that the way order sets are structured or introduced is important to ensure they achieve the desired changes in practice. I could not agree more, but, unfortunately, the structure of their order set only secured the desired change in 44% of patients, even if you count compression stockings as adequate prophylaxis. This relatively poor result should have sparked a redesign of the VTE prevention component of the order set.

A more effective order set would reflect an institutional VTE prevention protocol.6, 7, 9, 13 A VTE prevention protocol consists of a standardized VTE risk assessment and contraindications to pharmacologic or heparin prophylaxis, linked to a menu of appropriate VTE prophylaxis options for each level of risk.13 The best protocols provide decision support at the point of care,9, 13 and yet preserve the ability to customize care for special patient situations or circumstances.

Ease of use issues and the lack of prospectively validated models have hindered widespread adoption of VTE risk assessment protocols (especially the point‐based models),14 but a simpler and more streamlined approach has been validated by the UCSD Medical Center experience,2 and by the general success of similar protocols in diverse medical centers taking part in the Society of Hospital Medicine (SHM) VTE Prevention Collaborative. This simpler method generally places patients into 1 of 3 levels of VTE risk, can be completed in seconds, and has excellent interobserver agreement. Reinforcing the expectation that pharmacologic prophylaxis is desirable for most ill inpatients (unless there is a contraindication to it) is likely more important than the finer details of the risk assessment model.

PROTOCOLS AND ORDER SETS MUST REACH THE GREAT MAJORITY OF PATIENTS

Protocols and order sets that sit on the shelf do not benefit patients. An order set that is used for one‐half of the targeted population has no chance of promoting excellent adherence to a protocol, and protocols/order sets must be widely adapted to be effective.13 Institutional mandates for the use of preprinted (or computerized) orders can be a very effective strategy. If the order set is constructed properly, it is easy to use and can actually save clinician time, thereby promoting widespread use, in some cases even without such a mandate. The SHM VTE Prevention Collaborative generally endorses an institution‐wide protocol and order set module that covers a variety of patient populations. A plug and play modular order set design allows the VTE prevention order set to be incorporated into all appropriate admission and transfer orders, and lends itself well to paper or computerized order formats.

LAYER ON ADDITIONAL INTERVENTIONS TO ENHANCE THE POWER OF THE PROTOCOL

Skillful introduction of a good order set that reaches most patients has often yielded observed VTE prophylaxis rates of 75% to 80% in the SHM VTE Prevention Collaborative. To reach higher levels, a multifaceted approach using a variety of techniques has been an effective strategy in the literature24, 6, 7, 10, 13, 15 and in the Collaborative. Educational programs alone,4, 16, 17 while not generally sufficient to bring about reliable VTE prophylaxis, remain an important intervention that can foster a more enthusiastic and appropriate use of order sets and protocols. Periodic audit and feedback and computerized decision support can also be very effective,3, 1822 particularly when there is an institutional protocol to hold up as the defining standard for adequate prophylaxis. We favor a method that involves real‐time identification of outliers (i.e., patients without prophylaxis who have some VTE risk and no obvious contraindication to prophylaxis). This identification can be done manually, but automated reports are generally feasible and effective. A simple templated note or page from a nurse or pharmacist to the provider of an outlier patient can bolster VTE prevention rates to well over 95%.5 Fatigue from alerts is minimized if this strategy is deployed after substantial improvement in VTE prophylaxis rates has been achieved via a well‐implemented and uniformly‐utilized order set. Trending and discussing cases of hospital‐acquired VTE can also motivate medical staff and reduce resistance to standardization.2, 3

THE FOUNDATION FOR IMPROVEMENT MUST BE IN PLACE

To explain why they did not implement educational programs, guidelines, or provide feedback to providers on their performance, the authors cite a lack of resources common to community medical centers. Yet, they were able to achieve the most resource‐intensive and challenging component of a VTE prevention effort, data collection and analysis. While resources for quality improvement are indeed insufficient in many academic and community hospitals, suboptimal levels of improvement tend to reflect, as in this study, fundamental failures in approach or execution. In this case, the order set design and implementation issues outlined above do not require extensive resources. Moreover, the foundation for effective improvement must be in place to address these issues effectively. This foundation includes administrative buy‐in that VTE prevention is an institutional priority, a commitment to support standardization (even in the face of occasional medical staff resistance), and a willingness to examine and redesign processes.13 It is unclear whether the administration was convinced that the effort should be a priority or whether this improvement team reported results through appropriate medical staff committees. The key point, of course, is that a culture of shared purpose, cooperation, and high expectations between the medical staff and the administration is more important than extensive resources. The right foundational elements put most improvement resources within the grasp of most medical centers.

The authors present work that is praiseworthy in many respects, but their suboptimal levels of improvement should serve as a cautionary tale rather than a model for other centers. Core improvement principles are of key importance. A mere prompt to order VTE prophylaxis within voluntarily‐used order sets, without supporting strategies to enhance VTE prophylaxis, is a recipe for mediocre improvement. Far superior results have been demonstrated in both community and academic centers, even in environments with limited resources. A multifaceted approach, including order sets that promote VTE prophylaxis and provide decision support for the majority of the targeted population, proactive intervention applied to outliers who are not on appropriate prophylaxis, educational programs, and an institutional commitment to standardization, are the ingredients for excellent results.

In this issue of JHM, O'Connor et al.1 examine the impact of paper‐based admission order sets on several quality measures relevant to medical inpatients in a large community medical center, focusing the most attention on the use of venous thromboembolism (VTE) prophylaxis. Randomly selected medical admissions from 4 time periods were examined by chart review for use of the order set, and for the use of VTE prophylaxis (defined as either unfractionated heparin [UFH] 5,000 units subcutaneous [sc] twice daily [BID] or compression stockings). VTE prophylaxis was ordered in an abysmally low 10.9% of inpatients in the baseline period. In spite of the limitations inherent in a before and after study design and a failure to assess the appropriateness of VTE prophylaxis, VTE rates, or side effects, the authors present convincing evidence that improvement in VTE prophylaxis did occur. However, it was a very limited and suboptimal improvement. By the fourteenth and fifteenth month after order set introduction, only about one‐half of admissions used the order set, and even when the order set was used, only 44% had VTE prophylaxis ordered. The percent of patient‐days with pharmacologic VTE prophylaxis in medical inpatients improved after order set implementation, but remained very low, at 26%. Therefore, the key lessons to be learned from this study are likely derived from what went wrong, rather than what went right.

Why did VTE prophylaxis rates stay so low in the face of a multiyear effort? An examination of more successful efforts,25 recent reviews in the VTE and quality improvement literature,610 and the Society of Hospital Medicine VTE Prevention Collaborative experience reveals several principles for effective improvement that were not followed in this study.

A VTE PREVENTION ORDER SET SHOULD PROVIDE DECISION SUPPORT (NOT JUST A PROMPT)

A simple prompt for mechanical prophylaxis or for UFH 5,000 units sc BID was embedded into a voluntary order set in this effort. Mechanical prophylaxis, pharmacologic prophylaxis, and no prophylaxis were treated as equal options, even though most medical inpatients have significant VTE risk factors,11, 12 and in spite of strong evidence‐based recommendations12 relegating mechanical prophylaxis to an adjunctive role for pharmacologic prophylaxis (unless there are contraindications to pharmacologic prophylaxis). The authors point out that the way order sets are structured or introduced is important to ensure they achieve the desired changes in practice. I could not agree more, but, unfortunately, the structure of their order set only secured the desired change in 44% of patients, even if you count compression stockings as adequate prophylaxis. This relatively poor result should have sparked a redesign of the VTE prevention component of the order set.

A more effective order set would reflect an institutional VTE prevention protocol.6, 7, 9, 13 A VTE prevention protocol consists of a standardized VTE risk assessment and contraindications to pharmacologic or heparin prophylaxis, linked to a menu of appropriate VTE prophylaxis options for each level of risk.13 The best protocols provide decision support at the point of care,9, 13 and yet preserve the ability to customize care for special patient situations or circumstances.

Ease of use issues and the lack of prospectively validated models have hindered widespread adoption of VTE risk assessment protocols (especially the point‐based models),14 but a simpler and more streamlined approach has been validated by the UCSD Medical Center experience,2 and by the general success of similar protocols in diverse medical centers taking part in the Society of Hospital Medicine (SHM) VTE Prevention Collaborative. This simpler method generally places patients into 1 of 3 levels of VTE risk, can be completed in seconds, and has excellent interobserver agreement. Reinforcing the expectation that pharmacologic prophylaxis is desirable for most ill inpatients (unless there is a contraindication to it) is likely more important than the finer details of the risk assessment model.

PROTOCOLS AND ORDER SETS MUST REACH THE GREAT MAJORITY OF PATIENTS

Protocols and order sets that sit on the shelf do not benefit patients. An order set that is used for one‐half of the targeted population has no chance of promoting excellent adherence to a protocol, and protocols/order sets must be widely adapted to be effective.13 Institutional mandates for the use of preprinted (or computerized) orders can be a very effective strategy. If the order set is constructed properly, it is easy to use and can actually save clinician time, thereby promoting widespread use, in some cases even without such a mandate. The SHM VTE Prevention Collaborative generally endorses an institution‐wide protocol and order set module that covers a variety of patient populations. A plug and play modular order set design allows the VTE prevention order set to be incorporated into all appropriate admission and transfer orders, and lends itself well to paper or computerized order formats.

LAYER ON ADDITIONAL INTERVENTIONS TO ENHANCE THE POWER OF THE PROTOCOL

Skillful introduction of a good order set that reaches most patients has often yielded observed VTE prophylaxis rates of 75% to 80% in the SHM VTE Prevention Collaborative. To reach higher levels, a multifaceted approach using a variety of techniques has been an effective strategy in the literature24, 6, 7, 10, 13, 15 and in the Collaborative. Educational programs alone,4, 16, 17 while not generally sufficient to bring about reliable VTE prophylaxis, remain an important intervention that can foster a more enthusiastic and appropriate use of order sets and protocols. Periodic audit and feedback and computerized decision support can also be very effective,3, 1822 particularly when there is an institutional protocol to hold up as the defining standard for adequate prophylaxis. We favor a method that involves real‐time identification of outliers (i.e., patients without prophylaxis who have some VTE risk and no obvious contraindication to prophylaxis). This identification can be done manually, but automated reports are generally feasible and effective. A simple templated note or page from a nurse or pharmacist to the provider of an outlier patient can bolster VTE prevention rates to well over 95%.5 Fatigue from alerts is minimized if this strategy is deployed after substantial improvement in VTE prophylaxis rates has been achieved via a well‐implemented and uniformly‐utilized order set. Trending and discussing cases of hospital‐acquired VTE can also motivate medical staff and reduce resistance to standardization.2, 3

THE FOUNDATION FOR IMPROVEMENT MUST BE IN PLACE

To explain why they did not implement educational programs, guidelines, or provide feedback to providers on their performance, the authors cite a lack of resources common to community medical centers. Yet, they were able to achieve the most resource‐intensive and challenging component of a VTE prevention effort, data collection and analysis. While resources for quality improvement are indeed insufficient in many academic and community hospitals, suboptimal levels of improvement tend to reflect, as in this study, fundamental failures in approach or execution. In this case, the order set design and implementation issues outlined above do not require extensive resources. Moreover, the foundation for effective improvement must be in place to address these issues effectively. This foundation includes administrative buy‐in that VTE prevention is an institutional priority, a commitment to support standardization (even in the face of occasional medical staff resistance), and a willingness to examine and redesign processes.13 It is unclear whether the administration was convinced that the effort should be a priority or whether this improvement team reported results through appropriate medical staff committees. The key point, of course, is that a culture of shared purpose, cooperation, and high expectations between the medical staff and the administration is more important than extensive resources. The right foundational elements put most improvement resources within the grasp of most medical centers.

The authors present work that is praiseworthy in many respects, but their suboptimal levels of improvement should serve as a cautionary tale rather than a model for other centers. Core improvement principles are of key importance. A mere prompt to order VTE prophylaxis within voluntarily‐used order sets, without supporting strategies to enhance VTE prophylaxis, is a recipe for mediocre improvement. Far superior results have been demonstrated in both community and academic centers, even in environments with limited resources. A multifaceted approach, including order sets that promote VTE prophylaxis and provide decision support for the majority of the targeted population, proactive intervention applied to outliers who are not on appropriate prophylaxis, educational programs, and an institutional commitment to standardization, are the ingredients for excellent results.

In this issue of JHM, O'Connor et al.1 examine the impact of paper‐based admission order sets on several quality measures relevant to medical inpatients in a large community medical center, focusing the most attention on the use of venous thromboembolism (VTE) prophylaxis. Randomly selected medical admissions from 4 time periods were examined by chart review for use of the order set, and for the use of VTE prophylaxis (defined as either unfractionated heparin [UFH] 5,000 units subcutaneous [sc] twice daily [BID] or compression stockings). VTE prophylaxis was ordered in an abysmally low 10.9% of inpatients in the baseline period. In spite of the limitations inherent in a before and after study design and a failure to assess the appropriateness of VTE prophylaxis, VTE rates, or side effects, the authors present convincing evidence that improvement in VTE prophylaxis did occur. However, it was a very limited and suboptimal improvement. By the fourteenth and fifteenth month after order set introduction, only about one‐half of admissions used the order set, and even when the order set was used, only 44% had VTE prophylaxis ordered. The percent of patient‐days with pharmacologic VTE prophylaxis in medical inpatients improved after order set implementation, but remained very low, at 26%. Therefore, the key lessons to be learned from this study are likely derived from what went wrong, rather than what went right.

Why did VTE prophylaxis rates stay so low in the face of a multiyear effort? An examination of more successful efforts,25 recent reviews in the VTE and quality improvement literature,610 and the Society of Hospital Medicine VTE Prevention Collaborative experience reveals several principles for effective improvement that were not followed in this study.

A VTE PREVENTION ORDER SET SHOULD PROVIDE DECISION SUPPORT (NOT JUST A PROMPT)

A simple prompt for mechanical prophylaxis or for UFH 5,000 units sc BID was embedded into a voluntary order set in this effort. Mechanical prophylaxis, pharmacologic prophylaxis, and no prophylaxis were treated as equal options, even though most medical inpatients have significant VTE risk factors,11, 12 and in spite of strong evidence‐based recommendations12 relegating mechanical prophylaxis to an adjunctive role for pharmacologic prophylaxis (unless there are contraindications to pharmacologic prophylaxis). The authors point out that the way order sets are structured or introduced is important to ensure they achieve the desired changes in practice. I could not agree more, but, unfortunately, the structure of their order set only secured the desired change in 44% of patients, even if you count compression stockings as adequate prophylaxis. This relatively poor result should have sparked a redesign of the VTE prevention component of the order set.

A more effective order set would reflect an institutional VTE prevention protocol.6, 7, 9, 13 A VTE prevention protocol consists of a standardized VTE risk assessment and contraindications to pharmacologic or heparin prophylaxis, linked to a menu of appropriate VTE prophylaxis options for each level of risk.13 The best protocols provide decision support at the point of care,9, 13 and yet preserve the ability to customize care for special patient situations or circumstances.

Ease of use issues and the lack of prospectively validated models have hindered widespread adoption of VTE risk assessment protocols (especially the point‐based models),14 but a simpler and more streamlined approach has been validated by the UCSD Medical Center experience,2 and by the general success of similar protocols in diverse medical centers taking part in the Society of Hospital Medicine (SHM) VTE Prevention Collaborative. This simpler method generally places patients into 1 of 3 levels of VTE risk, can be completed in seconds, and has excellent interobserver agreement. Reinforcing the expectation that pharmacologic prophylaxis is desirable for most ill inpatients (unless there is a contraindication to it) is likely more important than the finer details of the risk assessment model.

PROTOCOLS AND ORDER SETS MUST REACH THE GREAT MAJORITY OF PATIENTS

Protocols and order sets that sit on the shelf do not benefit patients. An order set that is used for one‐half of the targeted population has no chance of promoting excellent adherence to a protocol, and protocols/order sets must be widely adapted to be effective.13 Institutional mandates for the use of preprinted (or computerized) orders can be a very effective strategy. If the order set is constructed properly, it is easy to use and can actually save clinician time, thereby promoting widespread use, in some cases even without such a mandate. The SHM VTE Prevention Collaborative generally endorses an institution‐wide protocol and order set module that covers a variety of patient populations. A plug and play modular order set design allows the VTE prevention order set to be incorporated into all appropriate admission and transfer orders, and lends itself well to paper or computerized order formats.

LAYER ON ADDITIONAL INTERVENTIONS TO ENHANCE THE POWER OF THE PROTOCOL

Skillful introduction of a good order set that reaches most patients has often yielded observed VTE prophylaxis rates of 75% to 80% in the SHM VTE Prevention Collaborative. To reach higher levels, a multifaceted approach using a variety of techniques has been an effective strategy in the literature24, 6, 7, 10, 13, 15 and in the Collaborative. Educational programs alone,4, 16, 17 while not generally sufficient to bring about reliable VTE prophylaxis, remain an important intervention that can foster a more enthusiastic and appropriate use of order sets and protocols. Periodic audit and feedback and computerized decision support can also be very effective,3, 1822 particularly when there is an institutional protocol to hold up as the defining standard for adequate prophylaxis. We favor a method that involves real‐time identification of outliers (i.e., patients without prophylaxis who have some VTE risk and no obvious contraindication to prophylaxis). This identification can be done manually, but automated reports are generally feasible and effective. A simple templated note or page from a nurse or pharmacist to the provider of an outlier patient can bolster VTE prevention rates to well over 95%.5 Fatigue from alerts is minimized if this strategy is deployed after substantial improvement in VTE prophylaxis rates has been achieved via a well‐implemented and uniformly‐utilized order set. Trending and discussing cases of hospital‐acquired VTE can also motivate medical staff and reduce resistance to standardization.2, 3

THE FOUNDATION FOR IMPROVEMENT MUST BE IN PLACE

To explain why they did not implement educational programs, guidelines, or provide feedback to providers on their performance, the authors cite a lack of resources common to community medical centers. Yet, they were able to achieve the most resource‐intensive and challenging component of a VTE prevention effort, data collection and analysis. While resources for quality improvement are indeed insufficient in many academic and community hospitals, suboptimal levels of improvement tend to reflect, as in this study, fundamental failures in approach or execution. In this case, the order set design and implementation issues outlined above do not require extensive resources. Moreover, the foundation for effective improvement must be in place to address these issues effectively. This foundation includes administrative buy‐in that VTE prevention is an institutional priority, a commitment to support standardization (even in the face of occasional medical staff resistance), and a willingness to examine and redesign processes.13 It is unclear whether the administration was convinced that the effort should be a priority or whether this improvement team reported results through appropriate medical staff committees. The key point, of course, is that a culture of shared purpose, cooperation, and high expectations between the medical staff and the administration is more important than extensive resources. The right foundational elements put most improvement resources within the grasp of most medical centers.

The authors present work that is praiseworthy in many respects, but their suboptimal levels of improvement should serve as a cautionary tale rather than a model for other centers. Core improvement principles are of key importance. A mere prompt to order VTE prophylaxis within voluntarily‐used order sets, without supporting strategies to enhance VTE prophylaxis, is a recipe for mediocre improvement. Far superior results have been demonstrated in both community and academic centers, even in environments with limited resources. A multifaceted approach, including order sets that promote VTE prophylaxis and provide decision support for the majority of the targeted population, proactive intervention applied to outliers who are not on appropriate prophylaxis, educational programs, and an institutional commitment to standardization, are the ingredients for excellent results.

Blood cultures (BCs) have long been a mainstay of the diagnostic evaluation of patients hospitalized with community‐acquired pneumonia (CAP). They have been strongly recommended by professional societies13 and are often expected by admitting physicians. A large retrospective study of Medicare patients with pneumonia found that obtaining BCs is associated with lower mortality.4 In 2002, when the National Hospital Quality Measures (NHQM) were introduced, BCs were included as a quality measure for pneumonia.5, 6

However, there is uncertainty about the actual utility of BCs in CAP. In large studies they are true‐positive in only 7 to 11% of cases and false‐positive in 5%,2, 7 and whether they affect clinical management has been strongly questioned.810 Their impact may be limited by slow results, low frequency of bacterial resistance to the empiric antibiotic regimen, and reluctance of physicians to narrow antibiotic coverage.9, 11 Recent updates to professional society guidelines no longer recommend BCs in all admitted CAP patients.12

To evaluate the clinical utility of BCs and the appropriateness of pnemonia quality measures based on BCs, we performed a systematic review of the literature to determine the effect of BCs on the management of adults with CAP requiring hospitalization.

PATIENTS AND METHODS

DATA SYNTHESIS

Search Results

Our electronic database search yielded 3236 citations. From this list and the supplementary search of references, we reviewed 607 abstracts; of these, we selected 73 articles for full‐text review, and 15 were included in the final analysis (Figure 1). One study was narrowly excluded because it largely included CAP patients that had already been admitted to the hospital and failed an empiric antibiotic trial before BCs were obtained.13

Figure 1

DISCUSSION

Blood cultures (BCs) have long been a mainstay of the diagnostic evaluation of patients hospitalized with community‐acquired pneumonia (CAP). They have been strongly recommended by professional societies13 and are often expected by admitting physicians. A large retrospective study of Medicare patients with pneumonia found that obtaining BCs is associated with lower mortality.4 In 2002, when the National Hospital Quality Measures (NHQM) were introduced, BCs were included as a quality measure for pneumonia.5, 6

However, there is uncertainty about the actual utility of BCs in CAP. In large studies they are true‐positive in only 7 to 11% of cases and false‐positive in 5%,2, 7 and whether they affect clinical management has been strongly questioned.810 Their impact may be limited by slow results, low frequency of bacterial resistance to the empiric antibiotic regimen, and reluctance of physicians to narrow antibiotic coverage.9, 11 Recent updates to professional society guidelines no longer recommend BCs in all admitted CAP patients.12

To evaluate the clinical utility of BCs and the appropriateness of pnemonia quality measures based on BCs, we performed a systematic review of the literature to determine the effect of BCs on the management of adults with CAP requiring hospitalization.

PATIENTS AND METHODS

DATA SYNTHESIS

Search Results

Our electronic database search yielded 3236 citations. From this list and the supplementary search of references, we reviewed 607 abstracts; of these, we selected 73 articles for full‐text review, and 15 were included in the final analysis (Figure 1). One study was narrowly excluded because it largely included CAP patients that had already been admitted to the hospital and failed an empiric antibiotic trial before BCs were obtained.13

Figure 1

DISCUSSION

Blood cultures (BCs) have long been a mainstay of the diagnostic evaluation of patients hospitalized with community‐acquired pneumonia (CAP). They have been strongly recommended by professional societies13 and are often expected by admitting physicians. A large retrospective study of Medicare patients with pneumonia found that obtaining BCs is associated with lower mortality.4 In 2002, when the National Hospital Quality Measures (NHQM) were introduced, BCs were included as a quality measure for pneumonia.5, 6

However, there is uncertainty about the actual utility of BCs in CAP. In large studies they are true‐positive in only 7 to 11% of cases and false‐positive in 5%,2, 7 and whether they affect clinical management has been strongly questioned.810 Their impact may be limited by slow results, low frequency of bacterial resistance to the empiric antibiotic regimen, and reluctance of physicians to narrow antibiotic coverage.9, 11 Recent updates to professional society guidelines no longer recommend BCs in all admitted CAP patients.12

To evaluate the clinical utility of BCs and the appropriateness of pnemonia quality measures based on BCs, we performed a systematic review of the literature to determine the effect of BCs on the management of adults with CAP requiring hospitalization.

PATIENTS AND METHODS

DATA SYNTHESIS

Search Results

Our electronic database search yielded 3236 citations. From this list and the supplementary search of references, we reviewed 607 abstracts; of these, we selected 73 articles for full‐text review, and 15 were included in the final analysis (Figure 1). One study was narrowly excluded because it largely included CAP patients that had already been admitted to the hospital and failed an empiric antibiotic trial before BCs were obtained.13

Figure 1

DISCUSSION

Adverse drug eventsdefined as an injury resulting from medical intervention related to a drug1significantly contribute to health care expenditures. Over 770,000 people are injured or die every year in hospitals from adverse drug events, and national hospital expenses to treat patients who have suffered adverse drug events during hospitalization have been estimated to be between $1.56 and $4.2 billion annually.2 In a meta‐analysis of prospective studies, researchers found that adverse drug reactions, one important form of adverse drug events, may rank as the fourth to sixth leading cause of death in the United States, with more than 100,000 deaths per year.3 Understanding the factors associated with these adverse events may help in the development of prevention strategies, with resulting improving health care quality and lowering health care costs.

Among hospitalized patients, antibacterial adverse effects may account for approximately 25% of adverse drug reactions.1, 4 While the economic impact has been studied for overall adverse drug events in hospitalized patients in the 1990s, more recent detailed studies for the impact of antibiotic‐related adverse drug effects have not been published. As hospitalized patients with the primary diagnosis of pneumonia are invariably treated with antibiotics, and since pneumonia is the third leading cause for hospitalization in the United States,5 hospitalization databases that document pneumonia hospitalizations as well as adverse effects from antibiotics, using specific International Classification of Diseases, Ninth Revision (ICD‐9) clinical modification codes, constitute a unique and rich resource for quantifying and analyzing the incidence and impact of antibiotic‐associated adverse drug effects.

The purpose of this study was to describe the incidence and clinical manifestations of adverse drug effects in pneumonia hospitalizations in recent years, and to determine the types of patients and comorbidities, which are most commonly associated with adverse drug effects. The term adverse drug effect refers more to known side effects of medications, whereas adverse drug events and adverse drug reactions refer to an injury or a noxious, unintended, and undesired effect resulting from administration of a drug.6 As this study utilized medical coding for data abstraction, the broader classifications of adverse drug events or reactions could not be examined and instead the outcome of adverse drug effect was utilized.

METHODS

RESULTS

In the New York (SPARCS) database (NYS), 278,425 pneumonia admissions were identified. In HCUP‐NIS data subsets (NIS), 186,193 pneumonia admissions formed the cohort. In both cohorts, there was a predominance of females and older patients (Table 1). Diabetes and hypertension were common comorbidities. In the NYS cohort, 1,329 (0.48%) had an adverse effect related to an antibiotic or anti‐infective. In the NIS cohort, an estimated 0.53% had an adverse drug effect. There was a small but significant increase in the percentage of national hospitalizations associated with an antibiotic adverse drug effect over time (time effect significance; P = 0.0149; Table 1). However, this trend was not seen in the NYS cohort.

The most numerous adverse effects were noted in other specified antibiotics, followed by other unspecified antibiotics, then cephalosporins in both databases (Table 2). Cephalosporins accounted for 15% and 14% of cases with adverse drug effect due to antibiotics or anti‐infectives in the NYS and NIS cohorts, respectively. Adverse drug effects due to the penicillins and quinolones were similar in frequency and were the next most common identifiable classes of antibiotics with adverse drug effects after cephalosporins. Adverse effects to other specified antibiotics and unspecified antibiotics combined constituted 59% of adverse drug effects in both NYS and NIS cohorts.

Hospitalizations associated with an adverse drug effect had higher proportions of women than hospitalizations without an adverse drug effect in both the NIS (65% versus 54%) and NYS (62% versus 54%) databases. Hospitalizations associated with an adverse drug effect had a mean age that was about 1 year younger than that observed in hospitalizations without an adverse drug effect in both databases. Congestive heart failure was present in a lower proportion of hospitalizations associated with an adverse drug effect compared to hospitalizations without adverse drug effects (NYS 27% versus 30%, NIS 25% versus 29%). In the NIS database, adverse drug effect associated hospitalizations had a lower proportion of chronic obstructive pulmonary disease than other hospitalizations (32% versus 40%). Neither database showed any adverse drug effect associated disproportion with regard to hypertension and diabetes mellitus.

In logistic regression modeling, significant predictors for an adverse drug effect included non‐African American race, older age, female gender, not having Medicaid, and residence outside the greater NY area (only in the NYS data). Non‐African‐Americans were more likely than African‐Americans to have adverse drug effect admissions (adjusted odds ratio for NYS 2.2, 95% CI, 1.72.8; and for NIS 2.1, 95% CI, 1.63.0). Females were more likely than males to be associated with adverse drug effect admissions (adjusted odds ratio for NYS 1.5, 95% CI, 1.31.6; and for NIS 1.6, 95% CI, 1.41.8). In addition, residence outside the greater NY area was associated with adverse drug effect associated admissions (adjusted odds ratio 2.1, 95% CI, 1.82.3) in NYS data.

Skin and allergy manifestations potentially associated with adverse drug effects were reported in 34% and 43% of the NIS and NYS cohorts, respectively. In comparison, less than 1% of non‐adverse drug effect admissions had these manifestations (Table 3) in either cohort. In NYS, adverse drug effects due to sulfonamides had a slightly higher proportion skin/allergy manifestations when compared with other antibiotic classes (Table 2). In contrast, NIS estimates show that adverse effects due to cephalosporins had the highest proportion of skin/allergy manifestations (Table 2). Compared to adverse drug effects due to other specified antibiotics, erythromycin/macrolides were more likely to present with GI manifestations in both databases (Table 2). Dermatitis due to drugs taken internally was coded for in 34% (NYS) and 26% (NIS) of patients that experienced an adverse drug effect, making this condition the most common skin/allergy manifestation associated with an adverse drug effect (Table 3). This was followed in frequency by urticaria and pruritus. Diarrhea was also a common symptom related to adverse drug effects (Table 3). While 72% of adverse drug effects had either GI or skin/allergy manifestations in the NYS cohort, only 65% of the NIS cohort had these manifestations reported. No increase in mortality was observed in patients with adverse drug effects compared to those without adverse drug effects (data not shown).

Both databases showed that adverse drug effects affected both LOS and total charges (Table 4). In the NIS database, adjusted models showed that GI manifestations impacted hospital charges more than skin/allergy manifestations (Table 4). In both the NYS and NIS cohorts, the effect of adverse drug effects on hospital charges was attenuated after accounting for skin/allergy and GI manifestations. However, even after accounting for both manifestations, there still was a significant adverse drug effect influence on LOS. In the example patient, predicted excess hospitalization charges associated with the presence of an adverse drug effect was $1,243 and $3,373 for the NIS and NYS cohorts, while LOS increases associated with an adverse drug effect were about 1 day in both cohorts. Linear regression models, which included adjustment factors including comorbidities and demographic/financial factors, showed that the models accounted for 13% of the variance (R² values) in LOS and 40% in charges for the NYS but only 7% for LOS and 15% for charges for the NIS.

DISCUSSION

This study documents antibiotic related adverse drug effects as a predictable, but infrequent complication that occurs in adult patients hospitalized for pneumonia. While the incidence of antibiotic‐related adverse drug events has been calculated in both hospital2 and nursing home16 populations, these studies have not specified admissions that were associated with antibiotic use. Thus calculations of antibiotic adverse drug events for actual at‐risk patients (ie, those receiving antibiotics) are imprecise in these reports. In the present study, it is highly probable that nearly all admissions were associated with actual antibiotic administration. Thus, a rough incidence of an identifiable antibiotic adverse drug effect for adult pneumonia admissions can be expected to be roughly 0.5%. Of interest is the observation that the national incidence of this complication appears to be increasing slightly but steadily in recent years. This could be explained on the basis of increased coding slots or DRG creep17 in more recent years with the national cohort. Layde et al.12 utilized e‐codes to identify medical injury due to medications in Wisconsin hospital discharge data (excluding newborn delivery discharges), unselected for infectious diseases. They calculated an overall 0.5% incidence of antibiotic‐associated adverse effects. Since not all hospitalizations are associated with antibiotic administration, this would imply that the incidence of antibiotic associated adverse effects in hospitalized patients actually given antibiotics would be higher than 0.5%. The relatively low incidence of antibiotic‐associated adverse effects observed in the present study may relate to briefer and less complex hospitalizations for these patients compared with other patients treated with antibiotics. The use of ICD‐9 codes (including e‐codes) as flags of adverse drug events may also underestimate actual rates. In a study of Utah hospitalization discharges in 2001, Hougland et al.18 found that these flags had a sensitivity of 55% for confirmed adverse drug events of various categories. This contrasts with the 98.5% sensitivity of e‐codes to identify medical injury due to drugs, described in the Wisconsin hospital discharge data. The distinction between adverse drug effects, adverse drug reactions, and adverse drug events may be important in understanding these data. Adverse drug reactions have been defined as any noxious, unintended, and undesired effect of a drug, which occurs at doses used in humans for prophylaxis, diagnosis, or therapy and can be considered a subset of adverse drug events, which, unlike adverse drug reactions, may also be due to drug administration errors.6 The term adverse drug effect has been used more in pharmacology literature4 and in medical coding,18 and may refer more to known side effects of medications, whereas adverse drug events have been broadly defined as an injury resulting from administration of a drug.6 As this study utilized medical coding for data abstraction, we used the term adverse drug effect for all results.

Determining factors that influence hospital charges and length of stay are complex. Regional differences19 may be 1 factor, as suggested by difference in charges for the example admission models (Table 4). The multivariate analyses showed that nearly 40% of the variation in total New York State hospitalization charges could be explained on the basis of demographic, comorbidity factors, and between hospital variation, with an additional independent effect identifiable in the presence of an adverse drug effect to an antibiotic. The explained variance in the New York State hospitalization charges exceeds that observed in other published clinical predictor models of hospital charges based on statewide or province‐wide discharge databases.12, 20, 21 Although less variability was explained in national models, independent adverse drug effects influence on both total charge and length of stay were also observed. Higher charges with adverse drug effect associated admissions could be explained in part by increased illness severity, leading to more hospital days, and thus higher charges in per diem reimbursement schemes. In DRG‐based reimbursements, adverse drug effect presence in an admission, with its attendant increased length of stay, could be considered an outlier case, in which case an increased inpatient prospective payment system (IPPS) payment would be authorized in addition to the base payment.22

Because the impact of adverse drug effects on length of stay was still present even after controlling for both skin/allergy and GI manifestations, this may suggest that other factors relating to the adverse drug effects may have influenced LOS. These factors might include physician reluctance to discharge these patients or the influence of other organ systems not accounted for or reported in this study. The attenuation of the adverse drug effect's influence on hospital charge models by including possible clinical manifestations of adverse drug effects suggests that these manifestations were the main contributors to higher charges associated with adverse drug effect admissions. These disparate findings regarding adverse drug effects on LOS versus charges are consistent with the notion that hospital LOS and hospital charge are separate constructs, which may be associated with separate factors affecting these outcomes.23 The 13% variation in LOS explained in the New York state hospitalization regression models was similar to the 14% variation in LOS observed in clinical predictor models for congestive heart failure hospitalizations in New York state.24 Layde et al.12 found a 14.5% and 18.5% adjusted increase in charges and LOS associated with any medical injury in Wisconsin hospital discharges. The excess length of stay due to antibiotic related medical injury was 1.27 days. This magnitude of effect is comparable to that observed in the present study.

There are a number of limitations in this study related to coding practices and the retrospective nature of the investigation. Currently, there are no ICD‐9 e‐codes for adverse effects due to commonly used older antibiotics such as vancomycin, clindamycin, and metronidazole, or to the newer antibiotic classes. Since currently recommended treatments for community‐acquired pneumonia25 are among the specified drugs with adverse effect coding, and since other specified and unspecified anti‐infective drug‐associated (no drug names or categories provided) adverse effects were frequent, it suggests that infectious processes other than community‐acquired pneumonia were also being treated in many hospitalizations. Another limitation is that because the temporal sequence of events cannot be ascertained with this data, it is possible that an adverse drug effect was due to an antibiotic given prior to hospitalization. It has been suggested that onset of diagnosis or present on admission information be part of a new administrative data coding strategy which has been used in some states in the US.19 The adoption of ICD‐10 codes may allow for more specificity and detailing of adverse drug reactions using administrative data, as described for the United Kingdom by Waller et al.26 The actual incidence of hospitalization‐associated adverse drug effects could be underestimated if a significant number of adverse effects occurred after discharge as only in‐hospital events were recorded.

In summary, we found that although the incidence of adverse drug effects is small, there is a definite quantifiable impact of these adverse effects on LOS and hospital charges in patients hospitalized with pneumonia. To our knowledge, there have not been similar large‐scale database studies to evaluate the incidence and impact of adverse drug effects related to antibiotics in both national and statewide samples. These findings also have implications in studies of outcomes related to pneumonia hospitalizations.

Adverse drug eventsdefined as an injury resulting from medical intervention related to a drug1significantly contribute to health care expenditures. Over 770,000 people are injured or die every year in hospitals from adverse drug events, and national hospital expenses to treat patients who have suffered adverse drug events during hospitalization have been estimated to be between $1.56 and $4.2 billion annually.2 In a meta‐analysis of prospective studies, researchers found that adverse drug reactions, one important form of adverse drug events, may rank as the fourth to sixth leading cause of death in the United States, with more than 100,000 deaths per year.3 Understanding the factors associated with these adverse events may help in the development of prevention strategies, with resulting improving health care quality and lowering health care costs.

Among hospitalized patients, antibacterial adverse effects may account for approximately 25% of adverse drug reactions.1, 4 While the economic impact has been studied for overall adverse drug events in hospitalized patients in the 1990s, more recent detailed studies for the impact of antibiotic‐related adverse drug effects have not been published. As hospitalized patients with the primary diagnosis of pneumonia are invariably treated with antibiotics, and since pneumonia is the third leading cause for hospitalization in the United States,5 hospitalization databases that document pneumonia hospitalizations as well as adverse effects from antibiotics, using specific International Classification of Diseases, Ninth Revision (ICD‐9) clinical modification codes, constitute a unique and rich resource for quantifying and analyzing the incidence and impact of antibiotic‐associated adverse drug effects.

The purpose of this study was to describe the incidence and clinical manifestations of adverse drug effects in pneumonia hospitalizations in recent years, and to determine the types of patients and comorbidities, which are most commonly associated with adverse drug effects. The term adverse drug effect refers more to known side effects of medications, whereas adverse drug events and adverse drug reactions refer to an injury or a noxious, unintended, and undesired effect resulting from administration of a drug.6 As this study utilized medical coding for data abstraction, the broader classifications of adverse drug events or reactions could not be examined and instead the outcome of adverse drug effect was utilized.

METHODS

RESULTS

In the New York (SPARCS) database (NYS), 278,425 pneumonia admissions were identified. In HCUP‐NIS data subsets (NIS), 186,193 pneumonia admissions formed the cohort. In both cohorts, there was a predominance of females and older patients (Table 1). Diabetes and hypertension were common comorbidities. In the NYS cohort, 1,329 (0.48%) had an adverse effect related to an antibiotic or anti‐infective. In the NIS cohort, an estimated 0.53% had an adverse drug effect. There was a small but significant increase in the percentage of national hospitalizations associated with an antibiotic adverse drug effect over time (time effect significance; P = 0.0149; Table 1). However, this trend was not seen in the NYS cohort.

The most numerous adverse effects were noted in other specified antibiotics, followed by other unspecified antibiotics, then cephalosporins in both databases (Table 2). Cephalosporins accounted for 15% and 14% of cases with adverse drug effect due to antibiotics or anti‐infectives in the NYS and NIS cohorts, respectively. Adverse drug effects due to the penicillins and quinolones were similar in frequency and were the next most common identifiable classes of antibiotics with adverse drug effects after cephalosporins. Adverse effects to other specified antibiotics and unspecified antibiotics combined constituted 59% of adverse drug effects in both NYS and NIS cohorts.

Hospitalizations associated with an adverse drug effect had higher proportions of women than hospitalizations without an adverse drug effect in both the NIS (65% versus 54%) and NYS (62% versus 54%) databases. Hospitalizations associated with an adverse drug effect had a mean age that was about 1 year younger than that observed in hospitalizations without an adverse drug effect in both databases. Congestive heart failure was present in a lower proportion of hospitalizations associated with an adverse drug effect compared to hospitalizations without adverse drug effects (NYS 27% versus 30%, NIS 25% versus 29%). In the NIS database, adverse drug effect associated hospitalizations had a lower proportion of chronic obstructive pulmonary disease than other hospitalizations (32% versus 40%). Neither database showed any adverse drug effect associated disproportion with regard to hypertension and diabetes mellitus.

In logistic regression modeling, significant predictors for an adverse drug effect included non‐African American race, older age, female gender, not having Medicaid, and residence outside the greater NY area (only in the NYS data). Non‐African‐Americans were more likely than African‐Americans to have adverse drug effect admissions (adjusted odds ratio for NYS 2.2, 95% CI, 1.72.8; and for NIS 2.1, 95% CI, 1.63.0). Females were more likely than males to be associated with adverse drug effect admissions (adjusted odds ratio for NYS 1.5, 95% CI, 1.31.6; and for NIS 1.6, 95% CI, 1.41.8). In addition, residence outside the greater NY area was associated with adverse drug effect associated admissions (adjusted odds ratio 2.1, 95% CI, 1.82.3) in NYS data.

Skin and allergy manifestations potentially associated with adverse drug effects were reported in 34% and 43% of the NIS and NYS cohorts, respectively. In comparison, less than 1% of non‐adverse drug effect admissions had these manifestations (Table 3) in either cohort. In NYS, adverse drug effects due to sulfonamides had a slightly higher proportion skin/allergy manifestations when compared with other antibiotic classes (Table 2). In contrast, NIS estimates show that adverse effects due to cephalosporins had the highest proportion of skin/allergy manifestations (Table 2). Compared to adverse drug effects due to other specified antibiotics, erythromycin/macrolides were more likely to present with GI manifestations in both databases (Table 2). Dermatitis due to drugs taken internally was coded for in 34% (NYS) and 26% (NIS) of patients that experienced an adverse drug effect, making this condition the most common skin/allergy manifestation associated with an adverse drug effect (Table 3). This was followed in frequency by urticaria and pruritus. Diarrhea was also a common symptom related to adverse drug effects (Table 3). While 72% of adverse drug effects had either GI or skin/allergy manifestations in the NYS cohort, only 65% of the NIS cohort had these manifestations reported. No increase in mortality was observed in patients with adverse drug effects compared to those without adverse drug effects (data not shown).

Both databases showed that adverse drug effects affected both LOS and total charges (Table 4). In the NIS database, adjusted models showed that GI manifestations impacted hospital charges more than skin/allergy manifestations (Table 4). In both the NYS and NIS cohorts, the effect of adverse drug effects on hospital charges was attenuated after accounting for skin/allergy and GI manifestations. However, even after accounting for both manifestations, there still was a significant adverse drug effect influence on LOS. In the example patient, predicted excess hospitalization charges associated with the presence of an adverse drug effect was $1,243 and $3,373 for the NIS and NYS cohorts, while LOS increases associated with an adverse drug effect were about 1 day in both cohorts. Linear regression models, which included adjustment factors including comorbidities and demographic/financial factors, showed that the models accounted for 13% of the variance (R² values) in LOS and 40% in charges for the NYS but only 7% for LOS and 15% for charges for the NIS.

DISCUSSION

This study documents antibiotic related adverse drug effects as a predictable, but infrequent complication that occurs in adult patients hospitalized for pneumonia. While the incidence of antibiotic‐related adverse drug events has been calculated in both hospital2 and nursing home16 populations, these studies have not specified admissions that were associated with antibiotic use. Thus calculations of antibiotic adverse drug events for actual at‐risk patients (ie, those receiving antibiotics) are imprecise in these reports. In the present study, it is highly probable that nearly all admissions were associated with actual antibiotic administration. Thus, a rough incidence of an identifiable antibiotic adverse drug effect for adult pneumonia admissions can be expected to be roughly 0.5%. Of interest is the observation that the national incidence of this complication appears to be increasing slightly but steadily in recent years. This could be explained on the basis of increased coding slots or DRG creep17 in more recent years with the national cohort. Layde et al.12 utilized e‐codes to identify medical injury due to medications in Wisconsin hospital discharge data (excluding newborn delivery discharges), unselected for infectious diseases. They calculated an overall 0.5% incidence of antibiotic‐associated adverse effects. Since not all hospitalizations are associated with antibiotic administration, this would imply that the incidence of antibiotic associated adverse effects in hospitalized patients actually given antibiotics would be higher than 0.5%. The relatively low incidence of antibiotic‐associated adverse effects observed in the present study may relate to briefer and less complex hospitalizations for these patients compared with other patients treated with antibiotics. The use of ICD‐9 codes (including e‐codes) as flags of adverse drug events may also underestimate actual rates. In a study of Utah hospitalization discharges in 2001, Hougland et al.18 found that these flags had a sensitivity of 55% for confirmed adverse drug events of various categories. This contrasts with the 98.5% sensitivity of e‐codes to identify medical injury due to drugs, described in the Wisconsin hospital discharge data. The distinction between adverse drug effects, adverse drug reactions, and adverse drug events may be important in understanding these data. Adverse drug reactions have been defined as any noxious, unintended, and undesired effect of a drug, which occurs at doses used in humans for prophylaxis, diagnosis, or therapy and can be considered a subset of adverse drug events, which, unlike adverse drug reactions, may also be due to drug administration errors.6 The term adverse drug effect has been used more in pharmacology literature4 and in medical coding,18 and may refer more to known side effects of medications, whereas adverse drug events have been broadly defined as an injury resulting from administration of a drug.6 As this study utilized medical coding for data abstraction, we used the term adverse drug effect for all results.

Determining factors that influence hospital charges and length of stay are complex. Regional differences19 may be 1 factor, as suggested by difference in charges for the example admission models (Table 4). The multivariate analyses showed that nearly 40% of the variation in total New York State hospitalization charges could be explained on the basis of demographic, comorbidity factors, and between hospital variation, with an additional independent effect identifiable in the presence of an adverse drug effect to an antibiotic. The explained variance in the New York State hospitalization charges exceeds that observed in other published clinical predictor models of hospital charges based on statewide or province‐wide discharge databases.12, 20, 21 Although less variability was explained in national models, independent adverse drug effects influence on both total charge and length of stay were also observed. Higher charges with adverse drug effect associated admissions could be explained in part by increased illness severity, leading to more hospital days, and thus higher charges in per diem reimbursement schemes. In DRG‐based reimbursements, adverse drug effect presence in an admission, with its attendant increased length of stay, could be considered an outlier case, in which case an increased inpatient prospective payment system (IPPS) payment would be authorized in addition to the base payment.22

Because the impact of adverse drug effects on length of stay was still present even after controlling for both skin/allergy and GI manifestations, this may suggest that other factors relating to the adverse drug effects may have influenced LOS. These factors might include physician reluctance to discharge these patients or the influence of other organ systems not accounted for or reported in this study. The attenuation of the adverse drug effect's influence on hospital charge models by including possible clinical manifestations of adverse drug effects suggests that these manifestations were the main contributors to higher charges associated with adverse drug effect admissions. These disparate findings regarding adverse drug effects on LOS versus charges are consistent with the notion that hospital LOS and hospital charge are separate constructs, which may be associated with separate factors affecting these outcomes.23 The 13% variation in LOS explained in the New York state hospitalization regression models was similar to the 14% variation in LOS observed in clinical predictor models for congestive heart failure hospitalizations in New York state.24 Layde et al.12 found a 14.5% and 18.5% adjusted increase in charges and LOS associated with any medical injury in Wisconsin hospital discharges. The excess length of stay due to antibiotic related medical injury was 1.27 days. This magnitude of effect is comparable to that observed in the present study.

There are a number of limitations in this study related to coding practices and the retrospective nature of the investigation. Currently, there are no ICD‐9 e‐codes for adverse effects due to commonly used older antibiotics such as vancomycin, clindamycin, and metronidazole, or to the newer antibiotic classes. Since currently recommended treatments for community‐acquired pneumonia25 are among the specified drugs with adverse effect coding, and since other specified and unspecified anti‐infective drug‐associated (no drug names or categories provided) adverse effects were frequent, it suggests that infectious processes other than community‐acquired pneumonia were also being treated in many hospitalizations. Another limitation is that because the temporal sequence of events cannot be ascertained with this data, it is possible that an adverse drug effect was due to an antibiotic given prior to hospitalization. It has been suggested that onset of diagnosis or present on admission information be part of a new administrative data coding strategy which has been used in some states in the US.19 The adoption of ICD‐10 codes may allow for more specificity and detailing of adverse drug reactions using administrative data, as described for the United Kingdom by Waller et al.26 The actual incidence of hospitalization‐associated adverse drug effects could be underestimated if a significant number of adverse effects occurred after discharge as only in‐hospital events were recorded.

In summary, we found that although the incidence of adverse drug effects is small, there is a definite quantifiable impact of these adverse effects on LOS and hospital charges in patients hospitalized with pneumonia. To our knowledge, there have not been similar large‐scale database studies to evaluate the incidence and impact of adverse drug effects related to antibiotics in both national and statewide samples. These findings also have implications in studies of outcomes related to pneumonia hospitalizations.

Adverse drug eventsdefined as an injury resulting from medical intervention related to a drug1significantly contribute to health care expenditures. Over 770,000 people are injured or die every year in hospitals from adverse drug events, and national hospital expenses to treat patients who have suffered adverse drug events during hospitalization have been estimated to be between $1.56 and $4.2 billion annually.2 In a meta‐analysis of prospective studies, researchers found that adverse drug reactions, one important form of adverse drug events, may rank as the fourth to sixth leading cause of death in the United States, with more than 100,000 deaths per year.3 Understanding the factors associated with these adverse events may help in the development of prevention strategies, with resulting improving health care quality and lowering health care costs.

Among hospitalized patients, antibacterial adverse effects may account for approximately 25% of adverse drug reactions.1, 4 While the economic impact has been studied for overall adverse drug events in hospitalized patients in the 1990s, more recent detailed studies for the impact of antibiotic‐related adverse drug effects have not been published. As hospitalized patients with the primary diagnosis of pneumonia are invariably treated with antibiotics, and since pneumonia is the third leading cause for hospitalization in the United States,5 hospitalization databases that document pneumonia hospitalizations as well as adverse effects from antibiotics, using specific International Classification of Diseases, Ninth Revision (ICD‐9) clinical modification codes, constitute a unique and rich resource for quantifying and analyzing the incidence and impact of antibiotic‐associated adverse drug effects.

The purpose of this study was to describe the incidence and clinical manifestations of adverse drug effects in pneumonia hospitalizations in recent years, and to determine the types of patients and comorbidities, which are most commonly associated with adverse drug effects. The term adverse drug effect refers more to known side effects of medications, whereas adverse drug events and adverse drug reactions refer to an injury or a noxious, unintended, and undesired effect resulting from administration of a drug.6 As this study utilized medical coding for data abstraction, the broader classifications of adverse drug events or reactions could not be examined and instead the outcome of adverse drug effect was utilized.

METHODS

RESULTS

In the New York (SPARCS) database (NYS), 278,425 pneumonia admissions were identified. In HCUP‐NIS data subsets (NIS), 186,193 pneumonia admissions formed the cohort. In both cohorts, there was a predominance of females and older patients (Table 1). Diabetes and hypertension were common comorbidities. In the NYS cohort, 1,329 (0.48%) had an adverse effect related to an antibiotic or anti‐infective. In the NIS cohort, an estimated 0.53% had an adverse drug effect. There was a small but significant increase in the percentage of national hospitalizations associated with an antibiotic adverse drug effect over time (time effect significance; P = 0.0149; Table 1). However, this trend was not seen in the NYS cohort.

The most numerous adverse effects were noted in other specified antibiotics, followed by other unspecified antibiotics, then cephalosporins in both databases (Table 2). Cephalosporins accounted for 15% and 14% of cases with adverse drug effect due to antibiotics or anti‐infectives in the NYS and NIS cohorts, respectively. Adverse drug effects due to the penicillins and quinolones were similar in frequency and were the next most common identifiable classes of antibiotics with adverse drug effects after cephalosporins. Adverse effects to other specified antibiotics and unspecified antibiotics combined constituted 59% of adverse drug effects in both NYS and NIS cohorts.

Hospitalizations associated with an adverse drug effect had higher proportions of women than hospitalizations without an adverse drug effect in both the NIS (65% versus 54%) and NYS (62% versus 54%) databases. Hospitalizations associated with an adverse drug effect had a mean age that was about 1 year younger than that observed in hospitalizations without an adverse drug effect in both databases. Congestive heart failure was present in a lower proportion of hospitalizations associated with an adverse drug effect compared to hospitalizations without adverse drug effects (NYS 27% versus 30%, NIS 25% versus 29%). In the NIS database, adverse drug effect associated hospitalizations had a lower proportion of chronic obstructive pulmonary disease than other hospitalizations (32% versus 40%). Neither database showed any adverse drug effect associated disproportion with regard to hypertension and diabetes mellitus.

In logistic regression modeling, significant predictors for an adverse drug effect included non‐African American race, older age, female gender, not having Medicaid, and residence outside the greater NY area (only in the NYS data). Non‐African‐Americans were more likely than African‐Americans to have adverse drug effect admissions (adjusted odds ratio for NYS 2.2, 95% CI, 1.72.8; and for NIS 2.1, 95% CI, 1.63.0). Females were more likely than males to be associated with adverse drug effect admissions (adjusted odds ratio for NYS 1.5, 95% CI, 1.31.6; and for NIS 1.6, 95% CI, 1.41.8). In addition, residence outside the greater NY area was associated with adverse drug effect associated admissions (adjusted odds ratio 2.1, 95% CI, 1.82.3) in NYS data.

Skin and allergy manifestations potentially associated with adverse drug effects were reported in 34% and 43% of the NIS and NYS cohorts, respectively. In comparison, less than 1% of non‐adverse drug effect admissions had these manifestations (Table 3) in either cohort. In NYS, adverse drug effects due to sulfonamides had a slightly higher proportion skin/allergy manifestations when compared with other antibiotic classes (Table 2). In contrast, NIS estimates show that adverse effects due to cephalosporins had the highest proportion of skin/allergy manifestations (Table 2). Compared to adverse drug effects due to other specified antibiotics, erythromycin/macrolides were more likely to present with GI manifestations in both databases (Table 2). Dermatitis due to drugs taken internally was coded for in 34% (NYS) and 26% (NIS) of patients that experienced an adverse drug effect, making this condition the most common skin/allergy manifestation associated with an adverse drug effect (Table 3). This was followed in frequency by urticaria and pruritus. Diarrhea was also a common symptom related to adverse drug effects (Table 3). While 72% of adverse drug effects had either GI or skin/allergy manifestations in the NYS cohort, only 65% of the NIS cohort had these manifestations reported. No increase in mortality was observed in patients with adverse drug effects compared to those without adverse drug effects (data not shown).

Both databases showed that adverse drug effects affected both LOS and total charges (Table 4). In the NIS database, adjusted models showed that GI manifestations impacted hospital charges more than skin/allergy manifestations (Table 4). In both the NYS and NIS cohorts, the effect of adverse drug effects on hospital charges was attenuated after accounting for skin/allergy and GI manifestations. However, even after accounting for both manifestations, there still was a significant adverse drug effect influence on LOS. In the example patient, predicted excess hospitalization charges associated with the presence of an adverse drug effect was $1,243 and $3,373 for the NIS and NYS cohorts, while LOS increases associated with an adverse drug effect were about 1 day in both cohorts. Linear regression models, which included adjustment factors including comorbidities and demographic/financial factors, showed that the models accounted for 13% of the variance (R² values) in LOS and 40% in charges for the NYS but only 7% for LOS and 15% for charges for the NIS.