Yun Wang, PhD

Warfarin is 1 of the most common causes of adverse drug events, with hospitalized patients being particularly at risk compared to outpatients.[1] Despite the availability of new oral anticoagulants (NOACs), physicians commonly prescribe warfarin to hospitalized patients,[2] likely in part due to the greater difficulty in reversing NOACs compared to warfarin. Furthermore, uptake of the NOACs is likely to be slow in resource‐poor countries due to the lower cost of warfarin.[3] However, the narrow therapeutic index, frequent drug‐drug interactions, and patient variability in metabolism of warfarin makes management challenging.[4] Thus, warfarin remains a significant cause of adverse events in hospitalized patients, occurring in approximately 3% to 8% of exposed patients, depending on underlying condition.[2, 5]

An elevated international normalized ratio (INR) is a strong predictor of drug‐associated adverse events (patient harm). In a study employing 21 different electronic triggers to identify potential adverse events, an elevated INR had the highest yield for events associated with harm (96% of INRs >5.0 associated with harm).[6] Although pharmacist‐managed inpatient anticoagulation services have been shown to improve warfarin management,[7, 8] there are evidence gaps regarding the causes of warfarin‐related adverse events and practice changes that could decrease their frequency. Although overanticoagulation is a well‐known risk factor for warfarin‐related adverse events,[9, 10] there are few evidence‐based warfarin monitoring and dosing recommendations for hospitalized patients.[10] For example, the 2012 American College of Chest Physicians Antithrombotic Guidelines[11] provide a weak recommendation on initial dosing of warfarin, but no recommendations on how frequently to monitor the INR, or appropriate dosing responses to INR levels. Although many hospitals employ protocols that suggest daily INR monitoring until stable, there are no evidence‐based guidelines to support this practice.[12] Conversely, there are reports of flags to order an INR level that are not activated unless greater than 2[13] or 3 days[14] pass since the prior INR. Protocols from some major academic medical centers suggest that after a therapeutic INR is reached, INR levels can be measured intermittently, as infrequently as twice a week.[15, 16]

The 2015 Joint Commission anticoagulant‐focused National Patient Safety Goal[17] (initially issued in 2008) mandates the assessment of baseline coagulation status before starting warfarin, and warfarin dosing based on a current INR; however, current is not defined. Neither the extent to which the mandate for assessing baseline coagulation status is adhered to nor the relationship between this process of care and patient outcomes is known. The importance of adverse drug events associated with anticoagulants, included warfarin, was also recently highlighted in the 2014 federal National Action Plan for Adverse Drug Event Prevention. In this document, the prevention of adverse drug events associated with anticoagulants was 1 of the 3 areas selected for special national attention and action.[18]

The Medicare Patient Safety Monitoring System (MPSMS) is a national chart abstraction‐based system that includes 21 in‐hospital adverse event measures, including warfarin‐associated adverse drug events.[2] Because of the importance of warfarin‐associated bleeding in hospitalized patients, we analyzed MPSMS data to determine what factors related to INR monitoring practices place patients at risk for these events. We were particularly interested in determining if we could detect potentially modifiable predictors of overanticoagulation and warfarin‐associated adverse events.

METHODS

RESULTS

There were 130,828 patients included in the 2009 to 2013 MPSMS sample, of whom 19,445 (14.9%) received warfarin during their hospital stay and had at least 1 INR measurement. Among these patients, 5228 (26.9%) had no INR level above 1.5 and were excluded from further analysis, leaving 14,217 included patients. Of these patients, 1055 (7.4%) developed a warfarin‐associated adverse event. Table 1 demonstrates the baseline demographics and comorbidities of the included patients.

Warfarin was started on hospital day 1 for 6825 (48.0%) of 14,217 patients. Among these patients, 6539 (95.8%) had an INR measured within 1 calendar day. We were unable to determine how many patients who started warfarin later in their hospital stay had a baseline INR, as we did not capture INRs performed prior to the day that warfarin was initiated.

Supporting Table 1 in the online version of this article demonstrates the association between an INR 6.0 and the occurrence of warfarin‐associated adverse events. A maximum INR 6.0 occurred in 469 (3.3%) of the patients included in the study, and among those patients, 133 (28.4%) experienced a warfarin‐associated adverse event compared to 922 (6.7%) adverse events in the 13,748 patients who did not develop an INR 6.0 (P < 0.001).

Among 8529 patients who received warfarin for at least 3 days, beginning on the third day of warfarin, 1549 patients (18.2%) did not have INR measured at least once each day that they received warfarin. Table 2 demonstrates that patients who had 2 or more days on which the INR was not measured had higher rates of INR 6.0 than patients for whom the INR was measured daily. A similar association was seen for warfarin‐associated adverse events (Table 2).

Figure 1A demonstrates the association between the number of days without an INR measurement and the subsequent development of an INR 6.0 or a warfarin‐associated adverse event, adjusted for baseline patient characteristics, receipt of heparin and LMWH, and number of days on warfarin. Patients with 1 or more days without an INR measurement had higher risk‐adjusted ORs of a subsequent INR 6.0, although the difference was not statistically significant for surgical patients. The analysis results based on inverse propensity scoring are seen in Figure 1B. Cardiac and surgical patients with 2 or more days without an INR measurement were at higher risk of having a warfarin‐associated adverse event, whereas cardiac and pneumonia patients with 1 or more days without an INR measurement were at higher risk of developing an INR 6.0.

Figure 1

Supporting Table 2 in the online version of this article demonstrates the relationship between patient characteristics and the occurrence of an INR 6.0 or a warfarin‐related adverse event. The only characteristic that was associated with either of these outcomes for all 3 patient conditions was renal disease, which was positively associated with a warfarin‐associated adverse event. Warfarin use prior to arrival was associated with lower risks of both an INR 6.0 and a warfarin‐associated adverse event, except for among surgical patients. Supporting Table 3 in the online version of this article demonstrates the differences in patient characteristics between patients who had daily INR measurement and those who had at least 1 day without an INR measurement.

Figure 2 illustrates the relationship of the maximum INR to the prior 1‐day change in INR in 4963 patients whose INR on the day prior to the maximum INR was 2.0 to 3.5. When the increase in INR was <0.9, the risk of the next day's INR being 6.0 was 0.7%, and if the increase was 0.9, the risk was 5.2%. The risk of developing an INR 5.0 was 1.9% if the preceding day's INR increase was <0.9 and 15.3% if the prior day's INR rise was 0.9. Overall, 51% of INRs 6.0 and 55% of INRs 5.0 were immediately preceded by an INR increase of 0.9. The positive likelihood ratio (LR) for a 0.9 rise in INR predicting an INR of 6.0 was 4.2, and the positive LR was 4.9 for predicting an INR 5.0.

Figure 2

There was no decline in the frequency of warfarin use among the patients in the MPSMS sample during the study period (16.7% in 2009 and 17.3% in 2013).

DISCUSSION

We studied warfarin‐associated adverse events in a nationally representative study of patients who received warfarin while in an acute care hospital for a primary diagnosis of cardiac disease, pneumonia, or major surgery. Several findings resulted from our analysis. First, warfarin is still commonly prescribed to hospitalized patients and remains a frequent cause of adverse events; 7.4% of the 2009 to 2013 MPSMS population who received warfarin and had at least 1 INR >1.5 developed a warfarin‐associated adverse event.

Over 95% of patients who received warfarin on the day of hospital admission had an INR performed within 1 day. This is similar to the results from a 2006 single center study in which 95% of patients had an INR measured prior to their first dose of warfarin.[10] Since 2008, The Joint Commission's National Patient Safety Goal has required the assessment of coagulation status before starting warfarin.[17] The high level of adherence to this standard suggests that further attention to this process of care is unlikely to significantly improve patient safety.

We also found that the lack of daily INR measurements was associated with an increased risk of an INR 6.0 and warfarin‐associated adverse events in some patient populations. There is limited evidence addressing the appropriate frequency of INR measurement in hospitalized patients receiving warfarin. The Joint Commission National Patient Safety Goal requires use of a current INR to adjust this therapy, but provides no specifics.[17] Although some experts believe that INRs should be monitored daily in hospitalized patients, this does not appear to be uniformly accepted. In some reports, 2[13] or 3[14] consecutive days without the performance of an INR was required to activate a reminder. Protocols from some major teaching hospitals specify intermittent monitoring once the INR is therapeutic.[15, 16] Because our results suggest that lapses in INR measurement lead to overanticoagulation and warfarin‐related adverse events, it may be appropriate to measure INRs daily in most hospitalized patients receiving warfarin. This would be consistent with the many known causes of INR instability in patients admitted to the hospital, including drug‐drug interactions, hepatic dysfunction, and changes in volume of distribution, such that truly stable hospitalized patients are likely rare. Indeed, hospital admission is a well‐known predictor of instability of warfarin effect. [9] Although our results suggest that daily INR measurement is associated with a lower rate of overanticoagulation, future studies might better define lower risk patients for whom daily INR measurement would not be necessary.

A prior INR increase 0.9 in 1 day was associated with an increased risk of subsequent overanticoagulation. Although a rapidly rising INR is known to predict overanticoagulation[10, 14] we could find no evidence as to what specific rate of rise confers this risk. Our results suggest that use of a warfarin dosing protocol that considers both the absolute value of the INR and the rate of rise could reduce warfarin‐related adverse events.

There are important limitations of our study. We did not abstract warfarin dosages, which precluded study of the appropriateness of both initial warfarin dosing and adjustment of the warfarin dose based on INR results. MPSMS does not reliably capture antiplatelet agents or other agents that result in drug‐drug interactions with warfarin, such as antibiotics, so this factor could theoretically have confounded our results. Antibiotic use seems unlikely to be a major confounder, because patients with acute cardiovascular disease demonstrated a similar relationship between INR measurement and an INR 6.0 to that seen with pneumonia and surgical patients, despite the latter patients likely having greater antibiotics exposure. Furthermore, MPSMS does not capture indices of severity of illness, so other unmeasured confounders could have influenced our results. Although we have data for patients admitted to the hospital for only 4 conditions, these are conditions that represent approximately 22% of hospital admissions in the United States.[2] Strengths of our study include the nationally representative and randomly selected cases and use of data that were obtained from chart abstraction as opposed to administrative data. Through the use of centralized data abstraction, we avoided the potential bias introduced when hospitals self‐report adverse events.

In summary, in a national sample of patients admitted to the hospital for 4 common conditions, warfarin‐associated adverse events were detected in 7.4% of patients who received warfarin. Lack of daily INR measurement was associated with an increased risk of overanticoagulation and warfarin‐associated adverse events in certain patient populations. A 1‐day increase in the INR of 0.9 predicted subsequent overanticoagulation. These results provide actionable opportunities to improve safety in some hospitalized patients receiving warfarin.

Warfarin is 1 of the most common causes of adverse drug events, with hospitalized patients being particularly at risk compared to outpatients.[1] Despite the availability of new oral anticoagulants (NOACs), physicians commonly prescribe warfarin to hospitalized patients,[2] likely in part due to the greater difficulty in reversing NOACs compared to warfarin. Furthermore, uptake of the NOACs is likely to be slow in resource‐poor countries due to the lower cost of warfarin.[3] However, the narrow therapeutic index, frequent drug‐drug interactions, and patient variability in metabolism of warfarin makes management challenging.[4] Thus, warfarin remains a significant cause of adverse events in hospitalized patients, occurring in approximately 3% to 8% of exposed patients, depending on underlying condition.[2, 5]

An elevated international normalized ratio (INR) is a strong predictor of drug‐associated adverse events (patient harm). In a study employing 21 different electronic triggers to identify potential adverse events, an elevated INR had the highest yield for events associated with harm (96% of INRs >5.0 associated with harm).[6] Although pharmacist‐managed inpatient anticoagulation services have been shown to improve warfarin management,[7, 8] there are evidence gaps regarding the causes of warfarin‐related adverse events and practice changes that could decrease their frequency. Although overanticoagulation is a well‐known risk factor for warfarin‐related adverse events,[9, 10] there are few evidence‐based warfarin monitoring and dosing recommendations for hospitalized patients.[10] For example, the 2012 American College of Chest Physicians Antithrombotic Guidelines[11] provide a weak recommendation on initial dosing of warfarin, but no recommendations on how frequently to monitor the INR, or appropriate dosing responses to INR levels. Although many hospitals employ protocols that suggest daily INR monitoring until stable, there are no evidence‐based guidelines to support this practice.[12] Conversely, there are reports of flags to order an INR level that are not activated unless greater than 2[13] or 3 days[14] pass since the prior INR. Protocols from some major academic medical centers suggest that after a therapeutic INR is reached, INR levels can be measured intermittently, as infrequently as twice a week.[15, 16]

The 2015 Joint Commission anticoagulant‐focused National Patient Safety Goal[17] (initially issued in 2008) mandates the assessment of baseline coagulation status before starting warfarin, and warfarin dosing based on a current INR; however, current is not defined. Neither the extent to which the mandate for assessing baseline coagulation status is adhered to nor the relationship between this process of care and patient outcomes is known. The importance of adverse drug events associated with anticoagulants, included warfarin, was also recently highlighted in the 2014 federal National Action Plan for Adverse Drug Event Prevention. In this document, the prevention of adverse drug events associated with anticoagulants was 1 of the 3 areas selected for special national attention and action.[18]

The Medicare Patient Safety Monitoring System (MPSMS) is a national chart abstraction‐based system that includes 21 in‐hospital adverse event measures, including warfarin‐associated adverse drug events.[2] Because of the importance of warfarin‐associated bleeding in hospitalized patients, we analyzed MPSMS data to determine what factors related to INR monitoring practices place patients at risk for these events. We were particularly interested in determining if we could detect potentially modifiable predictors of overanticoagulation and warfarin‐associated adverse events.

METHODS

RESULTS

There were 130,828 patients included in the 2009 to 2013 MPSMS sample, of whom 19,445 (14.9%) received warfarin during their hospital stay and had at least 1 INR measurement. Among these patients, 5228 (26.9%) had no INR level above 1.5 and were excluded from further analysis, leaving 14,217 included patients. Of these patients, 1055 (7.4%) developed a warfarin‐associated adverse event. Table 1 demonstrates the baseline demographics and comorbidities of the included patients.

Warfarin was started on hospital day 1 for 6825 (48.0%) of 14,217 patients. Among these patients, 6539 (95.8%) had an INR measured within 1 calendar day. We were unable to determine how many patients who started warfarin later in their hospital stay had a baseline INR, as we did not capture INRs performed prior to the day that warfarin was initiated.

Supporting Table 1 in the online version of this article demonstrates the association between an INR 6.0 and the occurrence of warfarin‐associated adverse events. A maximum INR 6.0 occurred in 469 (3.3%) of the patients included in the study, and among those patients, 133 (28.4%) experienced a warfarin‐associated adverse event compared to 922 (6.7%) adverse events in the 13,748 patients who did not develop an INR 6.0 (P < 0.001).

Among 8529 patients who received warfarin for at least 3 days, beginning on the third day of warfarin, 1549 patients (18.2%) did not have INR measured at least once each day that they received warfarin. Table 2 demonstrates that patients who had 2 or more days on which the INR was not measured had higher rates of INR 6.0 than patients for whom the INR was measured daily. A similar association was seen for warfarin‐associated adverse events (Table 2).

Figure 1A demonstrates the association between the number of days without an INR measurement and the subsequent development of an INR 6.0 or a warfarin‐associated adverse event, adjusted for baseline patient characteristics, receipt of heparin and LMWH, and number of days on warfarin. Patients with 1 or more days without an INR measurement had higher risk‐adjusted ORs of a subsequent INR 6.0, although the difference was not statistically significant for surgical patients. The analysis results based on inverse propensity scoring are seen in Figure 1B. Cardiac and surgical patients with 2 or more days without an INR measurement were at higher risk of having a warfarin‐associated adverse event, whereas cardiac and pneumonia patients with 1 or more days without an INR measurement were at higher risk of developing an INR 6.0.

Figure 1

Supporting Table 2 in the online version of this article demonstrates the relationship between patient characteristics and the occurrence of an INR 6.0 or a warfarin‐related adverse event. The only characteristic that was associated with either of these outcomes for all 3 patient conditions was renal disease, which was positively associated with a warfarin‐associated adverse event. Warfarin use prior to arrival was associated with lower risks of both an INR 6.0 and a warfarin‐associated adverse event, except for among surgical patients. Supporting Table 3 in the online version of this article demonstrates the differences in patient characteristics between patients who had daily INR measurement and those who had at least 1 day without an INR measurement.

Figure 2 illustrates the relationship of the maximum INR to the prior 1‐day change in INR in 4963 patients whose INR on the day prior to the maximum INR was 2.0 to 3.5. When the increase in INR was <0.9, the risk of the next day's INR being 6.0 was 0.7%, and if the increase was 0.9, the risk was 5.2%. The risk of developing an INR 5.0 was 1.9% if the preceding day's INR increase was <0.9 and 15.3% if the prior day's INR rise was 0.9. Overall, 51% of INRs 6.0 and 55% of INRs 5.0 were immediately preceded by an INR increase of 0.9. The positive likelihood ratio (LR) for a 0.9 rise in INR predicting an INR of 6.0 was 4.2, and the positive LR was 4.9 for predicting an INR 5.0.

Figure 2

There was no decline in the frequency of warfarin use among the patients in the MPSMS sample during the study period (16.7% in 2009 and 17.3% in 2013).

DISCUSSION

We studied warfarin‐associated adverse events in a nationally representative study of patients who received warfarin while in an acute care hospital for a primary diagnosis of cardiac disease, pneumonia, or major surgery. Several findings resulted from our analysis. First, warfarin is still commonly prescribed to hospitalized patients and remains a frequent cause of adverse events; 7.4% of the 2009 to 2013 MPSMS population who received warfarin and had at least 1 INR >1.5 developed a warfarin‐associated adverse event.

Over 95% of patients who received warfarin on the day of hospital admission had an INR performed within 1 day. This is similar to the results from a 2006 single center study in which 95% of patients had an INR measured prior to their first dose of warfarin.[10] Since 2008, The Joint Commission's National Patient Safety Goal has required the assessment of coagulation status before starting warfarin.[17] The high level of adherence to this standard suggests that further attention to this process of care is unlikely to significantly improve patient safety.

We also found that the lack of daily INR measurements was associated with an increased risk of an INR 6.0 and warfarin‐associated adverse events in some patient populations. There is limited evidence addressing the appropriate frequency of INR measurement in hospitalized patients receiving warfarin. The Joint Commission National Patient Safety Goal requires use of a current INR to adjust this therapy, but provides no specifics.[17] Although some experts believe that INRs should be monitored daily in hospitalized patients, this does not appear to be uniformly accepted. In some reports, 2[13] or 3[14] consecutive days without the performance of an INR was required to activate a reminder. Protocols from some major teaching hospitals specify intermittent monitoring once the INR is therapeutic.[15, 16] Because our results suggest that lapses in INR measurement lead to overanticoagulation and warfarin‐related adverse events, it may be appropriate to measure INRs daily in most hospitalized patients receiving warfarin. This would be consistent with the many known causes of INR instability in patients admitted to the hospital, including drug‐drug interactions, hepatic dysfunction, and changes in volume of distribution, such that truly stable hospitalized patients are likely rare. Indeed, hospital admission is a well‐known predictor of instability of warfarin effect. [9] Although our results suggest that daily INR measurement is associated with a lower rate of overanticoagulation, future studies might better define lower risk patients for whom daily INR measurement would not be necessary.

A prior INR increase 0.9 in 1 day was associated with an increased risk of subsequent overanticoagulation. Although a rapidly rising INR is known to predict overanticoagulation[10, 14] we could find no evidence as to what specific rate of rise confers this risk. Our results suggest that use of a warfarin dosing protocol that considers both the absolute value of the INR and the rate of rise could reduce warfarin‐related adverse events.

There are important limitations of our study. We did not abstract warfarin dosages, which precluded study of the appropriateness of both initial warfarin dosing and adjustment of the warfarin dose based on INR results. MPSMS does not reliably capture antiplatelet agents or other agents that result in drug‐drug interactions with warfarin, such as antibiotics, so this factor could theoretically have confounded our results. Antibiotic use seems unlikely to be a major confounder, because patients with acute cardiovascular disease demonstrated a similar relationship between INR measurement and an INR 6.0 to that seen with pneumonia and surgical patients, despite the latter patients likely having greater antibiotics exposure. Furthermore, MPSMS does not capture indices of severity of illness, so other unmeasured confounders could have influenced our results. Although we have data for patients admitted to the hospital for only 4 conditions, these are conditions that represent approximately 22% of hospital admissions in the United States.[2] Strengths of our study include the nationally representative and randomly selected cases and use of data that were obtained from chart abstraction as opposed to administrative data. Through the use of centralized data abstraction, we avoided the potential bias introduced when hospitals self‐report adverse events.

In summary, in a national sample of patients admitted to the hospital for 4 common conditions, warfarin‐associated adverse events were detected in 7.4% of patients who received warfarin. Lack of daily INR measurement was associated with an increased risk of overanticoagulation and warfarin‐associated adverse events in certain patient populations. A 1‐day increase in the INR of 0.9 predicted subsequent overanticoagulation. These results provide actionable opportunities to improve safety in some hospitalized patients receiving warfarin.

Warfarin is 1 of the most common causes of adverse drug events, with hospitalized patients being particularly at risk compared to outpatients.[1] Despite the availability of new oral anticoagulants (NOACs), physicians commonly prescribe warfarin to hospitalized patients,[2] likely in part due to the greater difficulty in reversing NOACs compared to warfarin. Furthermore, uptake of the NOACs is likely to be slow in resource‐poor countries due to the lower cost of warfarin.[3] However, the narrow therapeutic index, frequent drug‐drug interactions, and patient variability in metabolism of warfarin makes management challenging.[4] Thus, warfarin remains a significant cause of adverse events in hospitalized patients, occurring in approximately 3% to 8% of exposed patients, depending on underlying condition.[2, 5]

An elevated international normalized ratio (INR) is a strong predictor of drug‐associated adverse events (patient harm). In a study employing 21 different electronic triggers to identify potential adverse events, an elevated INR had the highest yield for events associated with harm (96% of INRs >5.0 associated with harm).[6] Although pharmacist‐managed inpatient anticoagulation services have been shown to improve warfarin management,[7, 8] there are evidence gaps regarding the causes of warfarin‐related adverse events and practice changes that could decrease their frequency. Although overanticoagulation is a well‐known risk factor for warfarin‐related adverse events,[9, 10] there are few evidence‐based warfarin monitoring and dosing recommendations for hospitalized patients.[10] For example, the 2012 American College of Chest Physicians Antithrombotic Guidelines[11] provide a weak recommendation on initial dosing of warfarin, but no recommendations on how frequently to monitor the INR, or appropriate dosing responses to INR levels. Although many hospitals employ protocols that suggest daily INR monitoring until stable, there are no evidence‐based guidelines to support this practice.[12] Conversely, there are reports of flags to order an INR level that are not activated unless greater than 2[13] or 3 days[14] pass since the prior INR. Protocols from some major academic medical centers suggest that after a therapeutic INR is reached, INR levels can be measured intermittently, as infrequently as twice a week.[15, 16]

The 2015 Joint Commission anticoagulant‐focused National Patient Safety Goal[17] (initially issued in 2008) mandates the assessment of baseline coagulation status before starting warfarin, and warfarin dosing based on a current INR; however, current is not defined. Neither the extent to which the mandate for assessing baseline coagulation status is adhered to nor the relationship between this process of care and patient outcomes is known. The importance of adverse drug events associated with anticoagulants, included warfarin, was also recently highlighted in the 2014 federal National Action Plan for Adverse Drug Event Prevention. In this document, the prevention of adverse drug events associated with anticoagulants was 1 of the 3 areas selected for special national attention and action.[18]

The Medicare Patient Safety Monitoring System (MPSMS) is a national chart abstraction‐based system that includes 21 in‐hospital adverse event measures, including warfarin‐associated adverse drug events.[2] Because of the importance of warfarin‐associated bleeding in hospitalized patients, we analyzed MPSMS data to determine what factors related to INR monitoring practices place patients at risk for these events. We were particularly interested in determining if we could detect potentially modifiable predictors of overanticoagulation and warfarin‐associated adverse events.

METHODS

RESULTS

There were 130,828 patients included in the 2009 to 2013 MPSMS sample, of whom 19,445 (14.9%) received warfarin during their hospital stay and had at least 1 INR measurement. Among these patients, 5228 (26.9%) had no INR level above 1.5 and were excluded from further analysis, leaving 14,217 included patients. Of these patients, 1055 (7.4%) developed a warfarin‐associated adverse event. Table 1 demonstrates the baseline demographics and comorbidities of the included patients.

Warfarin was started on hospital day 1 for 6825 (48.0%) of 14,217 patients. Among these patients, 6539 (95.8%) had an INR measured within 1 calendar day. We were unable to determine how many patients who started warfarin later in their hospital stay had a baseline INR, as we did not capture INRs performed prior to the day that warfarin was initiated.

Supporting Table 1 in the online version of this article demonstrates the association between an INR 6.0 and the occurrence of warfarin‐associated adverse events. A maximum INR 6.0 occurred in 469 (3.3%) of the patients included in the study, and among those patients, 133 (28.4%) experienced a warfarin‐associated adverse event compared to 922 (6.7%) adverse events in the 13,748 patients who did not develop an INR 6.0 (P < 0.001).

Among 8529 patients who received warfarin for at least 3 days, beginning on the third day of warfarin, 1549 patients (18.2%) did not have INR measured at least once each day that they received warfarin. Table 2 demonstrates that patients who had 2 or more days on which the INR was not measured had higher rates of INR 6.0 than patients for whom the INR was measured daily. A similar association was seen for warfarin‐associated adverse events (Table 2).

Figure 1A demonstrates the association between the number of days without an INR measurement and the subsequent development of an INR 6.0 or a warfarin‐associated adverse event, adjusted for baseline patient characteristics, receipt of heparin and LMWH, and number of days on warfarin. Patients with 1 or more days without an INR measurement had higher risk‐adjusted ORs of a subsequent INR 6.0, although the difference was not statistically significant for surgical patients. The analysis results based on inverse propensity scoring are seen in Figure 1B. Cardiac and surgical patients with 2 or more days without an INR measurement were at higher risk of having a warfarin‐associated adverse event, whereas cardiac and pneumonia patients with 1 or more days without an INR measurement were at higher risk of developing an INR 6.0.

Figure 1

Supporting Table 2 in the online version of this article demonstrates the relationship between patient characteristics and the occurrence of an INR 6.0 or a warfarin‐related adverse event. The only characteristic that was associated with either of these outcomes for all 3 patient conditions was renal disease, which was positively associated with a warfarin‐associated adverse event. Warfarin use prior to arrival was associated with lower risks of both an INR 6.0 and a warfarin‐associated adverse event, except for among surgical patients. Supporting Table 3 in the online version of this article demonstrates the differences in patient characteristics between patients who had daily INR measurement and those who had at least 1 day without an INR measurement.

Figure 2 illustrates the relationship of the maximum INR to the prior 1‐day change in INR in 4963 patients whose INR on the day prior to the maximum INR was 2.0 to 3.5. When the increase in INR was <0.9, the risk of the next day's INR being 6.0 was 0.7%, and if the increase was 0.9, the risk was 5.2%. The risk of developing an INR 5.0 was 1.9% if the preceding day's INR increase was <0.9 and 15.3% if the prior day's INR rise was 0.9. Overall, 51% of INRs 6.0 and 55% of INRs 5.0 were immediately preceded by an INR increase of 0.9. The positive likelihood ratio (LR) for a 0.9 rise in INR predicting an INR of 6.0 was 4.2, and the positive LR was 4.9 for predicting an INR 5.0.

Figure 2

There was no decline in the frequency of warfarin use among the patients in the MPSMS sample during the study period (16.7% in 2009 and 17.3% in 2013).

DISCUSSION

We studied warfarin‐associated adverse events in a nationally representative study of patients who received warfarin while in an acute care hospital for a primary diagnosis of cardiac disease, pneumonia, or major surgery. Several findings resulted from our analysis. First, warfarin is still commonly prescribed to hospitalized patients and remains a frequent cause of adverse events; 7.4% of the 2009 to 2013 MPSMS population who received warfarin and had at least 1 INR >1.5 developed a warfarin‐associated adverse event.

Over 95% of patients who received warfarin on the day of hospital admission had an INR performed within 1 day. This is similar to the results from a 2006 single center study in which 95% of patients had an INR measured prior to their first dose of warfarin.[10] Since 2008, The Joint Commission's National Patient Safety Goal has required the assessment of coagulation status before starting warfarin.[17] The high level of adherence to this standard suggests that further attention to this process of care is unlikely to significantly improve patient safety.

We also found that the lack of daily INR measurements was associated with an increased risk of an INR 6.0 and warfarin‐associated adverse events in some patient populations. There is limited evidence addressing the appropriate frequency of INR measurement in hospitalized patients receiving warfarin. The Joint Commission National Patient Safety Goal requires use of a current INR to adjust this therapy, but provides no specifics.[17] Although some experts believe that INRs should be monitored daily in hospitalized patients, this does not appear to be uniformly accepted. In some reports, 2[13] or 3[14] consecutive days without the performance of an INR was required to activate a reminder. Protocols from some major teaching hospitals specify intermittent monitoring once the INR is therapeutic.[15, 16] Because our results suggest that lapses in INR measurement lead to overanticoagulation and warfarin‐related adverse events, it may be appropriate to measure INRs daily in most hospitalized patients receiving warfarin. This would be consistent with the many known causes of INR instability in patients admitted to the hospital, including drug‐drug interactions, hepatic dysfunction, and changes in volume of distribution, such that truly stable hospitalized patients are likely rare. Indeed, hospital admission is a well‐known predictor of instability of warfarin effect. [9] Although our results suggest that daily INR measurement is associated with a lower rate of overanticoagulation, future studies might better define lower risk patients for whom daily INR measurement would not be necessary.

A prior INR increase 0.9 in 1 day was associated with an increased risk of subsequent overanticoagulation. Although a rapidly rising INR is known to predict overanticoagulation[10, 14] we could find no evidence as to what specific rate of rise confers this risk. Our results suggest that use of a warfarin dosing protocol that considers both the absolute value of the INR and the rate of rise could reduce warfarin‐related adverse events.

There are important limitations of our study. We did not abstract warfarin dosages, which precluded study of the appropriateness of both initial warfarin dosing and adjustment of the warfarin dose based on INR results. MPSMS does not reliably capture antiplatelet agents or other agents that result in drug‐drug interactions with warfarin, such as antibiotics, so this factor could theoretically have confounded our results. Antibiotic use seems unlikely to be a major confounder, because patients with acute cardiovascular disease demonstrated a similar relationship between INR measurement and an INR 6.0 to that seen with pneumonia and surgical patients, despite the latter patients likely having greater antibiotics exposure. Furthermore, MPSMS does not capture indices of severity of illness, so other unmeasured confounders could have influenced our results. Although we have data for patients admitted to the hospital for only 4 conditions, these are conditions that represent approximately 22% of hospital admissions in the United States.[2] Strengths of our study include the nationally representative and randomly selected cases and use of data that were obtained from chart abstraction as opposed to administrative data. Through the use of centralized data abstraction, we avoided the potential bias introduced when hospitals self‐report adverse events.

In summary, in a national sample of patients admitted to the hospital for 4 common conditions, warfarin‐associated adverse events were detected in 7.4% of patients who received warfarin. Lack of daily INR measurement was associated with an increased risk of overanticoagulation and warfarin‐associated adverse events in certain patient populations. A 1‐day increase in the INR of 0.9 predicted subsequent overanticoagulation. These results provide actionable opportunities to improve safety in some hospitalized patients receiving warfarin.

Over the past decade, the use of chest computed tomography scans with pulmonary angiography (CTPA) for diagnosis of pulmonary embolism (PE) has soared due to the ease of acquisition, the desire for the additional information that CT scanning may provide, and heightened sensitivity to medical liability.[1, 2, 3, 4, 5, 6] In parallel with this shift, the incidence of PE has nearly doubled, despite no recorded increase in the pretest probability of the disease, increasing from 62 per 100,000 to 112 per 100,000 during the period of 1993 to 2006.⁶ One major explanation for this increase is that the improvement in CTPA resolution has enabled radiologists to identify more small peripheral (ie, segmental and subsegmental) filling defects. When confronted with the finding of a small peripheral filling defect on CTPA, clinicians often face a management quandary. Case series and retrospective series on outcomes of these patients do not support treatment, but they are limited by having small numbers of patients; the largest examined 93 patients and provided no insight into the treatment decision.[7] Uncertainty exists, furthermore, about the pathologic meaning of small peripheral filling defects.[8] Clinicians must weigh these arguments and the risk of anticoagulation against concerns about the consequences of untreated pulmonary thromboemboli. More information is needed, therefore, on the outcomes of patients with peripheral filling defects, and on variables impacting the treatment decision, in order to help clinicians manage these patients.[9]

In this study, we analyzed cases of patients with a single peripheral filling defect (SPFD). We choose to look at patients with a SPFD because they represent the starkest decision‐making treatment dilemma and are not infrequent. We assessed the 90‐day mortality and rate of postdischarge venous thromboembolism (VTE) of treated and untreated patients and identified characteristics of treated and untreated patients with a SPFD. We wished to determine the incidence of SPFD among patients evaluated with CTPA and to determine how often the defect is called a PE by the radiologist. We also aimed to determine what role secondary studies play in helping to clarify the diagnosis and management of SPFD and to identify other factors that may influence the decision to treat patients with this finding.

METHODS

RESULTS

A total of 4906 CTPAs were screened during the 66 months reviewed, identifying 518 (10.6%) with any filling defect and 153 (3.1%) with a SPFD. Thirteen patients were excluded from the primary analysis because their records could not be located, and another 6 were excluded because they had a concurrently positive CUS. The primary analysis was performed, therefore, with 134 patients. The inpatient service ordered 78% of the CTPAs. The initial radiologist stated in the impression section of the report that a PE was present in 99 of 134 (73.9%) studies. On over‐read of the 134 studies, 100 of these were considered to be positive for a PE. There was modest agreement between the initial impression and the consensus impression at over‐read (=0.69).

DISCUSSION

This very large retrospective study examines treatment and outcomes in patients with a SPFD. We found that SPFDs were common, showing up in approximately 3% of all the CTPAs performed. Among the studies that were deemed positive for PE, SPFD comprised nearly one‐third. Treatment of SPFD, whether concluded as PE or not, was not associated with a mortality benefit or difference in postdischarge VTE within 90 days. Our results add to the weight of smaller case‐control and retrospective series that also found no benefit from treating small PE.[7, 12, 13, 14, 15]

Given this data, why might physicians choose to treat? Physicians may feel compelled to anticoagulate due to extrapolation of data from the early studies showing a fatality rate of up to 30% in untreated PE.[2] Also, physicians may harbor the concern that, though small emboli may pose no immediate danger, they serve as a marker of hypercoagulability and as such are a harbinger of subsequent large clots. A reflexive treatment response to the radiologist's conclusion that the filling defect is a PE may also play a part. Balancing this concern is the recognition that the treatment for acute PE is not benign. The age‐adjusted incidence of major bleeding (eg, gastrointestinal or intracranial) with warfarin has increased by 71%, from 3.1 to 5.3 per 100,000, since the introduction of CTPA.[6] Also, as seen in this study, a substantial percentage of patients will incur the morbidity and cost of IVC‐filter placement.

When physicians face management uncertainty, they consider risk factors for the condition investigated, consult experts, employ additional studies, and weigh patient preference. In this study, history of immobility and VTE were, indeed, positively associated with treatment, but change in mental status was negatively so. Given that the PESI score is higher with change in mental status, this finding is superficially paradoxical but unsurprising. Mental‐status change could not likely stem from a SPFD and its presence heightens the risks of anticoagulation, hence dissuading treatment. Pulmonary consultations were documented in less than half of the cases and did not clearly sway the treatment decision. Determining whether more patients would have been treated if pulmonologists were not involved would require a prospective study.

The most important association with treatment was how the radiologist interpreted the SPFD. Even then, the influence of the radiologist's interpretation was far from complete: 40% of the cases in which PE was called went untreated, and 4 cases received treatment despite PE not being called. The value of the radiologist's interpretation is further undercut by the modest interobserver agreement found on over‐read, which is line with previous reports and reflective of lack of a gold standard for diagnosing isolated peripheral PE.[3, 12, 16]

Even if radiologists could agree upon what they are seeing, the question remains about the pathological importance. Unrecognized PE incidental to the cause of death are commonly found at autopsy. Autopsy studies reveal that up to 52% to 64% of patients have PE; and, if multiple blocks of lung tissue are studied, the prevalence increases up to 90%.[17, 18] In the series by Freiman et al., 59% of the identified thrombi were small enough not to be recognized on routine gross examination.[17] Furthermore, an unknown percentage of small clots, especially in the upper lobes, are in situ thrombi rather than emboli.[18] In the case of small dot‐like clots, Suh and colleagues have speculated that they represent normal embolic activity from the lower limbs, which are cleared routinely by the lung serving in its role as a filter.[19] Although our study only examined SPFD, the accumulation of small emboli could have pathologic consequences. In their review, Gali and Kim reported that 12% of patients with chronic thromboembolic pulmonary hypertension who underwent pulmonary endarterectomy had disease confined to the distal segmental and subsegmental arteries.[13]

Use of secondary studies could mitigate some of the diagnostic and management uncertainty, but they were obtained in only about a quarter of the cases. The use of a second lung‐imaging study following CTPA is not recommended in guidelines or diagnostic algorithms, but in our institution a significant minority of physicians were employing these tests to clarify the nature of the filling defects.[20] Tapson, speaking to the treatment dilemma that small PEs present, has suggested that prospective trials on this topic employ tests that investigate risk for poor outcome if untreated including cardiopulmonary reserve, D‐dimer, and presence of lower‐limb thrombus.[21] Indeed, a study is ongoing examining the outcome at 90 days of patients with single or multiple subsegmental embolism with negative CUS.[22]

Ten of the 134 patients (7.4%) with peripheral filling defects died within 90 days. It is difficult to establish whether these deaths were PE‐specific mortalities because there was a high degree of comorbid illness in this cohort. Five of the 134 (3.7%) had recurrent VTE, which is comparable to the outcomes in other studies.[23]

There are limitations to this study. This study is the first to limit analysis of the filling defects to single defects in the segmental or subsegmental pulmonary arteries. This subset of patients includes those with the least clot burden, therefore representing the starkest decision‐making treatment dilemma, and the incidence of these clots is not insignificant. As a retrospective study, we could not fully capture all of the considerations that may have factored into the clinicians' decision‐making regarding treatment, including patient preference. Because of inadequate documentation, especially in the emergency department notes, we were unable to calculate pretest probability. Also, we cannot exclude that subclinical VTEs were occurring that would later harm the patients. We did not analyze the role of D‐dimer testing because that test is validated to guide the decision to obtain lung‐imaging studies and not to inform the treatment decision. In our cohort, 89 of 134 (66%) of our patients were already hospitalized for other diagnoses prior to PE being queried. Moreover, many of these patients had active malignancy or were being treated for pneumonia, which would decrease the positive predictive value of the D‐dimer test. D‐dimer performs poorly when used for prognosis.[24] This is a single‐center study, therefore the comparability of our findings to other centers may be an issue, although our findings generally accord with those from other single‐center studies.[7, 12, 24, 25] We determined the recurrence rate from the hospital records and could have missed cases diagnosed elsewhere. However, our hospital is the only one in the city and serves the vast majority of patients in the area, and 88% of our cohort had a repeat visit to our hospital subsequently. In addition, the radiology service is the only one in the area that provides outpatient CUS, CTPA, and ventilation‐perfusion scan studies. Our study is the largest to date on this issue. However, our sample size is somewhat modest, and consequently the factors associated with treatment have large confidence intervals. We are therefore constrained in recommending empiric application of our findings. Nonetheless, our results in terms of no difference in mortality and recurrence between treated and untreated patients are in keeping with other studies on this topic. Also, our simulation analysis did reveal factors that were highly associated with the decision to treat. These findings as a whole strongly point to the need for a larger study on this issue, because, as we and other authors have argued, the consequences of treatment are not benign.[6]

In conclusion, this study shows that SPFDs are common and that there was no difference in 90‐day mortality between treated and untreated patients, regardless of whether the defects were interpreted as PE or not. Physicians appear to rely heavily on the radiologist's interpretation for their treatment decision, but they will also treat when the interpretation is not PE and not infrequently abstain when it is. Treatment remains common despite the modest agreement among radiologists whether the peripheral filling defect even represents PE. When secondary imaging studies are obtained and negative, physicians forgo treatment. Larger studies are needed to help clarify our findings and should include decision‐making algorithms that include secondary imaging studies, because these studies may provide enough reassurance when negative to sway physicians against treatment.

Over the past decade, the use of chest computed tomography scans with pulmonary angiography (CTPA) for diagnosis of pulmonary embolism (PE) has soared due to the ease of acquisition, the desire for the additional information that CT scanning may provide, and heightened sensitivity to medical liability.[1, 2, 3, 4, 5, 6] In parallel with this shift, the incidence of PE has nearly doubled, despite no recorded increase in the pretest probability of the disease, increasing from 62 per 100,000 to 112 per 100,000 during the period of 1993 to 2006.⁶ One major explanation for this increase is that the improvement in CTPA resolution has enabled radiologists to identify more small peripheral (ie, segmental and subsegmental) filling defects. When confronted with the finding of a small peripheral filling defect on CTPA, clinicians often face a management quandary. Case series and retrospective series on outcomes of these patients do not support treatment, but they are limited by having small numbers of patients; the largest examined 93 patients and provided no insight into the treatment decision.[7] Uncertainty exists, furthermore, about the pathologic meaning of small peripheral filling defects.[8] Clinicians must weigh these arguments and the risk of anticoagulation against concerns about the consequences of untreated pulmonary thromboemboli. More information is needed, therefore, on the outcomes of patients with peripheral filling defects, and on variables impacting the treatment decision, in order to help clinicians manage these patients.[9]

In this study, we analyzed cases of patients with a single peripheral filling defect (SPFD). We choose to look at patients with a SPFD because they represent the starkest decision‐making treatment dilemma and are not infrequent. We assessed the 90‐day mortality and rate of postdischarge venous thromboembolism (VTE) of treated and untreated patients and identified characteristics of treated and untreated patients with a SPFD. We wished to determine the incidence of SPFD among patients evaluated with CTPA and to determine how often the defect is called a PE by the radiologist. We also aimed to determine what role secondary studies play in helping to clarify the diagnosis and management of SPFD and to identify other factors that may influence the decision to treat patients with this finding.

METHODS

RESULTS

A total of 4906 CTPAs were screened during the 66 months reviewed, identifying 518 (10.6%) with any filling defect and 153 (3.1%) with a SPFD. Thirteen patients were excluded from the primary analysis because their records could not be located, and another 6 were excluded because they had a concurrently positive CUS. The primary analysis was performed, therefore, with 134 patients. The inpatient service ordered 78% of the CTPAs. The initial radiologist stated in the impression section of the report that a PE was present in 99 of 134 (73.9%) studies. On over‐read of the 134 studies, 100 of these were considered to be positive for a PE. There was modest agreement between the initial impression and the consensus impression at over‐read (=0.69).

DISCUSSION

This very large retrospective study examines treatment and outcomes in patients with a SPFD. We found that SPFDs were common, showing up in approximately 3% of all the CTPAs performed. Among the studies that were deemed positive for PE, SPFD comprised nearly one‐third. Treatment of SPFD, whether concluded as PE or not, was not associated with a mortality benefit or difference in postdischarge VTE within 90 days. Our results add to the weight of smaller case‐control and retrospective series that also found no benefit from treating small PE.[7, 12, 13, 14, 15]

Given this data, why might physicians choose to treat? Physicians may feel compelled to anticoagulate due to extrapolation of data from the early studies showing a fatality rate of up to 30% in untreated PE.[2] Also, physicians may harbor the concern that, though small emboli may pose no immediate danger, they serve as a marker of hypercoagulability and as such are a harbinger of subsequent large clots. A reflexive treatment response to the radiologist's conclusion that the filling defect is a PE may also play a part. Balancing this concern is the recognition that the treatment for acute PE is not benign. The age‐adjusted incidence of major bleeding (eg, gastrointestinal or intracranial) with warfarin has increased by 71%, from 3.1 to 5.3 per 100,000, since the introduction of CTPA.[6] Also, as seen in this study, a substantial percentage of patients will incur the morbidity and cost of IVC‐filter placement.

When physicians face management uncertainty, they consider risk factors for the condition investigated, consult experts, employ additional studies, and weigh patient preference. In this study, history of immobility and VTE were, indeed, positively associated with treatment, but change in mental status was negatively so. Given that the PESI score is higher with change in mental status, this finding is superficially paradoxical but unsurprising. Mental‐status change could not likely stem from a SPFD and its presence heightens the risks of anticoagulation, hence dissuading treatment. Pulmonary consultations were documented in less than half of the cases and did not clearly sway the treatment decision. Determining whether more patients would have been treated if pulmonologists were not involved would require a prospective study.

The most important association with treatment was how the radiologist interpreted the SPFD. Even then, the influence of the radiologist's interpretation was far from complete: 40% of the cases in which PE was called went untreated, and 4 cases received treatment despite PE not being called. The value of the radiologist's interpretation is further undercut by the modest interobserver agreement found on over‐read, which is line with previous reports and reflective of lack of a gold standard for diagnosing isolated peripheral PE.[3, 12, 16]

Even if radiologists could agree upon what they are seeing, the question remains about the pathological importance. Unrecognized PE incidental to the cause of death are commonly found at autopsy. Autopsy studies reveal that up to 52% to 64% of patients have PE; and, if multiple blocks of lung tissue are studied, the prevalence increases up to 90%.[17, 18] In the series by Freiman et al., 59% of the identified thrombi were small enough not to be recognized on routine gross examination.[17] Furthermore, an unknown percentage of small clots, especially in the upper lobes, are in situ thrombi rather than emboli.[18] In the case of small dot‐like clots, Suh and colleagues have speculated that they represent normal embolic activity from the lower limbs, which are cleared routinely by the lung serving in its role as a filter.[19] Although our study only examined SPFD, the accumulation of small emboli could have pathologic consequences. In their review, Gali and Kim reported that 12% of patients with chronic thromboembolic pulmonary hypertension who underwent pulmonary endarterectomy had disease confined to the distal segmental and subsegmental arteries.[13]

Use of secondary studies could mitigate some of the diagnostic and management uncertainty, but they were obtained in only about a quarter of the cases. The use of a second lung‐imaging study following CTPA is not recommended in guidelines or diagnostic algorithms, but in our institution a significant minority of physicians were employing these tests to clarify the nature of the filling defects.[20] Tapson, speaking to the treatment dilemma that small PEs present, has suggested that prospective trials on this topic employ tests that investigate risk for poor outcome if untreated including cardiopulmonary reserve, D‐dimer, and presence of lower‐limb thrombus.[21] Indeed, a study is ongoing examining the outcome at 90 days of patients with single or multiple subsegmental embolism with negative CUS.[22]

Ten of the 134 patients (7.4%) with peripheral filling defects died within 90 days. It is difficult to establish whether these deaths were PE‐specific mortalities because there was a high degree of comorbid illness in this cohort. Five of the 134 (3.7%) had recurrent VTE, which is comparable to the outcomes in other studies.[23]

There are limitations to this study. This study is the first to limit analysis of the filling defects to single defects in the segmental or subsegmental pulmonary arteries. This subset of patients includes those with the least clot burden, therefore representing the starkest decision‐making treatment dilemma, and the incidence of these clots is not insignificant. As a retrospective study, we could not fully capture all of the considerations that may have factored into the clinicians' decision‐making regarding treatment, including patient preference. Because of inadequate documentation, especially in the emergency department notes, we were unable to calculate pretest probability. Also, we cannot exclude that subclinical VTEs were occurring that would later harm the patients. We did not analyze the role of D‐dimer testing because that test is validated to guide the decision to obtain lung‐imaging studies and not to inform the treatment decision. In our cohort, 89 of 134 (66%) of our patients were already hospitalized for other diagnoses prior to PE being queried. Moreover, many of these patients had active malignancy or were being treated for pneumonia, which would decrease the positive predictive value of the D‐dimer test. D‐dimer performs poorly when used for prognosis.[24] This is a single‐center study, therefore the comparability of our findings to other centers may be an issue, although our findings generally accord with those from other single‐center studies.[7, 12, 24, 25] We determined the recurrence rate from the hospital records and could have missed cases diagnosed elsewhere. However, our hospital is the only one in the city and serves the vast majority of patients in the area, and 88% of our cohort had a repeat visit to our hospital subsequently. In addition, the radiology service is the only one in the area that provides outpatient CUS, CTPA, and ventilation‐perfusion scan studies. Our study is the largest to date on this issue. However, our sample size is somewhat modest, and consequently the factors associated with treatment have large confidence intervals. We are therefore constrained in recommending empiric application of our findings. Nonetheless, our results in terms of no difference in mortality and recurrence between treated and untreated patients are in keeping with other studies on this topic. Also, our simulation analysis did reveal factors that were highly associated with the decision to treat. These findings as a whole strongly point to the need for a larger study on this issue, because, as we and other authors have argued, the consequences of treatment are not benign.[6]

In conclusion, this study shows that SPFDs are common and that there was no difference in 90‐day mortality between treated and untreated patients, regardless of whether the defects were interpreted as PE or not. Physicians appear to rely heavily on the radiologist's interpretation for their treatment decision, but they will also treat when the interpretation is not PE and not infrequently abstain when it is. Treatment remains common despite the modest agreement among radiologists whether the peripheral filling defect even represents PE. When secondary imaging studies are obtained and negative, physicians forgo treatment. Larger studies are needed to help clarify our findings and should include decision‐making algorithms that include secondary imaging studies, because these studies may provide enough reassurance when negative to sway physicians against treatment.

Over the past decade, the use of chest computed tomography scans with pulmonary angiography (CTPA) for diagnosis of pulmonary embolism (PE) has soared due to the ease of acquisition, the desire for the additional information that CT scanning may provide, and heightened sensitivity to medical liability.[1, 2, 3, 4, 5, 6] In parallel with this shift, the incidence of PE has nearly doubled, despite no recorded increase in the pretest probability of the disease, increasing from 62 per 100,000 to 112 per 100,000 during the period of 1993 to 2006.⁶ One major explanation for this increase is that the improvement in CTPA resolution has enabled radiologists to identify more small peripheral (ie, segmental and subsegmental) filling defects. When confronted with the finding of a small peripheral filling defect on CTPA, clinicians often face a management quandary. Case series and retrospective series on outcomes of these patients do not support treatment, but they are limited by having small numbers of patients; the largest examined 93 patients and provided no insight into the treatment decision.[7] Uncertainty exists, furthermore, about the pathologic meaning of small peripheral filling defects.[8] Clinicians must weigh these arguments and the risk of anticoagulation against concerns about the consequences of untreated pulmonary thromboemboli. More information is needed, therefore, on the outcomes of patients with peripheral filling defects, and on variables impacting the treatment decision, in order to help clinicians manage these patients.[9]

In this study, we analyzed cases of patients with a single peripheral filling defect (SPFD). We choose to look at patients with a SPFD because they represent the starkest decision‐making treatment dilemma and are not infrequent. We assessed the 90‐day mortality and rate of postdischarge venous thromboembolism (VTE) of treated and untreated patients and identified characteristics of treated and untreated patients with a SPFD. We wished to determine the incidence of SPFD among patients evaluated with CTPA and to determine how often the defect is called a PE by the radiologist. We also aimed to determine what role secondary studies play in helping to clarify the diagnosis and management of SPFD and to identify other factors that may influence the decision to treat patients with this finding.

METHODS

RESULTS

A total of 4906 CTPAs were screened during the 66 months reviewed, identifying 518 (10.6%) with any filling defect and 153 (3.1%) with a SPFD. Thirteen patients were excluded from the primary analysis because their records could not be located, and another 6 were excluded because they had a concurrently positive CUS. The primary analysis was performed, therefore, with 134 patients. The inpatient service ordered 78% of the CTPAs. The initial radiologist stated in the impression section of the report that a PE was present in 99 of 134 (73.9%) studies. On over‐read of the 134 studies, 100 of these were considered to be positive for a PE. There was modest agreement between the initial impression and the consensus impression at over‐read (=0.69).

DISCUSSION

This very large retrospective study examines treatment and outcomes in patients with a SPFD. We found that SPFDs were common, showing up in approximately 3% of all the CTPAs performed. Among the studies that were deemed positive for PE, SPFD comprised nearly one‐third. Treatment of SPFD, whether concluded as PE or not, was not associated with a mortality benefit or difference in postdischarge VTE within 90 days. Our results add to the weight of smaller case‐control and retrospective series that also found no benefit from treating small PE.[7, 12, 13, 14, 15]

Given this data, why might physicians choose to treat? Physicians may feel compelled to anticoagulate due to extrapolation of data from the early studies showing a fatality rate of up to 30% in untreated PE.[2] Also, physicians may harbor the concern that, though small emboli may pose no immediate danger, they serve as a marker of hypercoagulability and as such are a harbinger of subsequent large clots. A reflexive treatment response to the radiologist's conclusion that the filling defect is a PE may also play a part. Balancing this concern is the recognition that the treatment for acute PE is not benign. The age‐adjusted incidence of major bleeding (eg, gastrointestinal or intracranial) with warfarin has increased by 71%, from 3.1 to 5.3 per 100,000, since the introduction of CTPA.[6] Also, as seen in this study, a substantial percentage of patients will incur the morbidity and cost of IVC‐filter placement.

When physicians face management uncertainty, they consider risk factors for the condition investigated, consult experts, employ additional studies, and weigh patient preference. In this study, history of immobility and VTE were, indeed, positively associated with treatment, but change in mental status was negatively so. Given that the PESI score is higher with change in mental status, this finding is superficially paradoxical but unsurprising. Mental‐status change could not likely stem from a SPFD and its presence heightens the risks of anticoagulation, hence dissuading treatment. Pulmonary consultations were documented in less than half of the cases and did not clearly sway the treatment decision. Determining whether more patients would have been treated if pulmonologists were not involved would require a prospective study.

The most important association with treatment was how the radiologist interpreted the SPFD. Even then, the influence of the radiologist's interpretation was far from complete: 40% of the cases in which PE was called went untreated, and 4 cases received treatment despite PE not being called. The value of the radiologist's interpretation is further undercut by the modest interobserver agreement found on over‐read, which is line with previous reports and reflective of lack of a gold standard for diagnosing isolated peripheral PE.[3, 12, 16]

Even if radiologists could agree upon what they are seeing, the question remains about the pathological importance. Unrecognized PE incidental to the cause of death are commonly found at autopsy. Autopsy studies reveal that up to 52% to 64% of patients have PE; and, if multiple blocks of lung tissue are studied, the prevalence increases up to 90%.[17, 18] In the series by Freiman et al., 59% of the identified thrombi were small enough not to be recognized on routine gross examination.[17] Furthermore, an unknown percentage of small clots, especially in the upper lobes, are in situ thrombi rather than emboli.[18] In the case of small dot‐like clots, Suh and colleagues have speculated that they represent normal embolic activity from the lower limbs, which are cleared routinely by the lung serving in its role as a filter.[19] Although our study only examined SPFD, the accumulation of small emboli could have pathologic consequences. In their review, Gali and Kim reported that 12% of patients with chronic thromboembolic pulmonary hypertension who underwent pulmonary endarterectomy had disease confined to the distal segmental and subsegmental arteries.[13]

Use of secondary studies could mitigate some of the diagnostic and management uncertainty, but they were obtained in only about a quarter of the cases. The use of a second lung‐imaging study following CTPA is not recommended in guidelines or diagnostic algorithms, but in our institution a significant minority of physicians were employing these tests to clarify the nature of the filling defects.[20] Tapson, speaking to the treatment dilemma that small PEs present, has suggested that prospective trials on this topic employ tests that investigate risk for poor outcome if untreated including cardiopulmonary reserve, D‐dimer, and presence of lower‐limb thrombus.[21] Indeed, a study is ongoing examining the outcome at 90 days of patients with single or multiple subsegmental embolism with negative CUS.[22]

Ten of the 134 patients (7.4%) with peripheral filling defects died within 90 days. It is difficult to establish whether these deaths were PE‐specific mortalities because there was a high degree of comorbid illness in this cohort. Five of the 134 (3.7%) had recurrent VTE, which is comparable to the outcomes in other studies.[23]

There are limitations to this study. This study is the first to limit analysis of the filling defects to single defects in the segmental or subsegmental pulmonary arteries. This subset of patients includes those with the least clot burden, therefore representing the starkest decision‐making treatment dilemma, and the incidence of these clots is not insignificant. As a retrospective study, we could not fully capture all of the considerations that may have factored into the clinicians' decision‐making regarding treatment, including patient preference. Because of inadequate documentation, especially in the emergency department notes, we were unable to calculate pretest probability. Also, we cannot exclude that subclinical VTEs were occurring that would later harm the patients. We did not analyze the role of D‐dimer testing because that test is validated to guide the decision to obtain lung‐imaging studies and not to inform the treatment decision. In our cohort, 89 of 134 (66%) of our patients were already hospitalized for other diagnoses prior to PE being queried. Moreover, many of these patients had active malignancy or were being treated for pneumonia, which would decrease the positive predictive value of the D‐dimer test. D‐dimer performs poorly when used for prognosis.[24] This is a single‐center study, therefore the comparability of our findings to other centers may be an issue, although our findings generally accord with those from other single‐center studies.[7, 12, 24, 25] We determined the recurrence rate from the hospital records and could have missed cases diagnosed elsewhere. However, our hospital is the only one in the city and serves the vast majority of patients in the area, and 88% of our cohort had a repeat visit to our hospital subsequently. In addition, the radiology service is the only one in the area that provides outpatient CUS, CTPA, and ventilation‐perfusion scan studies. Our study is the largest to date on this issue. However, our sample size is somewhat modest, and consequently the factors associated with treatment have large confidence intervals. We are therefore constrained in recommending empiric application of our findings. Nonetheless, our results in terms of no difference in mortality and recurrence between treated and untreated patients are in keeping with other studies on this topic. Also, our simulation analysis did reveal factors that were highly associated with the decision to treat. These findings as a whole strongly point to the need for a larger study on this issue, because, as we and other authors have argued, the consequences of treatment are not benign.[6]

In conclusion, this study shows that SPFDs are common and that there was no difference in 90‐day mortality between treated and untreated patients, regardless of whether the defects were interpreted as PE or not. Physicians appear to rely heavily on the radiologist's interpretation for their treatment decision, but they will also treat when the interpretation is not PE and not infrequently abstain when it is. Treatment remains common despite the modest agreement among radiologists whether the peripheral filling defect even represents PE. When secondary imaging studies are obtained and negative, physicians forgo treatment. Larger studies are needed to help clarify our findings and should include decision‐making algorithms that include secondary imaging studies, because these studies may provide enough reassurance when negative to sway physicians against treatment.

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

Pneumonia results in some 1.2 million hospital admissions each year in the United States, is the second leading cause of hospitalization among patients over 65, and accounts for more than $10 billion annually in hospital expenditures.1, 2 As a result of complex demographic and clinical forces, including an aging population, increasing prevalence of comorbidities, and changes in antimicrobial resistance patterns, between the periods 1988 to 1990 and 2000 to 2002 the number of patients hospitalized for pneumonia grew by 20%, and pneumonia was the leading infectious cause of death.3, 4

Given its public health significance, pneumonia has been the subject of intensive quality measurement and improvement efforts for well over a decade. Two of the largest initiatives are the Centers for Medicare & Medicaid Services (CMS) National Pneumonia Project and The Joint Commission ORYX program.5, 6 These efforts have largely entailed measuring hospital performance on pneumonia‐specific processes of care, such as whether blood oxygen levels were assessed, whether blood cultures were drawn before antibiotic treatment was initiated, the choice and timing of antibiotics, and smoking cessation counseling and vaccination at the time of discharge. While measuring processes of care (especially when they are based on sound evidence), can provide insights about quality, and can help guide hospital improvement efforts, these measures necessarily focus on a narrow spectrum of the overall care provided. Outcomes can complement process measures by directing attention to the results of care, which are influenced by both measured and unmeasured factors, and which may be more relevant from the patient's perspective.79

In 2008 CMS expanded its public reporting initiatives by adding risk‐standardized hospital mortality rates for pneumonia to the Hospital Compare website (http://www.hospitalcompare.hhs.gov/).10 Readmission rates were added in 2009. We sought to examine patterns of hospital and regional performance for patients with pneumonia as reflected in 30‐day risk‐standardized readmission and mortality rates. Our report complements the June 2010 annual release of data on the Hospital Compare website. CMS also reports 30‐day risk‐standardized mortality and readmission for acute myocardial infarction and heart failure; a description of the 2010 reporting results for those measures are described elsewhere.

Methods

Results

Hospital‐Specific Risk‐Standardized 30‐Day Mortality and Readmission Rates

Of the 1,118,583 patients included in the mortality analysis 129,444 (11.6%) died within 30 days of hospital admission. The median (Q1, Q3) hospital 30‐day risk‐standardized mortality rate was 11.1% (10.0%, 12.3%), and ranged from 6.7% to 20.9% (Table 1, Figure 1). Hospitals at the 10th percentile had 30‐day risk‐standardized mortality rates of 9.0% while for those at the 90th percentile of performance the rate was 13.5%. The odds of all‐cause mortality for a patient treated at a hospital that was one standard deviation above the national average was 1.68 times higher than that of a patient treated at a hospital that was one standard deviation below the national average.

Figure 1

Table 1.Risk‐Standardized Hospital 30‐Day Pneumonia Mortality and Readmission Rates

	Mortality	Readmission
Abbreviation: SD, standard deviation.
Patients (n)	1118583	1161817
Hospitals (n)	4788	4813
Patient age, years, median (Q1, Q3)	81 (74,86)	80 (74,86)
Nonwhite, %	11.1	11.1
Hospital case volume, median (Q1, Q3)	168 (77,323)	174 (79,334)
Risk‐standardized hospital rate, mean (SD)	11.2 (1.2)	18.3 (0.9)
Minimum	6.7	13.6
1st percentile	7.5	14.9
5th percentile	8.5	15.8
10th percentile	9.0	16.4
25th percentile	10.0	17.2
Median	11.1	18.2
75th percentile	12.3	19.2
90th percentile	13.5	20.4
95th percentile	14.4	21.1
99th percentile	16.1	22.8
Maximum	20.9	26.7
Model fit statistics
c‐Statistic	0.72	0.63
Intrahospital Correlation	0.07	0.03

For the 3‐year period 2006 to 2009, 222 (4.7%) hospitals were categorized as having a mortality rate that was better than the national average, 3968 (83.7%) were no different than the national average, 221 (4.6%) were worse and 332 (7.0%) did not meet the minimum case threshold.

Among the 1,161,817 patients included in the readmission analysis 212,638 (18.3%) were readmitted within 30 days of hospital discharge. The median (Q1,Q3) 30‐day risk‐standardized readmission rate was 18.2% (17.2%, 19.2%) and ranged from 13.6% to 26.7% (Table 1, Figure 2). Hospitals at the 10th percentile had 30‐day risk‐standardized readmission rates of 16.4% while for those at the 90th percentile of performance the rate was 20.4%. The odds of all‐cause readmission for a patient treated at a hospital that was one standard deviation above the national average was 1.40 times higher than the odds of all‐cause readmission if treated at a hospital that was one standard deviation below the national average.

Figure 2

For the 3‐year period 2006 to 2009, 64 (1.3%) hospitals were categorized as having a readmission rate that was better than the national average, 4203 (88.2%) were no different than the national average, 163 (3.4%) were worse and 333 (7.0%) had less than 25 cases and were therefore not categorized.

While risk‐standardized readmission rates were substantially higher than risk‐standardized mortality rates, mortality rates varied more. For example, the top 10% of hospitals had a relative mortality rate that was 33% lower than those in the bottom 10%, as compared with just a 20% relative difference for readmission rates. The coefficient of variation, a normalized measure of dispersion that unlike the standard deviation is independent of the population mean, was 10.7 for risk‐standardized mortality rates and 4.9 for readmission rates.