Hospital Medicine

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 50-year-old man presented to the emergency department with a 3-day-old cut on his anterior right shin. Associated redness, warmth, pain, and swelling had progressed. The patient had no history of prior DVT or pulmonary embolism (PE). His temperature was 38.5°C, and his white blood cell count of 18,000. On review of systems, he denied shortness of breath and chest pain. He was diagnosed with cellulitis and administered intravenous fluids and cefazolin. The clinician wondered whether to perform lower extremity ultrasound to rule out concurrent DVT.

WHY YOU MIGHT THINK ULTRASOUND IS HELPFUL IN RULING OUT DVT IN CELLULITIS

Lower extremity cellulitis, a common infection of the skin and subcutaneous tissues, is characterized by unilateral erythema, pain, warmth, and swelling. The infection usually follows a skin breach that allows bacteria to enter. DVT may present similarly, and symptoms can include mild leukocytosis and elevated temperature. Because of the clinical similarities, clinicians often order compression ultrasound of the extremity to rule out concurrent DVT in cellulitis. Further impetus for testing stems from fear of the potential complications of untreated DVT, including post-thrombotic syndrome, chronic venous insufficiency, and venous ulceration. A subsequent PE can be fatal, or can cause significant morbidity, including chronic VTE with associated pulmonary hypertension. An estimated quarter of all PEs present as sudden death.¹

WHY ULTRASOUND IS NOT HELPFUL IN THIS SETTING

Studies have shown that ultrasound is ordered for 16% to 73% of patients with a cellulitis diagnosis.^2,3 Although testing is commonly performed, a meta-analysis of 9 studies of cellulitis patients who underwent ultrasound testing for concurrent DVT revealed a low pooled incidence of total DVT (3.1%) and proximal DVT (2.1%).⁴ Maze et al.² retrospectively reviewed 1515 cellulitis cases (identified by International Classification of Diseases, Ninth Revision codes) at a single center in New Zealand over 3 years. Of the 1515 patients, 240 (16%) had ultrasound performed, and only 3 (1.3%) were found to have DVT. Two of the 3 had active malignancy, and the third had injected battery acid into the area. In a 5-year retrospective cohort study at a Veterans Administration hospital in Connecticut, Gunderson and Chang³ reviewed the cases of 183 patients with cellulitis and found ultrasound testing commonly performed (73% of cases) to assess for DVT. Only 1 patient (<1%) was diagnosed with new DVT in the ipsilateral leg, and acute DVT was diagnosed in the contralateral leg of 2 other patients. Overall, these studies indicate the incidence of concurrent DVT in cellulitis is low, regardless of the frequency of ultrasound testing.

Although the cost of a single ultrasound test is not prohibitive, annual total costs hospital-wide and nationally are large. In the United States, the charge for a unilateral duplex ultrasound of the extremity ranges from $260 to $1300, and there is an additional charge for interpretation by a radiologist.⁵ In a retrospective study spanning 3.5 years and involving 2 community hospitals in Michigan, an estimated $290,000 was spent on ultrasound tests defined as unnecessary for patients with cellulitis.⁶ A limitation of the study was defining a test as unnecessary based on its result being negative.

DOES WELLS SCORE WITH D-DIMER HELP DEFINE A LOW-RISK POPULATION?

The Wells clinical prediction rule is commonly used to assess the pretest probability of DVT in patients presenting with unilateral leg symptoms. The Wells score is often combined with D-dimer testing to help determine whether ultrasound is necessary. Studies of patients with suspected DVT have found that those considered low risk according to the Wells criteria had a 6.5% incidence of DVT.⁷ However, the predictive value is lower in the setting of presumed cellulitis. In a prospective cohort study of 200 patients with cellulitis, Maze et al.⁸ reported that use of the Wells score with D-dimer testing overestimated the DVT risk. D-dimer level was elevated for 74% of patients, and 20.5% were high-risk by Wells criteria. An algorithm determined that—among patients with a high-risk Wells score, a positive D-dimer result, or both—only 1 (0.5%) was diagnosed with ipsilateral DVT after ultrasound testing. Two patients were diagnosed with DVT in the contralateral leg. These results suggest that a strategy that incorporates the Wells score and D-dimer testing in the setting of acute cellulitis provides little value. The authors concluded that, in the absence of a known hypercoagulable state, ultrasound is not warranted. However, their study did not assess whether there are any specific hypercoagulable states for which further testing may be indicated.

WHEN MIGHT ULTRASOUND BE HELPFUL IN CELLULITIS?

Investigators have described possible DVT risk factors in patients with cellulitis, but definitive associations are lacking because of the insufficient number of patients studied.^8,9 The most consistently identified DVT risk factor is history of previous thromboembolism. In a retrospective analysis of patients with cellulitis, Afzal et al.⁶ found that, of the 66.8% who underwent ultrasound testing, 5.5% were identified as having concurrent DVT. The authors performed univariate analyses of 15 potential risk factors, including active malignancy, oral contraceptive pill use, recent hospitalization, and surgery. A higher incidence of DVT was found for patients with history of VTE (odds ratio [OR], 5.7; 95% confidence interval [CI], 2.3-13.7), calf swelling (OR, 4.5; 95% CI, 1.3-15.8), CVA (OR, 3.5; 95% CI, 1.2-10.1), or hypertension (OR, 3.5; 95% CI, 0.98-12.2). Given the wide confidence intervals, paucity of studies, and lack of definitive data in the setting of cellulitis, clinicians may want to consider the risk factors established in larger trials in other settings, including known immobility (OR, <2); thrombophilia, CHF, and CVA with hemiparesis (OR, 2-9); and trauma and recent surgery (OR, >10).¹⁰

WHAT YOU SHOULD DO INSTEAD

As the incidence of concurrent VTE in patients with cellulitis is low, the essential step is to make a clear diagnosis of cellulitis based on its established signs and symptoms. A 2-center trial of 145 patients found that cellulitis was diagnosed accurately by general medicine and emergency medicine physicians 72% of the time, with evaluation by dermatologists and infectious disease specialists used as the gold standard. Only 5% of the misdiagnosed patients were diagnosed with DVT; stasis dermatitis was the most common alternative diagnosis. Taking a thorough history may elicit risk factors consistent with cellulitis, such as a recent injury with a break in the skin. On examination, cellulitis should be suspected for patients with fever and localized pain, redness, swelling, and warmth—the cardinal signs of dolor, rubor, tumor, and calor. An injury or entry site and leukocytosis also support the diagnosis of cellulitis. Distinct margins of erythema on the skin are highly suspicious for erysipelas.¹¹ Other physical findings (eg, laceration, purulent drainage, lymphangitic spread, fluctuating mass) also are consistent with a diagnosis of cellulitis.

The patient’s history is also essential in determining whether any DVT risk factors are present. Past medical history of VTE or CVA, or recent history of surgery, immobility, or trauma, should alert the clinician to the possibility of DVT. Family history of VTE increases the likelihood of DVT. Acute shortness of breath or chest pain in the setting of concerning lower extremity findings for DVT should raise concern for DVT and concurrent PE.

If the classic features of cellulitis are present, empiric antibiotics should be initiated. Routine ultrasound testing for all patients with cellulitis is of low value. However, as the incidence of DVT in this population is not negligible, those with VTE risk factors should be targeted for testing. Studies in the setting of cellulitis provide little guidance regarding specific risk factors that can be used to determine who should undergo further testing. Given this limitation, we suggest that clinicians incorporate into their decision making the well-established VTE risk factors identified for large populations studied in other settings, such as the postoperative period. Specifically, clinicians should consider ultrasound testing for patients with cellulitis and prior history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery.^10-12 Ultrasound should also be considered for patients with cellulitis that does not improve and for patients whose localized symptoms worsen despite use of antibiotics.

• Do not routinely perform ultrasound to rule out concurrent DVT in cases of cellulitis.

• Consider compression ultrasound if there is a history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery. Also consider it for patients who do not respond to antibiotics.

• In cases of cellulitis, avoid use of the Wells score alone or with D-dimer testing, as it likely overestimates the DVT risk.

The current evidence shows that, for most patients with cellulitis, routine ultrasound testing for DVT is unnecessary. Ultrasound should be considered for patients with potent VTE risk factors. If symptoms do not improve, or if they worsen despite use of antibiotics, clinicians should be alert to potential anchoring bias and consider DVT. The Wells clinical prediction rule overestimates the incidence of DVT in cellulitis and has little value in this setting.

Disclosure

Nothing to report.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook and don’t forget to “Like It” on Facebook or retweet it on Twitter. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 50-year-old man presented to the emergency department with a 3-day-old cut on his anterior right shin. Associated redness, warmth, pain, and swelling had progressed. The patient had no history of prior DVT or pulmonary embolism (PE). His temperature was 38.5°C, and his white blood cell count of 18,000. On review of systems, he denied shortness of breath and chest pain. He was diagnosed with cellulitis and administered intravenous fluids and cefazolin. The clinician wondered whether to perform lower extremity ultrasound to rule out concurrent DVT.

WHY YOU MIGHT THINK ULTRASOUND IS HELPFUL IN RULING OUT DVT IN CELLULITIS

Lower extremity cellulitis, a common infection of the skin and subcutaneous tissues, is characterized by unilateral erythema, pain, warmth, and swelling. The infection usually follows a skin breach that allows bacteria to enter. DVT may present similarly, and symptoms can include mild leukocytosis and elevated temperature. Because of the clinical similarities, clinicians often order compression ultrasound of the extremity to rule out concurrent DVT in cellulitis. Further impetus for testing stems from fear of the potential complications of untreated DVT, including post-thrombotic syndrome, chronic venous insufficiency, and venous ulceration. A subsequent PE can be fatal, or can cause significant morbidity, including chronic VTE with associated pulmonary hypertension. An estimated quarter of all PEs present as sudden death.¹

WHY ULTRASOUND IS NOT HELPFUL IN THIS SETTING

Studies have shown that ultrasound is ordered for 16% to 73% of patients with a cellulitis diagnosis.^2,3 Although testing is commonly performed, a meta-analysis of 9 studies of cellulitis patients who underwent ultrasound testing for concurrent DVT revealed a low pooled incidence of total DVT (3.1%) and proximal DVT (2.1%).⁴ Maze et al.² retrospectively reviewed 1515 cellulitis cases (identified by International Classification of Diseases, Ninth Revision codes) at a single center in New Zealand over 3 years. Of the 1515 patients, 240 (16%) had ultrasound performed, and only 3 (1.3%) were found to have DVT. Two of the 3 had active malignancy, and the third had injected battery acid into the area. In a 5-year retrospective cohort study at a Veterans Administration hospital in Connecticut, Gunderson and Chang³ reviewed the cases of 183 patients with cellulitis and found ultrasound testing commonly performed (73% of cases) to assess for DVT. Only 1 patient (<1%) was diagnosed with new DVT in the ipsilateral leg, and acute DVT was diagnosed in the contralateral leg of 2 other patients. Overall, these studies indicate the incidence of concurrent DVT in cellulitis is low, regardless of the frequency of ultrasound testing.

Although the cost of a single ultrasound test is not prohibitive, annual total costs hospital-wide and nationally are large. In the United States, the charge for a unilateral duplex ultrasound of the extremity ranges from $260 to $1300, and there is an additional charge for interpretation by a radiologist.⁵ In a retrospective study spanning 3.5 years and involving 2 community hospitals in Michigan, an estimated $290,000 was spent on ultrasound tests defined as unnecessary for patients with cellulitis.⁶ A limitation of the study was defining a test as unnecessary based on its result being negative.

DOES WELLS SCORE WITH D-DIMER HELP DEFINE A LOW-RISK POPULATION?

The Wells clinical prediction rule is commonly used to assess the pretest probability of DVT in patients presenting with unilateral leg symptoms. The Wells score is often combined with D-dimer testing to help determine whether ultrasound is necessary. Studies of patients with suspected DVT have found that those considered low risk according to the Wells criteria had a 6.5% incidence of DVT.⁷ However, the predictive value is lower in the setting of presumed cellulitis. In a prospective cohort study of 200 patients with cellulitis, Maze et al.⁸ reported that use of the Wells score with D-dimer testing overestimated the DVT risk. D-dimer level was elevated for 74% of patients, and 20.5% were high-risk by Wells criteria. An algorithm determined that—among patients with a high-risk Wells score, a positive D-dimer result, or both—only 1 (0.5%) was diagnosed with ipsilateral DVT after ultrasound testing. Two patients were diagnosed with DVT in the contralateral leg. These results suggest that a strategy that incorporates the Wells score and D-dimer testing in the setting of acute cellulitis provides little value. The authors concluded that, in the absence of a known hypercoagulable state, ultrasound is not warranted. However, their study did not assess whether there are any specific hypercoagulable states for which further testing may be indicated.

WHEN MIGHT ULTRASOUND BE HELPFUL IN CELLULITIS?

Investigators have described possible DVT risk factors in patients with cellulitis, but definitive associations are lacking because of the insufficient number of patients studied.^8,9 The most consistently identified DVT risk factor is history of previous thromboembolism. In a retrospective analysis of patients with cellulitis, Afzal et al.⁶ found that, of the 66.8% who underwent ultrasound testing, 5.5% were identified as having concurrent DVT. The authors performed univariate analyses of 15 potential risk factors, including active malignancy, oral contraceptive pill use, recent hospitalization, and surgery. A higher incidence of DVT was found for patients with history of VTE (odds ratio [OR], 5.7; 95% confidence interval [CI], 2.3-13.7), calf swelling (OR, 4.5; 95% CI, 1.3-15.8), CVA (OR, 3.5; 95% CI, 1.2-10.1), or hypertension (OR, 3.5; 95% CI, 0.98-12.2). Given the wide confidence intervals, paucity of studies, and lack of definitive data in the setting of cellulitis, clinicians may want to consider the risk factors established in larger trials in other settings, including known immobility (OR, <2); thrombophilia, CHF, and CVA with hemiparesis (OR, 2-9); and trauma and recent surgery (OR, >10).¹⁰

WHAT YOU SHOULD DO INSTEAD

As the incidence of concurrent VTE in patients with cellulitis is low, the essential step is to make a clear diagnosis of cellulitis based on its established signs and symptoms. A 2-center trial of 145 patients found that cellulitis was diagnosed accurately by general medicine and emergency medicine physicians 72% of the time, with evaluation by dermatologists and infectious disease specialists used as the gold standard. Only 5% of the misdiagnosed patients were diagnosed with DVT; stasis dermatitis was the most common alternative diagnosis. Taking a thorough history may elicit risk factors consistent with cellulitis, such as a recent injury with a break in the skin. On examination, cellulitis should be suspected for patients with fever and localized pain, redness, swelling, and warmth—the cardinal signs of dolor, rubor, tumor, and calor. An injury or entry site and leukocytosis also support the diagnosis of cellulitis. Distinct margins of erythema on the skin are highly suspicious for erysipelas.¹¹ Other physical findings (eg, laceration, purulent drainage, lymphangitic spread, fluctuating mass) also are consistent with a diagnosis of cellulitis.

The patient’s history is also essential in determining whether any DVT risk factors are present. Past medical history of VTE or CVA, or recent history of surgery, immobility, or trauma, should alert the clinician to the possibility of DVT. Family history of VTE increases the likelihood of DVT. Acute shortness of breath or chest pain in the setting of concerning lower extremity findings for DVT should raise concern for DVT and concurrent PE.

If the classic features of cellulitis are present, empiric antibiotics should be initiated. Routine ultrasound testing for all patients with cellulitis is of low value. However, as the incidence of DVT in this population is not negligible, those with VTE risk factors should be targeted for testing. Studies in the setting of cellulitis provide little guidance regarding specific risk factors that can be used to determine who should undergo further testing. Given this limitation, we suggest that clinicians incorporate into their decision making the well-established VTE risk factors identified for large populations studied in other settings, such as the postoperative period. Specifically, clinicians should consider ultrasound testing for patients with cellulitis and prior history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery.^10-12 Ultrasound should also be considered for patients with cellulitis that does not improve and for patients whose localized symptoms worsen despite use of antibiotics.

• Do not routinely perform ultrasound to rule out concurrent DVT in cases of cellulitis.

• Consider compression ultrasound if there is a history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery. Also consider it for patients who do not respond to antibiotics.

• In cases of cellulitis, avoid use of the Wells score alone or with D-dimer testing, as it likely overestimates the DVT risk.

The current evidence shows that, for most patients with cellulitis, routine ultrasound testing for DVT is unnecessary. Ultrasound should be considered for patients with potent VTE risk factors. If symptoms do not improve, or if they worsen despite use of antibiotics, clinicians should be alert to potential anchoring bias and consider DVT. The Wells clinical prediction rule overestimates the incidence of DVT in cellulitis and has little value in this setting.

Disclosure

Nothing to report.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook and don’t forget to “Like It” on Facebook or retweet it on Twitter. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 50-year-old man presented to the emergency department with a 3-day-old cut on his anterior right shin. Associated redness, warmth, pain, and swelling had progressed. The patient had no history of prior DVT or pulmonary embolism (PE). His temperature was 38.5°C, and his white blood cell count of 18,000. On review of systems, he denied shortness of breath and chest pain. He was diagnosed with cellulitis and administered intravenous fluids and cefazolin. The clinician wondered whether to perform lower extremity ultrasound to rule out concurrent DVT.

WHY YOU MIGHT THINK ULTRASOUND IS HELPFUL IN RULING OUT DVT IN CELLULITIS

Lower extremity cellulitis, a common infection of the skin and subcutaneous tissues, is characterized by unilateral erythema, pain, warmth, and swelling. The infection usually follows a skin breach that allows bacteria to enter. DVT may present similarly, and symptoms can include mild leukocytosis and elevated temperature. Because of the clinical similarities, clinicians often order compression ultrasound of the extremity to rule out concurrent DVT in cellulitis. Further impetus for testing stems from fear of the potential complications of untreated DVT, including post-thrombotic syndrome, chronic venous insufficiency, and venous ulceration. A subsequent PE can be fatal, or can cause significant morbidity, including chronic VTE with associated pulmonary hypertension. An estimated quarter of all PEs present as sudden death.¹

WHY ULTRASOUND IS NOT HELPFUL IN THIS SETTING

Studies have shown that ultrasound is ordered for 16% to 73% of patients with a cellulitis diagnosis.^2,3 Although testing is commonly performed, a meta-analysis of 9 studies of cellulitis patients who underwent ultrasound testing for concurrent DVT revealed a low pooled incidence of total DVT (3.1%) and proximal DVT (2.1%).⁴ Maze et al.² retrospectively reviewed 1515 cellulitis cases (identified by International Classification of Diseases, Ninth Revision codes) at a single center in New Zealand over 3 years. Of the 1515 patients, 240 (16%) had ultrasound performed, and only 3 (1.3%) were found to have DVT. Two of the 3 had active malignancy, and the third had injected battery acid into the area. In a 5-year retrospective cohort study at a Veterans Administration hospital in Connecticut, Gunderson and Chang³ reviewed the cases of 183 patients with cellulitis and found ultrasound testing commonly performed (73% of cases) to assess for DVT. Only 1 patient (<1%) was diagnosed with new DVT in the ipsilateral leg, and acute DVT was diagnosed in the contralateral leg of 2 other patients. Overall, these studies indicate the incidence of concurrent DVT in cellulitis is low, regardless of the frequency of ultrasound testing.

Although the cost of a single ultrasound test is not prohibitive, annual total costs hospital-wide and nationally are large. In the United States, the charge for a unilateral duplex ultrasound of the extremity ranges from $260 to $1300, and there is an additional charge for interpretation by a radiologist.⁵ In a retrospective study spanning 3.5 years and involving 2 community hospitals in Michigan, an estimated $290,000 was spent on ultrasound tests defined as unnecessary for patients with cellulitis.⁶ A limitation of the study was defining a test as unnecessary based on its result being negative.

DOES WELLS SCORE WITH D-DIMER HELP DEFINE A LOW-RISK POPULATION?

The Wells clinical prediction rule is commonly used to assess the pretest probability of DVT in patients presenting with unilateral leg symptoms. The Wells score is often combined with D-dimer testing to help determine whether ultrasound is necessary. Studies of patients with suspected DVT have found that those considered low risk according to the Wells criteria had a 6.5% incidence of DVT.⁷ However, the predictive value is lower in the setting of presumed cellulitis. In a prospective cohort study of 200 patients with cellulitis, Maze et al.⁸ reported that use of the Wells score with D-dimer testing overestimated the DVT risk. D-dimer level was elevated for 74% of patients, and 20.5% were high-risk by Wells criteria. An algorithm determined that—among patients with a high-risk Wells score, a positive D-dimer result, or both—only 1 (0.5%) was diagnosed with ipsilateral DVT after ultrasound testing. Two patients were diagnosed with DVT in the contralateral leg. These results suggest that a strategy that incorporates the Wells score and D-dimer testing in the setting of acute cellulitis provides little value. The authors concluded that, in the absence of a known hypercoagulable state, ultrasound is not warranted. However, their study did not assess whether there are any specific hypercoagulable states for which further testing may be indicated.

WHEN MIGHT ULTRASOUND BE HELPFUL IN CELLULITIS?

Investigators have described possible DVT risk factors in patients with cellulitis, but definitive associations are lacking because of the insufficient number of patients studied.^8,9 The most consistently identified DVT risk factor is history of previous thromboembolism. In a retrospective analysis of patients with cellulitis, Afzal et al.⁶ found that, of the 66.8% who underwent ultrasound testing, 5.5% were identified as having concurrent DVT. The authors performed univariate analyses of 15 potential risk factors, including active malignancy, oral contraceptive pill use, recent hospitalization, and surgery. A higher incidence of DVT was found for patients with history of VTE (odds ratio [OR], 5.7; 95% confidence interval [CI], 2.3-13.7), calf swelling (OR, 4.5; 95% CI, 1.3-15.8), CVA (OR, 3.5; 95% CI, 1.2-10.1), or hypertension (OR, 3.5; 95% CI, 0.98-12.2). Given the wide confidence intervals, paucity of studies, and lack of definitive data in the setting of cellulitis, clinicians may want to consider the risk factors established in larger trials in other settings, including known immobility (OR, <2); thrombophilia, CHF, and CVA with hemiparesis (OR, 2-9); and trauma and recent surgery (OR, >10).¹⁰

WHAT YOU SHOULD DO INSTEAD

As the incidence of concurrent VTE in patients with cellulitis is low, the essential step is to make a clear diagnosis of cellulitis based on its established signs and symptoms. A 2-center trial of 145 patients found that cellulitis was diagnosed accurately by general medicine and emergency medicine physicians 72% of the time, with evaluation by dermatologists and infectious disease specialists used as the gold standard. Only 5% of the misdiagnosed patients were diagnosed with DVT; stasis dermatitis was the most common alternative diagnosis. Taking a thorough history may elicit risk factors consistent with cellulitis, such as a recent injury with a break in the skin. On examination, cellulitis should be suspected for patients with fever and localized pain, redness, swelling, and warmth—the cardinal signs of dolor, rubor, tumor, and calor. An injury or entry site and leukocytosis also support the diagnosis of cellulitis. Distinct margins of erythema on the skin are highly suspicious for erysipelas.¹¹ Other physical findings (eg, laceration, purulent drainage, lymphangitic spread, fluctuating mass) also are consistent with a diagnosis of cellulitis.

The patient’s history is also essential in determining whether any DVT risk factors are present. Past medical history of VTE or CVA, or recent history of surgery, immobility, or trauma, should alert the clinician to the possibility of DVT. Family history of VTE increases the likelihood of DVT. Acute shortness of breath or chest pain in the setting of concerning lower extremity findings for DVT should raise concern for DVT and concurrent PE.

If the classic features of cellulitis are present, empiric antibiotics should be initiated. Routine ultrasound testing for all patients with cellulitis is of low value. However, as the incidence of DVT in this population is not negligible, those with VTE risk factors should be targeted for testing. Studies in the setting of cellulitis provide little guidance regarding specific risk factors that can be used to determine who should undergo further testing. Given this limitation, we suggest that clinicians incorporate into their decision making the well-established VTE risk factors identified for large populations studied in other settings, such as the postoperative period. Specifically, clinicians should consider ultrasound testing for patients with cellulitis and prior history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery.^10-12 Ultrasound should also be considered for patients with cellulitis that does not improve and for patients whose localized symptoms worsen despite use of antibiotics.

• Do not routinely perform ultrasound to rule out concurrent DVT in cases of cellulitis.

• Consider compression ultrasound if there is a history of VTE; immobility; thrombophilia, CHF, and CVA with hemiparesis; or trauma and recent surgery. Also consider it for patients who do not respond to antibiotics.

• In cases of cellulitis, avoid use of the Wells score alone or with D-dimer testing, as it likely overestimates the DVT risk.

The current evidence shows that, for most patients with cellulitis, routine ultrasound testing for DVT is unnecessary. Ultrasound should be considered for patients with potent VTE risk factors. If symptoms do not improve, or if they worsen despite use of antibiotics, clinicians should be alert to potential anchoring bias and consider DVT. The Wells clinical prediction rule overestimates the incidence of DVT in cellulitis and has little value in this setting.

Disclosure

Nothing to report.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook and don’t forget to “Like It” on Facebook or retweet it on Twitter. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing [email protected].

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant. The bolded text represents the patient’s case. Each paragraph that follows represents the discussant’s thoughts.

Two weeks after undergoing a below-knee amputation (BKA) and 10 days after being discharged to a skilled nursing facility (SNF), an 87-year-old man returned to the emergency department (ED) for evaluation of somnolence and altered mental state. In the ED, he was disoriented and unable to provide a detailed history.

The differential diagnosis for acute confusion and altered consciousness is broad. Initial possibilities include toxic-metabolic abnormalities, medication side effects, and infections. Urinary tract infection, pneumonia, and surgical-site infection should be assessed for first, as they are common causes of postoperative altered mentation. Next to be considered are subclinical seizure, ischemic stroke, and infectious encephalitis or meningitis, along with hemorrhagic stroke and subdural hematoma.

During initial assessment, the clinician should ascertain baseline mental state, the timeline of the change in mental status, recent medication changes, history of substance abuse, and concern about any recent trauma, such as a fall. Performing the physical examination, the clinician should assess vital signs and then focus on identifying localizing neurologic deficits.

First steps in the work-up include a complete metabolic panel, complete blood cell count, urinalysis with culture, and a urine toxicology screen. If the patient has a “toxic” appearance, blood cultures should be obtained. An electrocardiogram should be used to screen for drug toxicity or evidence of cardiac ischemia. If laboratory test results do not reveal an obvious infectious or metabolic cause, a noncontrast computed tomography (CT) of the head should be obtained. In terms of early interventions, a low glucose level should be treated with thiamine and then glucose, and naloxone should be given if there is any suspicion of narcotic overdose.

More history was obtained from the patient’s records. The BKA was performed to address a nonhealing transmetatarsal amputation. Two months earlier, the transmetatarsal amputation had been performed as treatment for a diabetic forefoot ulcer with chronic osteomyelitis. The patient’s post-BKA course was uncomplicated. He was started on intravenous (IV) ertapenem on postoperative day 1, and on postoperative day 4 was discharged to the SNF to complete a 6-week course of antibiotics for osteomyelitis. Past medical history included paroxysmal atrial fibrillation, coronary artery disease, congestive heart failure (ejection fraction 40%), and type 2 diabetes mellitus. Medications given at the SNF were oxycodone, acetaminophen, cholecalciferol, melatonin, digoxin, ondansetron, furosemide, gabapentin, correctional insulin, tamsulosin, senna, docusate, warfarin, and metoprolol. While there, the patient’s family expressed concern about his diminishing “mental ability.” They reported he had been fully alert and oriented on arrival at the SNF, and living independently with his wife before the BKA. Then, a week before the ED presentation, he started becoming more somnolent and forgetful. The gabapentin and oxycodone dosages were reduced to minimize their sedative effects, but he showed no improvement. At the SNF, a somnolence work-up was not performed.

Several of the patient’s medications can contribute to altered mental state. Ertapenem can cause seizures as well as profound mental status changes, though these are more likely in the setting of poor renal function. The mental status changes were noticed about a week into the patient’s course of antibiotics, which suggests a possible temporal correlation with the initiation of ertapenem. An electroencephalogram is required to diagnose nonconvulsive seizure activity. Narcotic overdose should still be considered, despite the recent reduction in oxycodone dosage. Digoxin toxicity, though less likely when the dose is stable and there are no changes in renal function, can cause a confused state. Concurrent use of furosemide could potentiate the toxic effects of digoxin.

Non-medication-related concerns include hypoglycemia, hyperglycemia, and, given his history of atrial fibrillation, cardioembolic stroke. Although generalized confusion is not a common manifestation of stroke, a thalamic stroke can alter mental state but be easily missed if not specifically considered. Additional lab work-up should include a digoxin level and, since he is taking warfarin, a prothrombin time/international normalized ratio (PT/INR). If the initial laboratory studies and head CT do not explain the altered mental state, magnetic resonance imaging (MRI) of the brain should be performed to further assess for stroke.

On physical examination in the ED, the patient was resting comfortably with eyes closed, and arousing to voice. He obeyed commands and participated in the examination. His Glasgow Coma Scale score was 13; temperature, 36.8°C, heart rate, 80 beats per minute; respiratory rate, 16 breaths per minute; blood pressure, 90/57 mm Hg; and 100% peripheral capillary oxygen saturation while breathing ambient air. He appeared well developed. His heart rhythm was irregularly irregular, without murmurs, rubs, or gallops. Respiratory and abdominal examination findings were normal. The left BKA incision was well approximated, with no drainage, dehiscence, fluctuance, or erythema. On neurologic examination, the patient was intermittently oriented only to self. Pupils were equal, round, and reactive to light; extraocular movements were intact; face was symmetric; tongue was midline; sensation on face was equal bilaterally; and shoulder shrug was intact. Strength was 5/5 and symmetric in the elbow and hip and 5/5 in the right knee and ankle (not tested on left because of BKA). Deep tendon reflexes were 3+ and symmetrical at the biceps, brachioradialis, and triceps tendons and 3+ in the right patellar and Achilles tendons. Sensation was intact and symmetrical in the upper and lower extremities. The patient’s speech was slow and slurred, and his answers were unrelated to the questions being asked.

The patient’s mental state is best described as lethargic. As he is only intermittently oriented, he meets the criteria for delirium. He is not obtunded or comatose, and his pupils are at least reactive, not pinpoint, so narcotic overdose is less likely. Thalamic stroke remains in the differential diagnosis; despite the seemingly symmetrical sensation examination, hemisensory deficits cannot be definitively ruled out given the patient’s mental state. A rare entity such as carcinomatosis meningitis or another diffuse, infiltrative neoplastic process could be causing his condition. However, because focal deficits other than abnormal speech and diffuse hyperreflexia are absent, toxic, infectious, or metabolic causes are more likely than structural abnormalities. Still possible is a medication toxicity, such as ertapenem toxicity or, less likely, digoxin toxicity. In terms of infectious possibilities, urinary tract infection could certainly present in this fashion, especially if the patient had a somewhat low neurologic reserve at baseline, and hypotension could be secondary to sepsis. Encephalitis or meningitis remains in the differential diagnosis, though the patient appears nontoxic, and therefore a bacterial etiology is very unlikely.

The patient’s hyperreflexia may be an important clue. Although the strength of his reflexes at baseline is unknown, seizures can cause transiently increased reflexes as well as a confused, lethargic mental state. Reflexes can also be increased by a drug overdose that has caused serotonin syndrome. Of the patient’s medications, only ondansetron can cause this reaction. Hyperthyroidism can cause brisk reflexes and confusion, though more typically it causes agitated confusion. A thyroid-stimulating hormone level should be added to the initial laboratory panel.

A complete blood count revealed white blood cell count 11.86 K/uL with neutrophilic predominance and immature granulocytes, hemoglobin 11.5 g/dL, and platelet count 323 K/uL. Serum sodium was 141 mEq/L, potassium 4.2 mEq/L, chloride 103 mEq/L, bicarbonate 30 mEq/L, creatinine 1.14 mg/dL (prior baseline of 0.8-1.0 mg/dL), blood urea nitrogen 26 mg/dL, blood glucose 159 mg/dL, and calcium 9.1 mg/dL. His digoxin level was 1.3 ng/mL (reference range 0.5-1.9 mg/mL) and troponin was undetectable. INR was 2.7 and partial thromboplastin time (PTT) 60 seconds. Vitamin B₁₂ level was 674 pg/mL (reference range >180). A urinalysis had 1+ hyaline casts and was negative for nitrites, leukocyte esterase, blood, and bacteria. An ECG revealed atrial fibrillation with a ventricular rate of 80 beats per minute. A chest radiograph showed clear lung fields. A CT of the head without IV contrast had no evidence of an acute intracranial abnormality. In the ED, 1 liter of IV normal saline was given and blood pressure improved to 127/72 mm Hg.

The head CT does not show intracranial bleeding, and, though it is reassuring that INR is in the therapeutic range, ischemic stroke must remain in the differential diagnosis. Sepsis is less likely given that the criteria for systemic inflammatory response syndrome are not met, and hypotension was rapidly corrected with administration of IV fluids. Urinary tract infection was ruled out with the negative urinalysis. Subclinical seizures remain possible, as does medication-related or other toxicity. A medication overdose, intentional or otherwise, should also be considered.

The patient was admitted to the hospital. On reassessment by the inpatient team, he was oriented only to self, frequently falling asleep, and not recalling earlier conversations when aroused. His speech remained slurred and difficult to understand. Neurologic examination findings were unchanged since the ED examination. On additional cerebellar examination, he had dysmetria with finger-to-nose testing bilaterally and dysdiadochokinesia (impaired rapid alternating movements) of the left hand.

His handedness is not mentioned; the dysdiadochokinesia of the left hand may reflect the patient’s being right-handed, or may signify a focal cerebellar lesion. The cerebellum is also implicated by the bilateral dysmetria. Persistent somnolence in the absence of CT findings suggests a metabolic or infectious process. Metabolic processes that can cause bilateral cerebellar ataxia and somnolence include overdose of a drug or medication. Use of alcohol or a medication such as phenytoin, valproic acid, or a benzodiazepine can cause the symptoms in this case, but was not reported by the family, and there was no documentation of it in the SNF records. Wernicke encephalopathy is rare and is not well supported by the patient’s presentation but should be considered, as it can be easily treated with thiamine. Meningoencephalitis affecting the cerebellum remains possible, but infection is less likely. Both electroencephalogram and brain MRI should be performed, with a specific interest in possible cerebellar lesions. If the MRI is unremarkable, a lumbar puncture should be performed to assess opening pressure and investigate for infectious etiologies.

MRI of the brain showed age-related volume loss and nonspecific white matter disease without acute changes. Lack of a clear explanation for the neurologic findings led to suspicion of a medication side effect. Ertapenem was stopped on admission because it has been reported to rarely cause altered mental status. IV moxifloxacin was started for the osteomyelitis. Over the next 2 days, symptoms began resolving; within 24 hours of ertapenem discontinuation, the patient was awake, alert, and talkative. On examination, he remained dysarthric but was no longer dysmetric. Within 48 hours, the dysarthria was completely resolved, and he was returned to the SNF to complete a course of IV moxifloxacin.

Among elderly patients presenting to the ED, altered mental status is a common complaint, accounting for 10% to 30% of visits.¹ Medications are a common cause of altered mental status among the elderly and are responsible for 40% of delirium cases.¹ The risk of adverse drug events (ADEs) rises with the number of medications prescribed.^1-3 Among patients older than 60 years, the incidence of polypharmacy (defined as taking >5 prescription medications) increased from roughly 20% in 1999 to 40% in 2012.^4,5 The most common ADEs in the ambulatory setting (25%) are central nervous system (CNS) symptoms, including dizziness, sleep disturbances, and mood changes.⁶ A medication effect should be suspected in any elderly patient presenting with altered mental state.

The present patient developed a constellation of neurologic symptoms after starting ertapenem, one of the carbapenem antibiotics, which is a class of medications that can cause CNS ADEs. Carbapenems are renally cleared, and adjustments must be made for acute or chronic changes in kidney function. Carbapenems are associated with increased risk of seizure; the incidence of seizure with ertapenem is 0.2%.^7,8 Food and Drug Administration postmarketing reports have noted ertapenem can cause somnolence and dyskinesia,⁹ and several case reports have described ertapenem-associated CNS side effects, including psychosis and encephalopathy.^10-13 Symptoms and examination findings can include confusion, disorientation, garbled speech, dysphagia, hallucinations, miosis, myoclonus, tremor, and agitation.^10-13 Although reports of dysmetria and dysdiadochokinesia are lacking, suspicion of an ADE in this case was heightened by the timing of the exposure and the absence of alternative infectious, metabolic, and vascular explanations for bilateral cerebellar dysfunction.

The Naranjo Adverse Drug Reaction (ADR) scale may help clinicians differentiate ADEs from other etiologies of symptoms. It uses 10 weighted questions (Table) to estimate the probability that an adverse clinical event is caused by a drug reaction.¹⁴ The present case was assigned 1 point for prior reports of neurologic ADEs associated with ertapenem, 2 for the temporal association, 1 for resolution after medication withdrawal, 2 for lack of alternative causes, and 1 for objective evidence of neurologic dysfunction—for a total of 7 points, indicating ertapenem was probably the cause of the patient’s neurologic symptoms. Of 4 prior cases in which carbapenem toxicity was suspected and the Naranjo scale was used, 3 found a probable relationship, and the fourth a highly probable one.^10,12 Confusion, disorientation, hallucinations, tangential thoughts, and garbled speech were reported in the 3 probable cases of ADEs. In the highly probable case, tangential thoughts, garbled speech, and miosis were noted on examination, and these findings returned after re-exposure to ertapenem. Of note, these ADEs occurred in patients with normal and abnormal renal function, and in middle-aged and elderly patients.^10,11,13

• Clinicians should include rare adverse effects of common medications in the differential diagnosis.

• The Naranjo score is a validated tool that can be used to systematically assess the probability of an adverse drug effect at the bedside.

• The presentation of ertapenem-associated neurotoxicity may include features of bilateral cerebellar dysfunction.

Disclosure

Nothing to report.

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant. The bolded text represents the patient’s case. Each paragraph that follows represents the discussant’s thoughts.

Two weeks after undergoing a below-knee amputation (BKA) and 10 days after being discharged to a skilled nursing facility (SNF), an 87-year-old man returned to the emergency department (ED) for evaluation of somnolence and altered mental state. In the ED, he was disoriented and unable to provide a detailed history.

The differential diagnosis for acute confusion and altered consciousness is broad. Initial possibilities include toxic-metabolic abnormalities, medication side effects, and infections. Urinary tract infection, pneumonia, and surgical-site infection should be assessed for first, as they are common causes of postoperative altered mentation. Next to be considered are subclinical seizure, ischemic stroke, and infectious encephalitis or meningitis, along with hemorrhagic stroke and subdural hematoma.

During initial assessment, the clinician should ascertain baseline mental state, the timeline of the change in mental status, recent medication changes, history of substance abuse, and concern about any recent trauma, such as a fall. Performing the physical examination, the clinician should assess vital signs and then focus on identifying localizing neurologic deficits.

First steps in the work-up include a complete metabolic panel, complete blood cell count, urinalysis with culture, and a urine toxicology screen. If the patient has a “toxic” appearance, blood cultures should be obtained. An electrocardiogram should be used to screen for drug toxicity or evidence of cardiac ischemia. If laboratory test results do not reveal an obvious infectious or metabolic cause, a noncontrast computed tomography (CT) of the head should be obtained. In terms of early interventions, a low glucose level should be treated with thiamine and then glucose, and naloxone should be given if there is any suspicion of narcotic overdose.

More history was obtained from the patient’s records. The BKA was performed to address a nonhealing transmetatarsal amputation. Two months earlier, the transmetatarsal amputation had been performed as treatment for a diabetic forefoot ulcer with chronic osteomyelitis. The patient’s post-BKA course was uncomplicated. He was started on intravenous (IV) ertapenem on postoperative day 1, and on postoperative day 4 was discharged to the SNF to complete a 6-week course of antibiotics for osteomyelitis. Past medical history included paroxysmal atrial fibrillation, coronary artery disease, congestive heart failure (ejection fraction 40%), and type 2 diabetes mellitus. Medications given at the SNF were oxycodone, acetaminophen, cholecalciferol, melatonin, digoxin, ondansetron, furosemide, gabapentin, correctional insulin, tamsulosin, senna, docusate, warfarin, and metoprolol. While there, the patient’s family expressed concern about his diminishing “mental ability.” They reported he had been fully alert and oriented on arrival at the SNF, and living independently with his wife before the BKA. Then, a week before the ED presentation, he started becoming more somnolent and forgetful. The gabapentin and oxycodone dosages were reduced to minimize their sedative effects, but he showed no improvement. At the SNF, a somnolence work-up was not performed.

Several of the patient’s medications can contribute to altered mental state. Ertapenem can cause seizures as well as profound mental status changes, though these are more likely in the setting of poor renal function. The mental status changes were noticed about a week into the patient’s course of antibiotics, which suggests a possible temporal correlation with the initiation of ertapenem. An electroencephalogram is required to diagnose nonconvulsive seizure activity. Narcotic overdose should still be considered, despite the recent reduction in oxycodone dosage. Digoxin toxicity, though less likely when the dose is stable and there are no changes in renal function, can cause a confused state. Concurrent use of furosemide could potentiate the toxic effects of digoxin.

Non-medication-related concerns include hypoglycemia, hyperglycemia, and, given his history of atrial fibrillation, cardioembolic stroke. Although generalized confusion is not a common manifestation of stroke, a thalamic stroke can alter mental state but be easily missed if not specifically considered. Additional lab work-up should include a digoxin level and, since he is taking warfarin, a prothrombin time/international normalized ratio (PT/INR). If the initial laboratory studies and head CT do not explain the altered mental state, magnetic resonance imaging (MRI) of the brain should be performed to further assess for stroke.

On physical examination in the ED, the patient was resting comfortably with eyes closed, and arousing to voice. He obeyed commands and participated in the examination. His Glasgow Coma Scale score was 13; temperature, 36.8°C, heart rate, 80 beats per minute; respiratory rate, 16 breaths per minute; blood pressure, 90/57 mm Hg; and 100% peripheral capillary oxygen saturation while breathing ambient air. He appeared well developed. His heart rhythm was irregularly irregular, without murmurs, rubs, or gallops. Respiratory and abdominal examination findings were normal. The left BKA incision was well approximated, with no drainage, dehiscence, fluctuance, or erythema. On neurologic examination, the patient was intermittently oriented only to self. Pupils were equal, round, and reactive to light; extraocular movements were intact; face was symmetric; tongue was midline; sensation on face was equal bilaterally; and shoulder shrug was intact. Strength was 5/5 and symmetric in the elbow and hip and 5/5 in the right knee and ankle (not tested on left because of BKA). Deep tendon reflexes were 3+ and symmetrical at the biceps, brachioradialis, and triceps tendons and 3+ in the right patellar and Achilles tendons. Sensation was intact and symmetrical in the upper and lower extremities. The patient’s speech was slow and slurred, and his answers were unrelated to the questions being asked.

The patient’s mental state is best described as lethargic. As he is only intermittently oriented, he meets the criteria for delirium. He is not obtunded or comatose, and his pupils are at least reactive, not pinpoint, so narcotic overdose is less likely. Thalamic stroke remains in the differential diagnosis; despite the seemingly symmetrical sensation examination, hemisensory deficits cannot be definitively ruled out given the patient’s mental state. A rare entity such as carcinomatosis meningitis or another diffuse, infiltrative neoplastic process could be causing his condition. However, because focal deficits other than abnormal speech and diffuse hyperreflexia are absent, toxic, infectious, or metabolic causes are more likely than structural abnormalities. Still possible is a medication toxicity, such as ertapenem toxicity or, less likely, digoxin toxicity. In terms of infectious possibilities, urinary tract infection could certainly present in this fashion, especially if the patient had a somewhat low neurologic reserve at baseline, and hypotension could be secondary to sepsis. Encephalitis or meningitis remains in the differential diagnosis, though the patient appears nontoxic, and therefore a bacterial etiology is very unlikely.

The patient’s hyperreflexia may be an important clue. Although the strength of his reflexes at baseline is unknown, seizures can cause transiently increased reflexes as well as a confused, lethargic mental state. Reflexes can also be increased by a drug overdose that has caused serotonin syndrome. Of the patient’s medications, only ondansetron can cause this reaction. Hyperthyroidism can cause brisk reflexes and confusion, though more typically it causes agitated confusion. A thyroid-stimulating hormone level should be added to the initial laboratory panel.

A complete blood count revealed white blood cell count 11.86 K/uL with neutrophilic predominance and immature granulocytes, hemoglobin 11.5 g/dL, and platelet count 323 K/uL. Serum sodium was 141 mEq/L, potassium 4.2 mEq/L, chloride 103 mEq/L, bicarbonate 30 mEq/L, creatinine 1.14 mg/dL (prior baseline of 0.8-1.0 mg/dL), blood urea nitrogen 26 mg/dL, blood glucose 159 mg/dL, and calcium 9.1 mg/dL. His digoxin level was 1.3 ng/mL (reference range 0.5-1.9 mg/mL) and troponin was undetectable. INR was 2.7 and partial thromboplastin time (PTT) 60 seconds. Vitamin B₁₂ level was 674 pg/mL (reference range >180). A urinalysis had 1+ hyaline casts and was negative for nitrites, leukocyte esterase, blood, and bacteria. An ECG revealed atrial fibrillation with a ventricular rate of 80 beats per minute. A chest radiograph showed clear lung fields. A CT of the head without IV contrast had no evidence of an acute intracranial abnormality. In the ED, 1 liter of IV normal saline was given and blood pressure improved to 127/72 mm Hg.

The head CT does not show intracranial bleeding, and, though it is reassuring that INR is in the therapeutic range, ischemic stroke must remain in the differential diagnosis. Sepsis is less likely given that the criteria for systemic inflammatory response syndrome are not met, and hypotension was rapidly corrected with administration of IV fluids. Urinary tract infection was ruled out with the negative urinalysis. Subclinical seizures remain possible, as does medication-related or other toxicity. A medication overdose, intentional or otherwise, should also be considered.

The patient was admitted to the hospital. On reassessment by the inpatient team, he was oriented only to self, frequently falling asleep, and not recalling earlier conversations when aroused. His speech remained slurred and difficult to understand. Neurologic examination findings were unchanged since the ED examination. On additional cerebellar examination, he had dysmetria with finger-to-nose testing bilaterally and dysdiadochokinesia (impaired rapid alternating movements) of the left hand.

His handedness is not mentioned; the dysdiadochokinesia of the left hand may reflect the patient’s being right-handed, or may signify a focal cerebellar lesion. The cerebellum is also implicated by the bilateral dysmetria. Persistent somnolence in the absence of CT findings suggests a metabolic or infectious process. Metabolic processes that can cause bilateral cerebellar ataxia and somnolence include overdose of a drug or medication. Use of alcohol or a medication such as phenytoin, valproic acid, or a benzodiazepine can cause the symptoms in this case, but was not reported by the family, and there was no documentation of it in the SNF records. Wernicke encephalopathy is rare and is not well supported by the patient’s presentation but should be considered, as it can be easily treated with thiamine. Meningoencephalitis affecting the cerebellum remains possible, but infection is less likely. Both electroencephalogram and brain MRI should be performed, with a specific interest in possible cerebellar lesions. If the MRI is unremarkable, a lumbar puncture should be performed to assess opening pressure and investigate for infectious etiologies.

MRI of the brain showed age-related volume loss and nonspecific white matter disease without acute changes. Lack of a clear explanation for the neurologic findings led to suspicion of a medication side effect. Ertapenem was stopped on admission because it has been reported to rarely cause altered mental status. IV moxifloxacin was started for the osteomyelitis. Over the next 2 days, symptoms began resolving; within 24 hours of ertapenem discontinuation, the patient was awake, alert, and talkative. On examination, he remained dysarthric but was no longer dysmetric. Within 48 hours, the dysarthria was completely resolved, and he was returned to the SNF to complete a course of IV moxifloxacin.

Among elderly patients presenting to the ED, altered mental status is a common complaint, accounting for 10% to 30% of visits.¹ Medications are a common cause of altered mental status among the elderly and are responsible for 40% of delirium cases.¹ The risk of adverse drug events (ADEs) rises with the number of medications prescribed.^1-3 Among patients older than 60 years, the incidence of polypharmacy (defined as taking >5 prescription medications) increased from roughly 20% in 1999 to 40% in 2012.^4,5 The most common ADEs in the ambulatory setting (25%) are central nervous system (CNS) symptoms, including dizziness, sleep disturbances, and mood changes.⁶ A medication effect should be suspected in any elderly patient presenting with altered mental state.

The present patient developed a constellation of neurologic symptoms after starting ertapenem, one of the carbapenem antibiotics, which is a class of medications that can cause CNS ADEs. Carbapenems are renally cleared, and adjustments must be made for acute or chronic changes in kidney function. Carbapenems are associated with increased risk of seizure; the incidence of seizure with ertapenem is 0.2%.^7,8 Food and Drug Administration postmarketing reports have noted ertapenem can cause somnolence and dyskinesia,⁹ and several case reports have described ertapenem-associated CNS side effects, including psychosis and encephalopathy.^10-13 Symptoms and examination findings can include confusion, disorientation, garbled speech, dysphagia, hallucinations, miosis, myoclonus, tremor, and agitation.^10-13 Although reports of dysmetria and dysdiadochokinesia are lacking, suspicion of an ADE in this case was heightened by the timing of the exposure and the absence of alternative infectious, metabolic, and vascular explanations for bilateral cerebellar dysfunction.

The Naranjo Adverse Drug Reaction (ADR) scale may help clinicians differentiate ADEs from other etiologies of symptoms. It uses 10 weighted questions (Table) to estimate the probability that an adverse clinical event is caused by a drug reaction.¹⁴ The present case was assigned 1 point for prior reports of neurologic ADEs associated with ertapenem, 2 for the temporal association, 1 for resolution after medication withdrawal, 2 for lack of alternative causes, and 1 for objective evidence of neurologic dysfunction—for a total of 7 points, indicating ertapenem was probably the cause of the patient’s neurologic symptoms. Of 4 prior cases in which carbapenem toxicity was suspected and the Naranjo scale was used, 3 found a probable relationship, and the fourth a highly probable one.^10,12 Confusion, disorientation, hallucinations, tangential thoughts, and garbled speech were reported in the 3 probable cases of ADEs. In the highly probable case, tangential thoughts, garbled speech, and miosis were noted on examination, and these findings returned after re-exposure to ertapenem. Of note, these ADEs occurred in patients with normal and abnormal renal function, and in middle-aged and elderly patients.^10,11,13

• Clinicians should include rare adverse effects of common medications in the differential diagnosis.

• The Naranjo score is a validated tool that can be used to systematically assess the probability of an adverse drug effect at the bedside.

• The presentation of ertapenem-associated neurotoxicity may include features of bilateral cerebellar dysfunction.

Disclosure

Nothing to report.

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant. The bolded text represents the patient’s case. Each paragraph that follows represents the discussant’s thoughts.

Two weeks after undergoing a below-knee amputation (BKA) and 10 days after being discharged to a skilled nursing facility (SNF), an 87-year-old man returned to the emergency department (ED) for evaluation of somnolence and altered mental state. In the ED, he was disoriented and unable to provide a detailed history.

The differential diagnosis for acute confusion and altered consciousness is broad. Initial possibilities include toxic-metabolic abnormalities, medication side effects, and infections. Urinary tract infection, pneumonia, and surgical-site infection should be assessed for first, as they are common causes of postoperative altered mentation. Next to be considered are subclinical seizure, ischemic stroke, and infectious encephalitis or meningitis, along with hemorrhagic stroke and subdural hematoma.

During initial assessment, the clinician should ascertain baseline mental state, the timeline of the change in mental status, recent medication changes, history of substance abuse, and concern about any recent trauma, such as a fall. Performing the physical examination, the clinician should assess vital signs and then focus on identifying localizing neurologic deficits.

First steps in the work-up include a complete metabolic panel, complete blood cell count, urinalysis with culture, and a urine toxicology screen. If the patient has a “toxic” appearance, blood cultures should be obtained. An electrocardiogram should be used to screen for drug toxicity or evidence of cardiac ischemia. If laboratory test results do not reveal an obvious infectious or metabolic cause, a noncontrast computed tomography (CT) of the head should be obtained. In terms of early interventions, a low glucose level should be treated with thiamine and then glucose, and naloxone should be given if there is any suspicion of narcotic overdose.

More history was obtained from the patient’s records. The BKA was performed to address a nonhealing transmetatarsal amputation. Two months earlier, the transmetatarsal amputation had been performed as treatment for a diabetic forefoot ulcer with chronic osteomyelitis. The patient’s post-BKA course was uncomplicated. He was started on intravenous (IV) ertapenem on postoperative day 1, and on postoperative day 4 was discharged to the SNF to complete a 6-week course of antibiotics for osteomyelitis. Past medical history included paroxysmal atrial fibrillation, coronary artery disease, congestive heart failure (ejection fraction 40%), and type 2 diabetes mellitus. Medications given at the SNF were oxycodone, acetaminophen, cholecalciferol, melatonin, digoxin, ondansetron, furosemide, gabapentin, correctional insulin, tamsulosin, senna, docusate, warfarin, and metoprolol. While there, the patient’s family expressed concern about his diminishing “mental ability.” They reported he had been fully alert and oriented on arrival at the SNF, and living independently with his wife before the BKA. Then, a week before the ED presentation, he started becoming more somnolent and forgetful. The gabapentin and oxycodone dosages were reduced to minimize their sedative effects, but he showed no improvement. At the SNF, a somnolence work-up was not performed.

Several of the patient’s medications can contribute to altered mental state. Ertapenem can cause seizures as well as profound mental status changes, though these are more likely in the setting of poor renal function. The mental status changes were noticed about a week into the patient’s course of antibiotics, which suggests a possible temporal correlation with the initiation of ertapenem. An electroencephalogram is required to diagnose nonconvulsive seizure activity. Narcotic overdose should still be considered, despite the recent reduction in oxycodone dosage. Digoxin toxicity, though less likely when the dose is stable and there are no changes in renal function, can cause a confused state. Concurrent use of furosemide could potentiate the toxic effects of digoxin.

Non-medication-related concerns include hypoglycemia, hyperglycemia, and, given his history of atrial fibrillation, cardioembolic stroke. Although generalized confusion is not a common manifestation of stroke, a thalamic stroke can alter mental state but be easily missed if not specifically considered. Additional lab work-up should include a digoxin level and, since he is taking warfarin, a prothrombin time/international normalized ratio (PT/INR). If the initial laboratory studies and head CT do not explain the altered mental state, magnetic resonance imaging (MRI) of the brain should be performed to further assess for stroke.

On physical examination in the ED, the patient was resting comfortably with eyes closed, and arousing to voice. He obeyed commands and participated in the examination. His Glasgow Coma Scale score was 13; temperature, 36.8°C, heart rate, 80 beats per minute; respiratory rate, 16 breaths per minute; blood pressure, 90/57 mm Hg; and 100% peripheral capillary oxygen saturation while breathing ambient air. He appeared well developed. His heart rhythm was irregularly irregular, without murmurs, rubs, or gallops. Respiratory and abdominal examination findings were normal. The left BKA incision was well approximated, with no drainage, dehiscence, fluctuance, or erythema. On neurologic examination, the patient was intermittently oriented only to self. Pupils were equal, round, and reactive to light; extraocular movements were intact; face was symmetric; tongue was midline; sensation on face was equal bilaterally; and shoulder shrug was intact. Strength was 5/5 and symmetric in the elbow and hip and 5/5 in the right knee and ankle (not tested on left because of BKA). Deep tendon reflexes were 3+ and symmetrical at the biceps, brachioradialis, and triceps tendons and 3+ in the right patellar and Achilles tendons. Sensation was intact and symmetrical in the upper and lower extremities. The patient’s speech was slow and slurred, and his answers were unrelated to the questions being asked.

The patient’s mental state is best described as lethargic. As he is only intermittently oriented, he meets the criteria for delirium. He is not obtunded or comatose, and his pupils are at least reactive, not pinpoint, so narcotic overdose is less likely. Thalamic stroke remains in the differential diagnosis; despite the seemingly symmetrical sensation examination, hemisensory deficits cannot be definitively ruled out given the patient’s mental state. A rare entity such as carcinomatosis meningitis or another diffuse, infiltrative neoplastic process could be causing his condition. However, because focal deficits other than abnormal speech and diffuse hyperreflexia are absent, toxic, infectious, or metabolic causes are more likely than structural abnormalities. Still possible is a medication toxicity, such as ertapenem toxicity or, less likely, digoxin toxicity. In terms of infectious possibilities, urinary tract infection could certainly present in this fashion, especially if the patient had a somewhat low neurologic reserve at baseline, and hypotension could be secondary to sepsis. Encephalitis or meningitis remains in the differential diagnosis, though the patient appears nontoxic, and therefore a bacterial etiology is very unlikely.

The patient’s hyperreflexia may be an important clue. Although the strength of his reflexes at baseline is unknown, seizures can cause transiently increased reflexes as well as a confused, lethargic mental state. Reflexes can also be increased by a drug overdose that has caused serotonin syndrome. Of the patient’s medications, only ondansetron can cause this reaction. Hyperthyroidism can cause brisk reflexes and confusion, though more typically it causes agitated confusion. A thyroid-stimulating hormone level should be added to the initial laboratory panel.

A complete blood count revealed white blood cell count 11.86 K/uL with neutrophilic predominance and immature granulocytes, hemoglobin 11.5 g/dL, and platelet count 323 K/uL. Serum sodium was 141 mEq/L, potassium 4.2 mEq/L, chloride 103 mEq/L, bicarbonate 30 mEq/L, creatinine 1.14 mg/dL (prior baseline of 0.8-1.0 mg/dL), blood urea nitrogen 26 mg/dL, blood glucose 159 mg/dL, and calcium 9.1 mg/dL. His digoxin level was 1.3 ng/mL (reference range 0.5-1.9 mg/mL) and troponin was undetectable. INR was 2.7 and partial thromboplastin time (PTT) 60 seconds. Vitamin B₁₂ level was 674 pg/mL (reference range >180). A urinalysis had 1+ hyaline casts and was negative for nitrites, leukocyte esterase, blood, and bacteria. An ECG revealed atrial fibrillation with a ventricular rate of 80 beats per minute. A chest radiograph showed clear lung fields. A CT of the head without IV contrast had no evidence of an acute intracranial abnormality. In the ED, 1 liter of IV normal saline was given and blood pressure improved to 127/72 mm Hg.

The head CT does not show intracranial bleeding, and, though it is reassuring that INR is in the therapeutic range, ischemic stroke must remain in the differential diagnosis. Sepsis is less likely given that the criteria for systemic inflammatory response syndrome are not met, and hypotension was rapidly corrected with administration of IV fluids. Urinary tract infection was ruled out with the negative urinalysis. Subclinical seizures remain possible, as does medication-related or other toxicity. A medication overdose, intentional or otherwise, should also be considered.

The patient was admitted to the hospital. On reassessment by the inpatient team, he was oriented only to self, frequently falling asleep, and not recalling earlier conversations when aroused. His speech remained slurred and difficult to understand. Neurologic examination findings were unchanged since the ED examination. On additional cerebellar examination, he had dysmetria with finger-to-nose testing bilaterally and dysdiadochokinesia (impaired rapid alternating movements) of the left hand.

His handedness is not mentioned; the dysdiadochokinesia of the left hand may reflect the patient’s being right-handed, or may signify a focal cerebellar lesion. The cerebellum is also implicated by the bilateral dysmetria. Persistent somnolence in the absence of CT findings suggests a metabolic or infectious process. Metabolic processes that can cause bilateral cerebellar ataxia and somnolence include overdose of a drug or medication. Use of alcohol or a medication such as phenytoin, valproic acid, or a benzodiazepine can cause the symptoms in this case, but was not reported by the family, and there was no documentation of it in the SNF records. Wernicke encephalopathy is rare and is not well supported by the patient’s presentation but should be considered, as it can be easily treated with thiamine. Meningoencephalitis affecting the cerebellum remains possible, but infection is less likely. Both electroencephalogram and brain MRI should be performed, with a specific interest in possible cerebellar lesions. If the MRI is unremarkable, a lumbar puncture should be performed to assess opening pressure and investigate for infectious etiologies.

MRI of the brain showed age-related volume loss and nonspecific white matter disease without acute changes. Lack of a clear explanation for the neurologic findings led to suspicion of a medication side effect. Ertapenem was stopped on admission because it has been reported to rarely cause altered mental status. IV moxifloxacin was started for the osteomyelitis. Over the next 2 days, symptoms began resolving; within 24 hours of ertapenem discontinuation, the patient was awake, alert, and talkative. On examination, he remained dysarthric but was no longer dysmetric. Within 48 hours, the dysarthria was completely resolved, and he was returned to the SNF to complete a course of IV moxifloxacin.

Among elderly patients presenting to the ED, altered mental status is a common complaint, accounting for 10% to 30% of visits.¹ Medications are a common cause of altered mental status among the elderly and are responsible for 40% of delirium cases.¹ The risk of adverse drug events (ADEs) rises with the number of medications prescribed.^1-3 Among patients older than 60 years, the incidence of polypharmacy (defined as taking >5 prescription medications) increased from roughly 20% in 1999 to 40% in 2012.^4,5 The most common ADEs in the ambulatory setting (25%) are central nervous system (CNS) symptoms, including dizziness, sleep disturbances, and mood changes.⁶ A medication effect should be suspected in any elderly patient presenting with altered mental state.

The present patient developed a constellation of neurologic symptoms after starting ertapenem, one of the carbapenem antibiotics, which is a class of medications that can cause CNS ADEs. Carbapenems are renally cleared, and adjustments must be made for acute or chronic changes in kidney function. Carbapenems are associated with increased risk of seizure; the incidence of seizure with ertapenem is 0.2%.^7,8 Food and Drug Administration postmarketing reports have noted ertapenem can cause somnolence and dyskinesia,⁹ and several case reports have described ertapenem-associated CNS side effects, including psychosis and encephalopathy.^10-13 Symptoms and examination findings can include confusion, disorientation, garbled speech, dysphagia, hallucinations, miosis, myoclonus, tremor, and agitation.^10-13 Although reports of dysmetria and dysdiadochokinesia are lacking, suspicion of an ADE in this case was heightened by the timing of the exposure and the absence of alternative infectious, metabolic, and vascular explanations for bilateral cerebellar dysfunction.

The Naranjo Adverse Drug Reaction (ADR) scale may help clinicians differentiate ADEs from other etiologies of symptoms. It uses 10 weighted questions (Table) to estimate the probability that an adverse clinical event is caused by a drug reaction.¹⁴ The present case was assigned 1 point for prior reports of neurologic ADEs associated with ertapenem, 2 for the temporal association, 1 for resolution after medication withdrawal, 2 for lack of alternative causes, and 1 for objective evidence of neurologic dysfunction—for a total of 7 points, indicating ertapenem was probably the cause of the patient’s neurologic symptoms. Of 4 prior cases in which carbapenem toxicity was suspected and the Naranjo scale was used, 3 found a probable relationship, and the fourth a highly probable one.^10,12 Confusion, disorientation, hallucinations, tangential thoughts, and garbled speech were reported in the 3 probable cases of ADEs. In the highly probable case, tangential thoughts, garbled speech, and miosis were noted on examination, and these findings returned after re-exposure to ertapenem. Of note, these ADEs occurred in patients with normal and abnormal renal function, and in middle-aged and elderly patients.^10,11,13

• Clinicians should include rare adverse effects of common medications in the differential diagnosis.

• The Naranjo score is a validated tool that can be used to systematically assess the probability of an adverse drug effect at the bedside.

• The presentation of ertapenem-associated neurotoxicity may include features of bilateral cerebellar dysfunction.

Disclosure

Nothing to report.

Pleural effusion can occur in myriad conditions including infection, heart failure, liver disease, and cancer.¹ Consequently, physicians from many disciplines routinely encounter both inpatients and outpatients with this diagnosis. Often, evaluation and treatment require thoracentesis to obtain fluid for analysis or symptom relief.

Although historically performed at the bedside without imaging guidance or intraprocedural monitoring, thoracentesis performed in this fashion carries considerable risk of complications. In fact, it has 1 of the highest rates of iatrogenic pneumothorax among bedside procedures.² However, recent advances in practice and adoption of newer technologies have helped to mitigate risks associated with this procedure. These advances are relevant because approximately 50% of thoracenteses are still performed at the bedside.³ In this review, we aim to identify the most recent key practices that enhance the safety and the effectiveness of thoracentesis for practicing clinicians.

Information Sources and Search Strategy

With the assistance of a research librarian, we performed a systematic search of PubMed-indexed articles from January 1, 2000 to September 30, 2015. Articles were identified using search terms such as thoracentesis, pleural effusion, safety, medical error, adverse event, and ultrasound in combination with Boolean operators. Of note, as thoracentesis is indexed as a subgroup of paracentesis in PubMed, this term was also included to increase the sensitivity of the search. The full search strategy is available in the Appendix. Any references cited in this review outside of the date range of our search are provided only to give relevant background information or establish the origin of commonly performed practices.

Study Eligibility and Selection Criteria

Studies were included if they reported clinical aspects related to thoracentesis. We defined clinical aspects as those strategies that focused on operator training, procedural techniques, technology, management, or prevention of complications. Non-English language articles, animal studies, case reports, conference proceedings, and abstracts were excluded. As our intention was to focus on the contemporary advances related to thoracentesis performance, (eg, ultrasound [US]), our search was limited to studies published after the year 2000. Two authors, Drs. Schildhouse and Lai independently screened studies to determine inclusion, excluding studies with weak methodology, very small sample sizes, and those only tangentially related to our aim. Disagreements regarding study inclusion were resolved by consensus. Drs. Lai, Barsuk, and Mourad identified additional studies by hand review of reference lists and content experts (Figure 1).

Conceptual Framework

All selected articles were categorized by temporal relationship to thoracentesis as pre-, intra-, or postprocedure. Pre-procedural topics were those outcomes that had been identified and addressed before attempting thoracentesis, such as physician training or perceived risks of harm. Intraprocedural considerations included aspects such as use of bedside US, pleural manometry, and large-volume drainage. Finally, postprocedural factors were those related to evaluation after thoracentesis, such as follow-up imaging. This conceptual framework is outlined in Figure 2.

The PubMed search returned a total of 1170 manuscripts, of which 56 articles met inclusion criteria. Four additional articles were identified by experts and included in the study.^4-7 Therefore, 60 articles were identified and included in this review. Study designs included cohort studies, case control studies, systematic reviews, meta-analyses, narrative reviews, consensus guidelines, and randomized controlled trials. A summary of all included articles by topic can be found in the Table.
 

PRE-PROCEDURAL CONSIDERATIONS

Physician Training

Studies indicate that graduate medical education may not adequately prepare clinicians to perform thoracentesis.⁸ In fact, residents have the least exposure and confidence in performing thoracentesis when compared to other bedside procedures.^9,10 In 1 survey, 69% of medical trainees desired more exposure to procedures, and 98% felt that procedural skills were important to master.¹¹ Not surprisingly, then, graduating internal medicine residents perform poorly when assessed on a thoracentesis simulator.¹²

Supplemental training outside of residency is useful to develop and maintain skills for thoracentesis, such as simulation with direct observation in a zero-risk environment. In 1 study, “simulation-based mastery learning” combined an educational video presentation with repeated, deliberate practice on a simulator until procedural competence was acquired, over two 2-hour sessions. In this study, 40 third-year medicine residents demonstrated a 71% improvement in clinical skills performance after course completion, with 93% achieving a passing score. The remaining 7% also achieved passing scores with extra practice time.¹² Others have built upon the concept of simulation-based training. For instance, 2 studies suggest that use of a simulation-based curriculum improved both thoracentesis knowledge and performance skills in a 3-hour session.^13,14 Similarly, 1 prospective study reported that a half-day thoracentesis workshop using simulation and 1:1 direct observation successfully lowered pneumothorax rates from 8.6% to 1.8% in a group of practicing clinicians. Notably, additional interventions including use of bedside US, limiting operators to a focused group, and standardization of equipment were also a part of this quality improvement initiative.⁷ Although repetition is required to gain proficiency when using a simulator, performance and confidence appear to plateau with only 4 simulator trials. In medical students, improvements derived through simulator-based teaching were sustained when retested 6 months following training.¹⁵

An instrument to ensure competency is necessary, given variability in procedural experience among both new graduates and practicing physicians,. Our search did not identify any clinically validated tools that adequately assessed thoracentesis performance. However, some have been proposed¹⁶ and 1 validated in a simulation environment.¹² Regarding the incorporation of US for effusion markup, 1 validated tool used an 11-domain assessment covering knowledge of US machine manipulation, recognition of images with common pleural effusion characteristics, and performance of thoracic US with puncture-site marking on a simulator. When used on 22 participants, scores with the tool could reliably differentiate between novice, intermediate, and advanced groups (P < 0.0001).¹⁷

Patient Selection

Coagulopathies and Anticoagulation. Historically, the accepted cutoff for performing thoracentesis is an international normalized ratio (INR) less than 1.5 and a platelet count greater than 50,000/µL. McVay et al.¹⁸ first showed in 1991 that use of these cutoffs was associated with low rates of periprocedural bleeding, leading to endorsement in the British Thoracic Society (BTS) Pleural Disease Guideline 2010.¹⁹ Other recommendations include the 2012 Society for Interventional Radiology guidelines that endorse correction of an INR greater than 2, or platelets less than 50,000/µL, based almost exclusively on expert opinion.⁵

However, data suggest that thoracentesis may be safely performed outside these parameters. For instance, a prospective study of approximately 9000 thoracenteses over 12 years found that patients with an INR of 1.5-2.9 or platelets of 20,000 - 49,000/µL experienced rates of bleeding complications similar to those with normal values.²⁰ Similarly, a 2014 review²¹ found that the overall risk of hemorrhage during thoracentesis in the setting of moderate coagulopathy (defined as an INR of 1.5 - 3 or platelets of 25,000-50,000/µL), was not increased. In 1 retrospective study of more than 1000 procedures, no differences in hemorrhagic events were noted in patients with bleeding diatheses that received prophylactic fresh frozen plasma or platelets vs. those who did not.²² Of note, included studies used a variety of criteria to define a hemorrhagic complication, which included: an isolated 2 g/dL or more decrement in hemoglobin, presence of bloody fluid on repeat tap with associated hemoglobin decrement, rapid re-accumulation of fluid with a hemoglobin decrement, or transfusion of 2 units or more of whole blood.

Whether it is safe to perform thoracentesis on patients taking antiplatelet therapy is less well understood. Although data are limited, a few small-scale studies^23,24 suggest that hemorrhagic complications following thoracentesis in patients receiving clopidogrel are comparable to the general population. We found no compelling data regarding the safety of thoracentesis in the setting of direct oral anticoagulants, heparin, low-molecular weight heparin, or intravenous direct thrombin inhibitors. Current practice is to generally avoid thoracentesis while these therapeutic anticoagulants are used.

Invasive mechanical ventilation. Pleural effusion is common in patients in the intensive care unit, including those requiring mechanical ventilation.²⁵ Thoracentesis in this population is clinically important: fluid analysis in 1 study was shown to aid the diagnosis in 45% of cases and changes in treatment in 33%.²⁶ However, clinicians may be reluctant to perform thoracentesis on patients who require mechanical ventilation, given the perception of a greater risk of pneumothorax from positive pressure ventilation.

Despite this concern, a 2011 meta-analysis including 19 studies and more than 1100 patients revealed rates of pneumothorax and hemothorax comparable to nonventilated patients.²⁵ Furthermore, a 2015 prospective study that examined thoracentesis in 1377 mechanically ventilated patients revealed no difference in complication rates as well.²⁰ Therefore, evidence suggests that performance of thoracentesis in mechanically ventilated patients is not contraindicated.

Skin Disinfection and Antisepsis Precautions

The 2010 BTS guidelines list empyema and wound infection as possible complications of thoracentesis.¹⁹ However, no data regarding incidence are provided. Additionally, an alcohol-based skin cleanser (such as 2% chlorhexidine gluconate/70% isopropyl alcohol), along with sterile gloves, field, and dressing are suggested as precautionary measures.¹⁹ In 1 single-center registry of 2489 thoracenteses performed using alcohol or iodine-based antiseptic and sterile drapes, no postprocedure infections were identified.²⁷ Of note, we did not find other studies (including case reports) that reported either incidence or rate of infectious complications such as wound infection and empyema. In an era of modern skin antiseptics that have effectively reduced complications such as catheter-related bloodstream infection,²⁸ the incidence of this event is thus likely to be low.

INTRAPROCEDURAL CONSIDERATIONS

Use of Bedside Ultrasound

Portable US has particular advantages for evaluation of pleural effusion vs other imaging modalities. Compared with computerized tomography (CT), bedside US offers similar performance but is less costly, avoids both radiation exposure and need for patient transportation, and provides results instantaneously.^29,30 Compared to chest x-ray (CXR), US is more sensitive at detecting the presence, volume, and characteristics of pleural fluid^30,31 and can be up to 100% sensitive for effusions greater than 100 mL.²⁹ Furthermore, whereas CXR typically requires 200 mL of fluid to be present for detection of an effusion, US can reliably detect as little as 20 mL of fluid.²⁹ When US was used to confirm thoracentesis puncture sites in a study involving 30 physicians of varying experience and 67 consecutive patients, 15% of sites found by clinical exam were inaccurate (less than 10 mm fluid present), 10% were at high risk for organ puncture, and a suitable fluid pocket was found 54% of times when exam could not.⁴

A 2010 meta-analysis of 24 studies and 6605 thoracenteses estimated the overall rate of pneumothorax at 6%; however, procedures performed with US guidance were associated with a 70% reduced risk of this event (odds ratio, 0.30; 95% confidence interval, 0.20 - 0.70).³² In a 2014 randomized control trial of 160 patients that compared thoracentesis with US guidance for site marking vs no US use, 10 pneumothoraces occurred in the control group vs 1 in the US group (12.5% vs 1.25%, P = 0.009).³³ Similarly, another retrospective review of 445 consecutive patients with malignant effusions revealed a pneumothorax rate of 0.97% using US in real time during needle insertion compared to 8.89% for unguided thoracenteses (P < 0.0001).³⁴ Several other studies using US guidance for either site markup or in real time reported similar pneumothorax rates, ranging from 1.1% - 4.8%.^35-37 However, it is unclear if real-time US specifically provides an additive effect vs site marking alone, as no studies directly comparing the 2 methods were found.

Benefits of US also include a higher rate of procedural success, with 1 study demonstrating a 99% success rate when using US vs. 90% without (P = 0.030).³³ A larger volume of fluid removed has been observed with US use as well, and methods have been described using fluid-pocket depth to guide puncture site localization and maximize drainage.³⁸ Finally, US use for thoracentesis has been associated with lower costs and length of stay.^39,40

Intercostal Artery Localization

Although rare (incidence, 0.18%-2%^20,21,39), the occurrence of hemothorax following thoracentesis is potentially catastrophic. This serious complication is often caused by laceration of the intercostal artery (ICA) or 1 of its branches during needle insertion.⁴¹

While risk of injury is theoretically reduced by needle insertion superior to the rib, studies using cadaver dissection and 3D angiography show significant tortuosity of the ICA.^6,41-43 The degree of tortuosity is increased within 6 cm of the midline, in more cephalad rib spaces, and in the elderly (older than 60 years).^41-43 Furthermore, 1 cadaveric study also demonstrated the presence of arterial collaterals branching off the ICA at multiple intercostal spaces, ranging between 8 cm and 11 cm from the midline.⁴¹ This anatomic variability may explain why some have observed low complication and hemothorax rates with an extreme lateral approach.³⁵ Bedside US with color flow Doppler imaging has been used to identify the ICA, with 88% sensitivity compared to CT imaging while adding little to exam time.^44,45 Of note, a 37% drop in the rate of hemothorax was observed in 1 study with routine US guidance alone.³⁹

Pleural Pressure Monitoring and Large-Volume Thoracentesis

While normal intrapleural pressures are approximately -5 to -10 cm H₂O,⁴⁶ the presence of a pleural effusion creates a complex interaction between fluid, compressed lung, and chest wall that can increase these pressures.⁴⁷ During drainage of an effusion, pleural pressures may rapidly drop, provoking re-expansion pulmonary edema (REPE). While rare (0 -1%), clinically-diagnosed REPE is a serious complication that can lead to rapid respiratory failure and death.^20,48 REPE is postulated to be caused by increased capillary permeability resulting from inflammation, driven by rapid re-inflation of the lung when exposed to highly negative intrapleural pressures.^47,49

Measurement of intrapleural pressure using a water manometer during thoracentesis may minimize REPE by terminating fluid drainage when intrapleural pressure begins to drop rapidly.^50,51 A cutoff of -20 cm H₂O has been cited repeatedly as safe since being suggested by Light in 1980, but this is based on animal models.^50,52 In 1 prospective study of 185 thoracenteses in which manometry was performed, 15% of patients had intrapleural pressure drop to less than -20 cm H₂O (at which point the procedure was terminated) but suffered no REPE.⁵⁰

Manometry is valuable in the identification of an unexpandable or trapped lung when pleural pressures drop rapidly with only minimal fluid volume removal.^47,53 Other findings correlated with an unexpandable lung include a negative opening pressure⁴⁷ and large fluctuations in pressure during the respiratory cycle.⁵⁴

While development of symptoms (eg, chest pain, cough, or dyspnea) is often used as a surrogate, the correlation between intrapleural pressure and patient symptoms is inconsistent and not a reliable proxy.⁵⁵ One study found that 22% of patients with chest pain during thoracentesis had intrapleural pressures lower than -20 cm H₂O compared with 8.6% of asymptomatic patients,⁵⁶ but it is unclear if the association is causal.

Thoracentesis is often performed for symptomatic relief and removal of large fluid volume. However, it remains common to halt fluid removal after 1.5 L, a threshold endorsed by BTS.¹⁹ While some investigators have suggested that removal of 2 L or more of pleural fluid does not compromise safety,^57,58 a 4- to 5-fold rise in the risk of pneumothorax was noted in 2 studies.^20,59 when more than 1.5 L of fluid was removed. The majority of these may be related to pneumothorax ex vacuo, a condition in which fluid is drained from the chest, but the lung is unable to expand and fill the space (eg, “trapped lung”), resulting in a persistent pneumothorax. This condition generally does not require treatment.⁶⁰ When manometry is employed at 200-mL intervals with termination at an intrapleural pressure of less than 20 mm H₂O, drainage of 3 L or more has been reported with low rates of pneumothorax and very low rates of REPE.^50,51 However, whether this is cause and effect is unknown because REPE is rare, and more work is needed to determine the role of manometry for its prevention.

POSTPROCEDURAL CONSIDERATIONS

Postprocedure Imaging

Performing an upright CXR following thoracentesis is a practice that remains routinely done by many practitioners to monitor for complications. Such imaging was also endorsed by the American Thoracic Society guidelines.⁶¹ However, more recent data question the utility of this practice. Multiple studies have confirmed that post-thoracentesis CXR is unnecessary unless clinical suspicion for pneumothorax or REPE is present.^36,58,62,63 The BTS guidelines also advocate this approach.¹⁹ Interestingly, a potentially more effective way to screen for postprocedure complications is through bedside US, which has been shown to be more sensitive than CXR in detecting pneumothorax.⁶⁴ In 1 study of 185 patients, bedside US demonstrated a sensitivity of 88% and a specificity of 97% for diagnosing pneumothorax in patients with adequate quality scans, with positive and negative likelihood ratios of 55 and 0.17, respectively.⁶⁵

Thoracentesis remains a core procedural skill for hospitalists, critical care physicians, and emergency physicians. It is the foundational component when investigating and treating pleural effusions. When the most current training, techniques, and technology are used, data suggest this procedure is safe to perform at the bedside. Our review highlights these strategies and evaluates which aspects might be most applicable to clinical practice.

Our findings have several implications for those who perform this procedure. First, appropriate training is central to procedural safety, and both simulation and direct observation by procedural experts have been shown by multiple investigators to improve knowledge and skill. This training should integrate the use of US in performing a focused thoracic exam.

Second, recommendations regarding coagulopathy and a “safe cutoff” of an INR less than 1.5 or platelets greater than 50,000/µL had limited evidentiary support. Rather, multiple studies suggest no difference in bleeding risk following thoracentesis with an INR as high as 3.0 and platelets greater than 25,000/µL. Furthermore, prophylactic transfusion with fresh frozen plasma or platelets before thoracentesis did not alter bleeding risk and exposes patients to transfusion complications. Thus, routine use of this practice can no longer be recommended. Third, further research is needed to understand the bleeding risk for patients on antiplatelet medications, heparin products, and also direct oral anticoagulants, given the growing popularity in their use and the potential consequences of even temporary cessation. Regarding patients on mechanical ventilation, thoracentesis demonstrated no difference in complication rates vs. the general population, and its performance in this population is encouraged when clinically indicated.

Intraprocedural considerations include the use of bedside US. Due to multiple benefits including effusion characterization, puncture site localization, and significantly lower rates of pneumothorax, the standard of care should be to perform thoracentesis with US guidance. Both use of US to mark an effusion immediately prior to puncture or in real time during needle insertion demonstrated benefit; however, it is unclear if 1 method is superior because no direct comparison studies were found. Further work is needed to investigate this potential.

Our review suggests that the location and course of the ICA is variable, especially near the midline, in the elderly, and in higher intercostal spaces, leaving it vulnerable to laceration. We recommend physicians only attempt thoracentesis at least 6 cm lateral to the midline due to ICA tortuosity and, ideally, 12 cm lateral, to avoid the presence of collaterals. Although only 2 small-scale studies were found pertaining to the use of US in identifying the ICA, we encourage physicians to consider learning how to screen for its presence as a part of their routine thoracic US exam in the area underlying the planned puncture site.

Manometry is beneficial because it can diagnose a nonexpandable lung and allows for pleural pressure monitoring.^52,53 A simple U-shaped manometer can be constructed from intravenous tubing included in most thoracentesis kits, which adds little to overall procedure time. While low rates of REPE have been observed when terminating thoracentesis if pressures drop below -20 cm H₂O or chest pain develops, neither measure appears to have reliable predictive value, limiting clinical utility. Further work is required to determine if a “safe pressure cutoff” exists. In general, we recommend the use of manometry when a nonexpandable (trapped) lung is suspected, because large drops in intrapleural pressure, a negative opening pressure, and respiratory variation can help confirm the diagnosis and avoid pneumothorax ex vacuo or unnecessary procedures in the future. As this condition appears to be more common in the setting of larger effusions, use of manometry when large-volume thoracenteses are planned is also reasonable.

Postprocedurally, routine imaging after thoracentesis is not recommended unless there is objective concern for complication. When indicated, bedside US is better positioned for this role compared with CXR, because it is more sensitive in detecting pneumothorax, provides instantaneous results, and avoids radiation exposure.

Our review has limitations. First, we searched only for articles between defined time periods, restricted our search to a single database, and excluded non-English articles. This has the potential to introduce selection bias, as nonprimary articles that fall within our time restrictions may cite older studies that are outside our search range. To minimize this effect, we performed a critical review of all included studies, especially nonprimary articles. Second, despite the focus of our search strategy to identify any articles related to patient safety and adverse events, we cannot guarantee that all relevant articles for any particular complication or risk factor were captured given the lack of more specific search terms. Third, although we performed a systematic search of the literature, we did not perform a formal systematic review or formally grade included studies. As the goal of our review was to categorize and operationalize clinical aspects, this approach was necessary, and we acknowledge that the quality of studies is variable. Lastly, we aimed to generate clinical recommendations for physicians performing thoracentesis at the bedside; others reviewing this literature may find or emphasize different aspects relevant to practice outside this setting.

In conclusion, evaluation and treatment of pleural effusions with bedside thoracentesis is an important skill for physicians of many disciplines. The evidence presented in this review will help inform the process and ensure patient safety. Physicians should consider incorporating these recommendations into their practice.

Acknowledgments

The authors thank Whitney Townsend, MLIS, health sciences informationist, for assistance with serial literature searches.

Disclosure

Nothing to report.

Pleural effusion can occur in myriad conditions including infection, heart failure, liver disease, and cancer.¹ Consequently, physicians from many disciplines routinely encounter both inpatients and outpatients with this diagnosis. Often, evaluation and treatment require thoracentesis to obtain fluid for analysis or symptom relief.

Although historically performed at the bedside without imaging guidance or intraprocedural monitoring, thoracentesis performed in this fashion carries considerable risk of complications. In fact, it has 1 of the highest rates of iatrogenic pneumothorax among bedside procedures.² However, recent advances in practice and adoption of newer technologies have helped to mitigate risks associated with this procedure. These advances are relevant because approximately 50% of thoracenteses are still performed at the bedside.³ In this review, we aim to identify the most recent key practices that enhance the safety and the effectiveness of thoracentesis for practicing clinicians.

Information Sources and Search Strategy

With the assistance of a research librarian, we performed a systematic search of PubMed-indexed articles from January 1, 2000 to September 30, 2015. Articles were identified using search terms such as thoracentesis, pleural effusion, safety, medical error, adverse event, and ultrasound in combination with Boolean operators. Of note, as thoracentesis is indexed as a subgroup of paracentesis in PubMed, this term was also included to increase the sensitivity of the search. The full search strategy is available in the Appendix. Any references cited in this review outside of the date range of our search are provided only to give relevant background information or establish the origin of commonly performed practices.

Study Eligibility and Selection Criteria

Studies were included if they reported clinical aspects related to thoracentesis. We defined clinical aspects as those strategies that focused on operator training, procedural techniques, technology, management, or prevention of complications. Non-English language articles, animal studies, case reports, conference proceedings, and abstracts were excluded. As our intention was to focus on the contemporary advances related to thoracentesis performance, (eg, ultrasound [US]), our search was limited to studies published after the year 2000. Two authors, Drs. Schildhouse and Lai independently screened studies to determine inclusion, excluding studies with weak methodology, very small sample sizes, and those only tangentially related to our aim. Disagreements regarding study inclusion were resolved by consensus. Drs. Lai, Barsuk, and Mourad identified additional studies by hand review of reference lists and content experts (Figure 1).

Conceptual Framework

All selected articles were categorized by temporal relationship to thoracentesis as pre-, intra-, or postprocedure. Pre-procedural topics were those outcomes that had been identified and addressed before attempting thoracentesis, such as physician training or perceived risks of harm. Intraprocedural considerations included aspects such as use of bedside US, pleural manometry, and large-volume drainage. Finally, postprocedural factors were those related to evaluation after thoracentesis, such as follow-up imaging. This conceptual framework is outlined in Figure 2.

The PubMed search returned a total of 1170 manuscripts, of which 56 articles met inclusion criteria. Four additional articles were identified by experts and included in the study.^4-7 Therefore, 60 articles were identified and included in this review. Study designs included cohort studies, case control studies, systematic reviews, meta-analyses, narrative reviews, consensus guidelines, and randomized controlled trials. A summary of all included articles by topic can be found in the Table.
 

PRE-PROCEDURAL CONSIDERATIONS

Physician Training

Studies indicate that graduate medical education may not adequately prepare clinicians to perform thoracentesis.⁸ In fact, residents have the least exposure and confidence in performing thoracentesis when compared to other bedside procedures.^9,10 In 1 survey, 69% of medical trainees desired more exposure to procedures, and 98% felt that procedural skills were important to master.¹¹ Not surprisingly, then, graduating internal medicine residents perform poorly when assessed on a thoracentesis simulator.¹²

Supplemental training outside of residency is useful to develop and maintain skills for thoracentesis, such as simulation with direct observation in a zero-risk environment. In 1 study, “simulation-based mastery learning” combined an educational video presentation with repeated, deliberate practice on a simulator until procedural competence was acquired, over two 2-hour sessions. In this study, 40 third-year medicine residents demonstrated a 71% improvement in clinical skills performance after course completion, with 93% achieving a passing score. The remaining 7% also achieved passing scores with extra practice time.¹² Others have built upon the concept of simulation-based training. For instance, 2 studies suggest that use of a simulation-based curriculum improved both thoracentesis knowledge and performance skills in a 3-hour session.^13,14 Similarly, 1 prospective study reported that a half-day thoracentesis workshop using simulation and 1:1 direct observation successfully lowered pneumothorax rates from 8.6% to 1.8% in a group of practicing clinicians. Notably, additional interventions including use of bedside US, limiting operators to a focused group, and standardization of equipment were also a part of this quality improvement initiative.⁷ Although repetition is required to gain proficiency when using a simulator, performance and confidence appear to plateau with only 4 simulator trials. In medical students, improvements derived through simulator-based teaching were sustained when retested 6 months following training.¹⁵

An instrument to ensure competency is necessary, given variability in procedural experience among both new graduates and practicing physicians,. Our search did not identify any clinically validated tools that adequately assessed thoracentesis performance. However, some have been proposed¹⁶ and 1 validated in a simulation environment.¹² Regarding the incorporation of US for effusion markup, 1 validated tool used an 11-domain assessment covering knowledge of US machine manipulation, recognition of images with common pleural effusion characteristics, and performance of thoracic US with puncture-site marking on a simulator. When used on 22 participants, scores with the tool could reliably differentiate between novice, intermediate, and advanced groups (P < 0.0001).¹⁷

Patient Selection

Coagulopathies and Anticoagulation. Historically, the accepted cutoff for performing thoracentesis is an international normalized ratio (INR) less than 1.5 and a platelet count greater than 50,000/µL. McVay et al.¹⁸ first showed in 1991 that use of these cutoffs was associated with low rates of periprocedural bleeding, leading to endorsement in the British Thoracic Society (BTS) Pleural Disease Guideline 2010.¹⁹ Other recommendations include the 2012 Society for Interventional Radiology guidelines that endorse correction of an INR greater than 2, or platelets less than 50,000/µL, based almost exclusively on expert opinion.⁵

However, data suggest that thoracentesis may be safely performed outside these parameters. For instance, a prospective study of approximately 9000 thoracenteses over 12 years found that patients with an INR of 1.5-2.9 or platelets of 20,000 - 49,000/µL experienced rates of bleeding complications similar to those with normal values.²⁰ Similarly, a 2014 review²¹ found that the overall risk of hemorrhage during thoracentesis in the setting of moderate coagulopathy (defined as an INR of 1.5 - 3 or platelets of 25,000-50,000/µL), was not increased. In 1 retrospective study of more than 1000 procedures, no differences in hemorrhagic events were noted in patients with bleeding diatheses that received prophylactic fresh frozen plasma or platelets vs. those who did not.²² Of note, included studies used a variety of criteria to define a hemorrhagic complication, which included: an isolated 2 g/dL or more decrement in hemoglobin, presence of bloody fluid on repeat tap with associated hemoglobin decrement, rapid re-accumulation of fluid with a hemoglobin decrement, or transfusion of 2 units or more of whole blood.

Whether it is safe to perform thoracentesis on patients taking antiplatelet therapy is less well understood. Although data are limited, a few small-scale studies^23,24 suggest that hemorrhagic complications following thoracentesis in patients receiving clopidogrel are comparable to the general population. We found no compelling data regarding the safety of thoracentesis in the setting of direct oral anticoagulants, heparin, low-molecular weight heparin, or intravenous direct thrombin inhibitors. Current practice is to generally avoid thoracentesis while these therapeutic anticoagulants are used.

Invasive mechanical ventilation. Pleural effusion is common in patients in the intensive care unit, including those requiring mechanical ventilation.²⁵ Thoracentesis in this population is clinically important: fluid analysis in 1 study was shown to aid the diagnosis in 45% of cases and changes in treatment in 33%.²⁶ However, clinicians may be reluctant to perform thoracentesis on patients who require mechanical ventilation, given the perception of a greater risk of pneumothorax from positive pressure ventilation.

Despite this concern, a 2011 meta-analysis including 19 studies and more than 1100 patients revealed rates of pneumothorax and hemothorax comparable to nonventilated patients.²⁵ Furthermore, a 2015 prospective study that examined thoracentesis in 1377 mechanically ventilated patients revealed no difference in complication rates as well.²⁰ Therefore, evidence suggests that performance of thoracentesis in mechanically ventilated patients is not contraindicated.

Skin Disinfection and Antisepsis Precautions

The 2010 BTS guidelines list empyema and wound infection as possible complications of thoracentesis.¹⁹ However, no data regarding incidence are provided. Additionally, an alcohol-based skin cleanser (such as 2% chlorhexidine gluconate/70% isopropyl alcohol), along with sterile gloves, field, and dressing are suggested as precautionary measures.¹⁹ In 1 single-center registry of 2489 thoracenteses performed using alcohol or iodine-based antiseptic and sterile drapes, no postprocedure infections were identified.²⁷ Of note, we did not find other studies (including case reports) that reported either incidence or rate of infectious complications such as wound infection and empyema. In an era of modern skin antiseptics that have effectively reduced complications such as catheter-related bloodstream infection,²⁸ the incidence of this event is thus likely to be low.

INTRAPROCEDURAL CONSIDERATIONS

Use of Bedside Ultrasound

Portable US has particular advantages for evaluation of pleural effusion vs other imaging modalities. Compared with computerized tomography (CT), bedside US offers similar performance but is less costly, avoids both radiation exposure and need for patient transportation, and provides results instantaneously.^29,30 Compared to chest x-ray (CXR), US is more sensitive at detecting the presence, volume, and characteristics of pleural fluid^30,31 and can be up to 100% sensitive for effusions greater than 100 mL.²⁹ Furthermore, whereas CXR typically requires 200 mL of fluid to be present for detection of an effusion, US can reliably detect as little as 20 mL of fluid.²⁹ When US was used to confirm thoracentesis puncture sites in a study involving 30 physicians of varying experience and 67 consecutive patients, 15% of sites found by clinical exam were inaccurate (less than 10 mm fluid present), 10% were at high risk for organ puncture, and a suitable fluid pocket was found 54% of times when exam could not.⁴

A 2010 meta-analysis of 24 studies and 6605 thoracenteses estimated the overall rate of pneumothorax at 6%; however, procedures performed with US guidance were associated with a 70% reduced risk of this event (odds ratio, 0.30; 95% confidence interval, 0.20 - 0.70).³² In a 2014 randomized control trial of 160 patients that compared thoracentesis with US guidance for site marking vs no US use, 10 pneumothoraces occurred in the control group vs 1 in the US group (12.5% vs 1.25%, P = 0.009).³³ Similarly, another retrospective review of 445 consecutive patients with malignant effusions revealed a pneumothorax rate of 0.97% using US in real time during needle insertion compared to 8.89% for unguided thoracenteses (P < 0.0001).³⁴ Several other studies using US guidance for either site markup or in real time reported similar pneumothorax rates, ranging from 1.1% - 4.8%.^35-37 However, it is unclear if real-time US specifically provides an additive effect vs site marking alone, as no studies directly comparing the 2 methods were found.

Benefits of US also include a higher rate of procedural success, with 1 study demonstrating a 99% success rate when using US vs. 90% without (P = 0.030).³³ A larger volume of fluid removed has been observed with US use as well, and methods have been described using fluid-pocket depth to guide puncture site localization and maximize drainage.³⁸ Finally, US use for thoracentesis has been associated with lower costs and length of stay.^39,40

Intercostal Artery Localization

Although rare (incidence, 0.18%-2%^20,21,39), the occurrence of hemothorax following thoracentesis is potentially catastrophic. This serious complication is often caused by laceration of the intercostal artery (ICA) or 1 of its branches during needle insertion.⁴¹

While risk of injury is theoretically reduced by needle insertion superior to the rib, studies using cadaver dissection and 3D angiography show significant tortuosity of the ICA.^6,41-43 The degree of tortuosity is increased within 6 cm of the midline, in more cephalad rib spaces, and in the elderly (older than 60 years).^41-43 Furthermore, 1 cadaveric study also demonstrated the presence of arterial collaterals branching off the ICA at multiple intercostal spaces, ranging between 8 cm and 11 cm from the midline.⁴¹ This anatomic variability may explain why some have observed low complication and hemothorax rates with an extreme lateral approach.³⁵ Bedside US with color flow Doppler imaging has been used to identify the ICA, with 88% sensitivity compared to CT imaging while adding little to exam time.^44,45 Of note, a 37% drop in the rate of hemothorax was observed in 1 study with routine US guidance alone.³⁹

Pleural Pressure Monitoring and Large-Volume Thoracentesis

While normal intrapleural pressures are approximately -5 to -10 cm H₂O,⁴⁶ the presence of a pleural effusion creates a complex interaction between fluid, compressed lung, and chest wall that can increase these pressures.⁴⁷ During drainage of an effusion, pleural pressures may rapidly drop, provoking re-expansion pulmonary edema (REPE). While rare (0 -1%), clinically-diagnosed REPE is a serious complication that can lead to rapid respiratory failure and death.^20,48 REPE is postulated to be caused by increased capillary permeability resulting from inflammation, driven by rapid re-inflation of the lung when exposed to highly negative intrapleural pressures.^47,49

Measurement of intrapleural pressure using a water manometer during thoracentesis may minimize REPE by terminating fluid drainage when intrapleural pressure begins to drop rapidly.^50,51 A cutoff of -20 cm H₂O has been cited repeatedly as safe since being suggested by Light in 1980, but this is based on animal models.^50,52 In 1 prospective study of 185 thoracenteses in which manometry was performed, 15% of patients had intrapleural pressure drop to less than -20 cm H₂O (at which point the procedure was terminated) but suffered no REPE.⁵⁰

Manometry is valuable in the identification of an unexpandable or trapped lung when pleural pressures drop rapidly with only minimal fluid volume removal.^47,53 Other findings correlated with an unexpandable lung include a negative opening pressure⁴⁷ and large fluctuations in pressure during the respiratory cycle.⁵⁴

While development of symptoms (eg, chest pain, cough, or dyspnea) is often used as a surrogate, the correlation between intrapleural pressure and patient symptoms is inconsistent and not a reliable proxy.⁵⁵ One study found that 22% of patients with chest pain during thoracentesis had intrapleural pressures lower than -20 cm H₂O compared with 8.6% of asymptomatic patients,⁵⁶ but it is unclear if the association is causal.

Thoracentesis is often performed for symptomatic relief and removal of large fluid volume. However, it remains common to halt fluid removal after 1.5 L, a threshold endorsed by BTS.¹⁹ While some investigators have suggested that removal of 2 L or more of pleural fluid does not compromise safety,^57,58 a 4- to 5-fold rise in the risk of pneumothorax was noted in 2 studies.^20,59 when more than 1.5 L of fluid was removed. The majority of these may be related to pneumothorax ex vacuo, a condition in which fluid is drained from the chest, but the lung is unable to expand and fill the space (eg, “trapped lung”), resulting in a persistent pneumothorax. This condition generally does not require treatment.⁶⁰ When manometry is employed at 200-mL intervals with termination at an intrapleural pressure of less than 20 mm H₂O, drainage of 3 L or more has been reported with low rates of pneumothorax and very low rates of REPE.^50,51 However, whether this is cause and effect is unknown because REPE is rare, and more work is needed to determine the role of manometry for its prevention.

POSTPROCEDURAL CONSIDERATIONS

Postprocedure Imaging

Performing an upright CXR following thoracentesis is a practice that remains routinely done by many practitioners to monitor for complications. Such imaging was also endorsed by the American Thoracic Society guidelines.⁶¹ However, more recent data question the utility of this practice. Multiple studies have confirmed that post-thoracentesis CXR is unnecessary unless clinical suspicion for pneumothorax or REPE is present.^36,58,62,63 The BTS guidelines also advocate this approach.¹⁹ Interestingly, a potentially more effective way to screen for postprocedure complications is through bedside US, which has been shown to be more sensitive than CXR in detecting pneumothorax.⁶⁴ In 1 study of 185 patients, bedside US demonstrated a sensitivity of 88% and a specificity of 97% for diagnosing pneumothorax in patients with adequate quality scans, with positive and negative likelihood ratios of 55 and 0.17, respectively.⁶⁵

Thoracentesis remains a core procedural skill for hospitalists, critical care physicians, and emergency physicians. It is the foundational component when investigating and treating pleural effusions. When the most current training, techniques, and technology are used, data suggest this procedure is safe to perform at the bedside. Our review highlights these strategies and evaluates which aspects might be most applicable to clinical practice.

Our findings have several implications for those who perform this procedure. First, appropriate training is central to procedural safety, and both simulation and direct observation by procedural experts have been shown by multiple investigators to improve knowledge and skill. This training should integrate the use of US in performing a focused thoracic exam.

Second, recommendations regarding coagulopathy and a “safe cutoff” of an INR less than 1.5 or platelets greater than 50,000/µL had limited evidentiary support. Rather, multiple studies suggest no difference in bleeding risk following thoracentesis with an INR as high as 3.0 and platelets greater than 25,000/µL. Furthermore, prophylactic transfusion with fresh frozen plasma or platelets before thoracentesis did not alter bleeding risk and exposes patients to transfusion complications. Thus, routine use of this practice can no longer be recommended. Third, further research is needed to understand the bleeding risk for patients on antiplatelet medications, heparin products, and also direct oral anticoagulants, given the growing popularity in their use and the potential consequences of even temporary cessation. Regarding patients on mechanical ventilation, thoracentesis demonstrated no difference in complication rates vs. the general population, and its performance in this population is encouraged when clinically indicated.

Intraprocedural considerations include the use of bedside US. Due to multiple benefits including effusion characterization, puncture site localization, and significantly lower rates of pneumothorax, the standard of care should be to perform thoracentesis with US guidance. Both use of US to mark an effusion immediately prior to puncture or in real time during needle insertion demonstrated benefit; however, it is unclear if 1 method is superior because no direct comparison studies were found. Further work is needed to investigate this potential.

Our review suggests that the location and course of the ICA is variable, especially near the midline, in the elderly, and in higher intercostal spaces, leaving it vulnerable to laceration. We recommend physicians only attempt thoracentesis at least 6 cm lateral to the midline due to ICA tortuosity and, ideally, 12 cm lateral, to avoid the presence of collaterals. Although only 2 small-scale studies were found pertaining to the use of US in identifying the ICA, we encourage physicians to consider learning how to screen for its presence as a part of their routine thoracic US exam in the area underlying the planned puncture site.

Manometry is beneficial because it can diagnose a nonexpandable lung and allows for pleural pressure monitoring.^52,53 A simple U-shaped manometer can be constructed from intravenous tubing included in most thoracentesis kits, which adds little to overall procedure time. While low rates of REPE have been observed when terminating thoracentesis if pressures drop below -20 cm H₂O or chest pain develops, neither measure appears to have reliable predictive value, limiting clinical utility. Further work is required to determine if a “safe pressure cutoff” exists. In general, we recommend the use of manometry when a nonexpandable (trapped) lung is suspected, because large drops in intrapleural pressure, a negative opening pressure, and respiratory variation can help confirm the diagnosis and avoid pneumothorax ex vacuo or unnecessary procedures in the future. As this condition appears to be more common in the setting of larger effusions, use of manometry when large-volume thoracenteses are planned is also reasonable.

Postprocedurally, routine imaging after thoracentesis is not recommended unless there is objective concern for complication. When indicated, bedside US is better positioned for this role compared with CXR, because it is more sensitive in detecting pneumothorax, provides instantaneous results, and avoids radiation exposure.

Our review has limitations. First, we searched only for articles between defined time periods, restricted our search to a single database, and excluded non-English articles. This has the potential to introduce selection bias, as nonprimary articles that fall within our time restrictions may cite older studies that are outside our search range. To minimize this effect, we performed a critical review of all included studies, especially nonprimary articles. Second, despite the focus of our search strategy to identify any articles related to patient safety and adverse events, we cannot guarantee that all relevant articles for any particular complication or risk factor were captured given the lack of more specific search terms. Third, although we performed a systematic search of the literature, we did not perform a formal systematic review or formally grade included studies. As the goal of our review was to categorize and operationalize clinical aspects, this approach was necessary, and we acknowledge that the quality of studies is variable. Lastly, we aimed to generate clinical recommendations for physicians performing thoracentesis at the bedside; others reviewing this literature may find or emphasize different aspects relevant to practice outside this setting.

In conclusion, evaluation and treatment of pleural effusions with bedside thoracentesis is an important skill for physicians of many disciplines. The evidence presented in this review will help inform the process and ensure patient safety. Physicians should consider incorporating these recommendations into their practice.

Acknowledgments

The authors thank Whitney Townsend, MLIS, health sciences informationist, for assistance with serial literature searches.

Disclosure

Nothing to report.

Pleural effusion can occur in myriad conditions including infection, heart failure, liver disease, and cancer.¹ Consequently, physicians from many disciplines routinely encounter both inpatients and outpatients with this diagnosis. Often, evaluation and treatment require thoracentesis to obtain fluid for analysis or symptom relief.

Although historically performed at the bedside without imaging guidance or intraprocedural monitoring, thoracentesis performed in this fashion carries considerable risk of complications. In fact, it has 1 of the highest rates of iatrogenic pneumothorax among bedside procedures.² However, recent advances in practice and adoption of newer technologies have helped to mitigate risks associated with this procedure. These advances are relevant because approximately 50% of thoracenteses are still performed at the bedside.³ In this review, we aim to identify the most recent key practices that enhance the safety and the effectiveness of thoracentesis for practicing clinicians.

Information Sources and Search Strategy

With the assistance of a research librarian, we performed a systematic search of PubMed-indexed articles from January 1, 2000 to September 30, 2015. Articles were identified using search terms such as thoracentesis, pleural effusion, safety, medical error, adverse event, and ultrasound in combination with Boolean operators. Of note, as thoracentesis is indexed as a subgroup of paracentesis in PubMed, this term was also included to increase the sensitivity of the search. The full search strategy is available in the Appendix. Any references cited in this review outside of the date range of our search are provided only to give relevant background information or establish the origin of commonly performed practices.

Study Eligibility and Selection Criteria

Studies were included if they reported clinical aspects related to thoracentesis. We defined clinical aspects as those strategies that focused on operator training, procedural techniques, technology, management, or prevention of complications. Non-English language articles, animal studies, case reports, conference proceedings, and abstracts were excluded. As our intention was to focus on the contemporary advances related to thoracentesis performance, (eg, ultrasound [US]), our search was limited to studies published after the year 2000. Two authors, Drs. Schildhouse and Lai independently screened studies to determine inclusion, excluding studies with weak methodology, very small sample sizes, and those only tangentially related to our aim. Disagreements regarding study inclusion were resolved by consensus. Drs. Lai, Barsuk, and Mourad identified additional studies by hand review of reference lists and content experts (Figure 1).

Conceptual Framework

All selected articles were categorized by temporal relationship to thoracentesis as pre-, intra-, or postprocedure. Pre-procedural topics were those outcomes that had been identified and addressed before attempting thoracentesis, such as physician training or perceived risks of harm. Intraprocedural considerations included aspects such as use of bedside US, pleural manometry, and large-volume drainage. Finally, postprocedural factors were those related to evaluation after thoracentesis, such as follow-up imaging. This conceptual framework is outlined in Figure 2.

The PubMed search returned a total of 1170 manuscripts, of which 56 articles met inclusion criteria. Four additional articles were identified by experts and included in the study.^4-7 Therefore, 60 articles were identified and included in this review. Study designs included cohort studies, case control studies, systematic reviews, meta-analyses, narrative reviews, consensus guidelines, and randomized controlled trials. A summary of all included articles by topic can be found in the Table.
 

PRE-PROCEDURAL CONSIDERATIONS

Physician Training

Studies indicate that graduate medical education may not adequately prepare clinicians to perform thoracentesis.⁸ In fact, residents have the least exposure and confidence in performing thoracentesis when compared to other bedside procedures.^9,10 In 1 survey, 69% of medical trainees desired more exposure to procedures, and 98% felt that procedural skills were important to master.¹¹ Not surprisingly, then, graduating internal medicine residents perform poorly when assessed on a thoracentesis simulator.¹²

Supplemental training outside of residency is useful to develop and maintain skills for thoracentesis, such as simulation with direct observation in a zero-risk environment. In 1 study, “simulation-based mastery learning” combined an educational video presentation with repeated, deliberate practice on a simulator until procedural competence was acquired, over two 2-hour sessions. In this study, 40 third-year medicine residents demonstrated a 71% improvement in clinical skills performance after course completion, with 93% achieving a passing score. The remaining 7% also achieved passing scores with extra practice time.¹² Others have built upon the concept of simulation-based training. For instance, 2 studies suggest that use of a simulation-based curriculum improved both thoracentesis knowledge and performance skills in a 3-hour session.^13,14 Similarly, 1 prospective study reported that a half-day thoracentesis workshop using simulation and 1:1 direct observation successfully lowered pneumothorax rates from 8.6% to 1.8% in a group of practicing clinicians. Notably, additional interventions including use of bedside US, limiting operators to a focused group, and standardization of equipment were also a part of this quality improvement initiative.⁷ Although repetition is required to gain proficiency when using a simulator, performance and confidence appear to plateau with only 4 simulator trials. In medical students, improvements derived through simulator-based teaching were sustained when retested 6 months following training.¹⁵

An instrument to ensure competency is necessary, given variability in procedural experience among both new graduates and practicing physicians,. Our search did not identify any clinically validated tools that adequately assessed thoracentesis performance. However, some have been proposed¹⁶ and 1 validated in a simulation environment.¹² Regarding the incorporation of US for effusion markup, 1 validated tool used an 11-domain assessment covering knowledge of US machine manipulation, recognition of images with common pleural effusion characteristics, and performance of thoracic US with puncture-site marking on a simulator. When used on 22 participants, scores with the tool could reliably differentiate between novice, intermediate, and advanced groups (P < 0.0001).¹⁷

Patient Selection

Coagulopathies and Anticoagulation. Historically, the accepted cutoff for performing thoracentesis is an international normalized ratio (INR) less than 1.5 and a platelet count greater than 50,000/µL. McVay et al.¹⁸ first showed in 1991 that use of these cutoffs was associated with low rates of periprocedural bleeding, leading to endorsement in the British Thoracic Society (BTS) Pleural Disease Guideline 2010.¹⁹ Other recommendations include the 2012 Society for Interventional Radiology guidelines that endorse correction of an INR greater than 2, or platelets less than 50,000/µL, based almost exclusively on expert opinion.⁵

However, data suggest that thoracentesis may be safely performed outside these parameters. For instance, a prospective study of approximately 9000 thoracenteses over 12 years found that patients with an INR of 1.5-2.9 or platelets of 20,000 - 49,000/µL experienced rates of bleeding complications similar to those with normal values.²⁰ Similarly, a 2014 review²¹ found that the overall risk of hemorrhage during thoracentesis in the setting of moderate coagulopathy (defined as an INR of 1.5 - 3 or platelets of 25,000-50,000/µL), was not increased. In 1 retrospective study of more than 1000 procedures, no differences in hemorrhagic events were noted in patients with bleeding diatheses that received prophylactic fresh frozen plasma or platelets vs. those who did not.²² Of note, included studies used a variety of criteria to define a hemorrhagic complication, which included: an isolated 2 g/dL or more decrement in hemoglobin, presence of bloody fluid on repeat tap with associated hemoglobin decrement, rapid re-accumulation of fluid with a hemoglobin decrement, or transfusion of 2 units or more of whole blood.

Whether it is safe to perform thoracentesis on patients taking antiplatelet therapy is less well understood. Although data are limited, a few small-scale studies^23,24 suggest that hemorrhagic complications following thoracentesis in patients receiving clopidogrel are comparable to the general population. We found no compelling data regarding the safety of thoracentesis in the setting of direct oral anticoagulants, heparin, low-molecular weight heparin, or intravenous direct thrombin inhibitors. Current practice is to generally avoid thoracentesis while these therapeutic anticoagulants are used.

Invasive mechanical ventilation. Pleural effusion is common in patients in the intensive care unit, including those requiring mechanical ventilation.²⁵ Thoracentesis in this population is clinically important: fluid analysis in 1 study was shown to aid the diagnosis in 45% of cases and changes in treatment in 33%.²⁶ However, clinicians may be reluctant to perform thoracentesis on patients who require mechanical ventilation, given the perception of a greater risk of pneumothorax from positive pressure ventilation.

Despite this concern, a 2011 meta-analysis including 19 studies and more than 1100 patients revealed rates of pneumothorax and hemothorax comparable to nonventilated patients.²⁵ Furthermore, a 2015 prospective study that examined thoracentesis in 1377 mechanically ventilated patients revealed no difference in complication rates as well.²⁰ Therefore, evidence suggests that performance of thoracentesis in mechanically ventilated patients is not contraindicated.

Skin Disinfection and Antisepsis Precautions

The 2010 BTS guidelines list empyema and wound infection as possible complications of thoracentesis.¹⁹ However, no data regarding incidence are provided. Additionally, an alcohol-based skin cleanser (such as 2% chlorhexidine gluconate/70% isopropyl alcohol), along with sterile gloves, field, and dressing are suggested as precautionary measures.¹⁹ In 1 single-center registry of 2489 thoracenteses performed using alcohol or iodine-based antiseptic and sterile drapes, no postprocedure infections were identified.²⁷ Of note, we did not find other studies (including case reports) that reported either incidence or rate of infectious complications such as wound infection and empyema. In an era of modern skin antiseptics that have effectively reduced complications such as catheter-related bloodstream infection,²⁸ the incidence of this event is thus likely to be low.

INTRAPROCEDURAL CONSIDERATIONS

Use of Bedside Ultrasound

Portable US has particular advantages for evaluation of pleural effusion vs other imaging modalities. Compared with computerized tomography (CT), bedside US offers similar performance but is less costly, avoids both radiation exposure and need for patient transportation, and provides results instantaneously.^29,30 Compared to chest x-ray (CXR), US is more sensitive at detecting the presence, volume, and characteristics of pleural fluid^30,31 and can be up to 100% sensitive for effusions greater than 100 mL.²⁹ Furthermore, whereas CXR typically requires 200 mL of fluid to be present for detection of an effusion, US can reliably detect as little as 20 mL of fluid.²⁹ When US was used to confirm thoracentesis puncture sites in a study involving 30 physicians of varying experience and 67 consecutive patients, 15% of sites found by clinical exam were inaccurate (less than 10 mm fluid present), 10% were at high risk for organ puncture, and a suitable fluid pocket was found 54% of times when exam could not.⁴

A 2010 meta-analysis of 24 studies and 6605 thoracenteses estimated the overall rate of pneumothorax at 6%; however, procedures performed with US guidance were associated with a 70% reduced risk of this event (odds ratio, 0.30; 95% confidence interval, 0.20 - 0.70).³² In a 2014 randomized control trial of 160 patients that compared thoracentesis with US guidance for site marking vs no US use, 10 pneumothoraces occurred in the control group vs 1 in the US group (12.5% vs 1.25%, P = 0.009).³³ Similarly, another retrospective review of 445 consecutive patients with malignant effusions revealed a pneumothorax rate of 0.97% using US in real time during needle insertion compared to 8.89% for unguided thoracenteses (P < 0.0001).³⁴ Several other studies using US guidance for either site markup or in real time reported similar pneumothorax rates, ranging from 1.1% - 4.8%.^35-37 However, it is unclear if real-time US specifically provides an additive effect vs site marking alone, as no studies directly comparing the 2 methods were found.

Benefits of US also include a higher rate of procedural success, with 1 study demonstrating a 99% success rate when using US vs. 90% without (P = 0.030).³³ A larger volume of fluid removed has been observed with US use as well, and methods have been described using fluid-pocket depth to guide puncture site localization and maximize drainage.³⁸ Finally, US use for thoracentesis has been associated with lower costs and length of stay.^39,40

Intercostal Artery Localization

Although rare (incidence, 0.18%-2%^20,21,39), the occurrence of hemothorax following thoracentesis is potentially catastrophic. This serious complication is often caused by laceration of the intercostal artery (ICA) or 1 of its branches during needle insertion.⁴¹

While risk of injury is theoretically reduced by needle insertion superior to the rib, studies using cadaver dissection and 3D angiography show significant tortuosity of the ICA.^6,41-43 The degree of tortuosity is increased within 6 cm of the midline, in more cephalad rib spaces, and in the elderly (older than 60 years).^41-43 Furthermore, 1 cadaveric study also demonstrated the presence of arterial collaterals branching off the ICA at multiple intercostal spaces, ranging between 8 cm and 11 cm from the midline.⁴¹ This anatomic variability may explain why some have observed low complication and hemothorax rates with an extreme lateral approach.³⁵ Bedside US with color flow Doppler imaging has been used to identify the ICA, with 88% sensitivity compared to CT imaging while adding little to exam time.^44,45 Of note, a 37% drop in the rate of hemothorax was observed in 1 study with routine US guidance alone.³⁹

Pleural Pressure Monitoring and Large-Volume Thoracentesis

While normal intrapleural pressures are approximately -5 to -10 cm H₂O,⁴⁶ the presence of a pleural effusion creates a complex interaction between fluid, compressed lung, and chest wall that can increase these pressures.⁴⁷ During drainage of an effusion, pleural pressures may rapidly drop, provoking re-expansion pulmonary edema (REPE). While rare (0 -1%), clinically-diagnosed REPE is a serious complication that can lead to rapid respiratory failure and death.^20,48 REPE is postulated to be caused by increased capillary permeability resulting from inflammation, driven by rapid re-inflation of the lung when exposed to highly negative intrapleural pressures.^47,49

Measurement of intrapleural pressure using a water manometer during thoracentesis may minimize REPE by terminating fluid drainage when intrapleural pressure begins to drop rapidly.^50,51 A cutoff of -20 cm H₂O has been cited repeatedly as safe since being suggested by Light in 1980, but this is based on animal models.^50,52 In 1 prospective study of 185 thoracenteses in which manometry was performed, 15% of patients had intrapleural pressure drop to less than -20 cm H₂O (at which point the procedure was terminated) but suffered no REPE.⁵⁰

Manometry is valuable in the identification of an unexpandable or trapped lung when pleural pressures drop rapidly with only minimal fluid volume removal.^47,53 Other findings correlated with an unexpandable lung include a negative opening pressure⁴⁷ and large fluctuations in pressure during the respiratory cycle.⁵⁴

While development of symptoms (eg, chest pain, cough, or dyspnea) is often used as a surrogate, the correlation between intrapleural pressure and patient symptoms is inconsistent and not a reliable proxy.⁵⁵ One study found that 22% of patients with chest pain during thoracentesis had intrapleural pressures lower than -20 cm H₂O compared with 8.6% of asymptomatic patients,⁵⁶ but it is unclear if the association is causal.

Thoracentesis is often performed for symptomatic relief and removal of large fluid volume. However, it remains common to halt fluid removal after 1.5 L, a threshold endorsed by BTS.¹⁹ While some investigators have suggested that removal of 2 L or more of pleural fluid does not compromise safety,^57,58 a 4- to 5-fold rise in the risk of pneumothorax was noted in 2 studies.^20,59 when more than 1.5 L of fluid was removed. The majority of these may be related to pneumothorax ex vacuo, a condition in which fluid is drained from the chest, but the lung is unable to expand and fill the space (eg, “trapped lung”), resulting in a persistent pneumothorax. This condition generally does not require treatment.⁶⁰ When manometry is employed at 200-mL intervals with termination at an intrapleural pressure of less than 20 mm H₂O, drainage of 3 L or more has been reported with low rates of pneumothorax and very low rates of REPE.^50,51 However, whether this is cause and effect is unknown because REPE is rare, and more work is needed to determine the role of manometry for its prevention.

POSTPROCEDURAL CONSIDERATIONS

Postprocedure Imaging

Performing an upright CXR following thoracentesis is a practice that remains routinely done by many practitioners to monitor for complications. Such imaging was also endorsed by the American Thoracic Society guidelines.⁶¹ However, more recent data question the utility of this practice. Multiple studies have confirmed that post-thoracentesis CXR is unnecessary unless clinical suspicion for pneumothorax or REPE is present.^36,58,62,63 The BTS guidelines also advocate this approach.¹⁹ Interestingly, a potentially more effective way to screen for postprocedure complications is through bedside US, which has been shown to be more sensitive than CXR in detecting pneumothorax.⁶⁴ In 1 study of 185 patients, bedside US demonstrated a sensitivity of 88% and a specificity of 97% for diagnosing pneumothorax in patients with adequate quality scans, with positive and negative likelihood ratios of 55 and 0.17, respectively.⁶⁵

Thoracentesis remains a core procedural skill for hospitalists, critical care physicians, and emergency physicians. It is the foundational component when investigating and treating pleural effusions. When the most current training, techniques, and technology are used, data suggest this procedure is safe to perform at the bedside. Our review highlights these strategies and evaluates which aspects might be most applicable to clinical practice.

Our findings have several implications for those who perform this procedure. First, appropriate training is central to procedural safety, and both simulation and direct observation by procedural experts have been shown by multiple investigators to improve knowledge and skill. This training should integrate the use of US in performing a focused thoracic exam.

Second, recommendations regarding coagulopathy and a “safe cutoff” of an INR less than 1.5 or platelets greater than 50,000/µL had limited evidentiary support. Rather, multiple studies suggest no difference in bleeding risk following thoracentesis with an INR as high as 3.0 and platelets greater than 25,000/µL. Furthermore, prophylactic transfusion with fresh frozen plasma or platelets before thoracentesis did not alter bleeding risk and exposes patients to transfusion complications. Thus, routine use of this practice can no longer be recommended. Third, further research is needed to understand the bleeding risk for patients on antiplatelet medications, heparin products, and also direct oral anticoagulants, given the growing popularity in their use and the potential consequences of even temporary cessation. Regarding patients on mechanical ventilation, thoracentesis demonstrated no difference in complication rates vs. the general population, and its performance in this population is encouraged when clinically indicated.

Intraprocedural considerations include the use of bedside US. Due to multiple benefits including effusion characterization, puncture site localization, and significantly lower rates of pneumothorax, the standard of care should be to perform thoracentesis with US guidance. Both use of US to mark an effusion immediately prior to puncture or in real time during needle insertion demonstrated benefit; however, it is unclear if 1 method is superior because no direct comparison studies were found. Further work is needed to investigate this potential.

Our review suggests that the location and course of the ICA is variable, especially near the midline, in the elderly, and in higher intercostal spaces, leaving it vulnerable to laceration. We recommend physicians only attempt thoracentesis at least 6 cm lateral to the midline due to ICA tortuosity and, ideally, 12 cm lateral, to avoid the presence of collaterals. Although only 2 small-scale studies were found pertaining to the use of US in identifying the ICA, we encourage physicians to consider learning how to screen for its presence as a part of their routine thoracic US exam in the area underlying the planned puncture site.

Manometry is beneficial because it can diagnose a nonexpandable lung and allows for pleural pressure monitoring.^52,53 A simple U-shaped manometer can be constructed from intravenous tubing included in most thoracentesis kits, which adds little to overall procedure time. While low rates of REPE have been observed when terminating thoracentesis if pressures drop below -20 cm H₂O or chest pain develops, neither measure appears to have reliable predictive value, limiting clinical utility. Further work is required to determine if a “safe pressure cutoff” exists. In general, we recommend the use of manometry when a nonexpandable (trapped) lung is suspected, because large drops in intrapleural pressure, a negative opening pressure, and respiratory variation can help confirm the diagnosis and avoid pneumothorax ex vacuo or unnecessary procedures in the future. As this condition appears to be more common in the setting of larger effusions, use of manometry when large-volume thoracenteses are planned is also reasonable.

Postprocedurally, routine imaging after thoracentesis is not recommended unless there is objective concern for complication. When indicated, bedside US is better positioned for this role compared with CXR, because it is more sensitive in detecting pneumothorax, provides instantaneous results, and avoids radiation exposure.

Our review has limitations. First, we searched only for articles between defined time periods, restricted our search to a single database, and excluded non-English articles. This has the potential to introduce selection bias, as nonprimary articles that fall within our time restrictions may cite older studies that are outside our search range. To minimize this effect, we performed a critical review of all included studies, especially nonprimary articles. Second, despite the focus of our search strategy to identify any articles related to patient safety and adverse events, we cannot guarantee that all relevant articles for any particular complication or risk factor were captured given the lack of more specific search terms. Third, although we performed a systematic search of the literature, we did not perform a formal systematic review or formally grade included studies. As the goal of our review was to categorize and operationalize clinical aspects, this approach was necessary, and we acknowledge that the quality of studies is variable. Lastly, we aimed to generate clinical recommendations for physicians performing thoracentesis at the bedside; others reviewing this literature may find or emphasize different aspects relevant to practice outside this setting.

In conclusion, evaluation and treatment of pleural effusions with bedside thoracentesis is an important skill for physicians of many disciplines. The evidence presented in this review will help inform the process and ensure patient safety. Physicians should consider incorporating these recommendations into their practice.

Acknowledgments

The authors thank Whitney Townsend, MLIS, health sciences informationist, for assistance with serial literature searches.

Disclosure

Nothing to report.

Of the 36 million US hospitalizations each year, 22% are surgical.¹ Although less frequent than medical hospitalizations, surgical hospitalizations are more than twice as costly.² Additionally, surgical hospitalizations are on average longer than medical hospitalizations.² Given the increased scrutiny on cost and efficiency of care, attention has turned to optimizing perioperative care. Hospitalists are well positioned to provide specific expertise in the complex interdisciplinary medical management of surgical patients.

In recent decades, multiple models of hospitalist involvement in perioperative care have evolved across the United States.^3-19 To consolidate knowledge and experience and to develop a framework for providing the best care for surgical patients, the Society of Hospital Medicine organized the Perioperative Care Work Group in 2015. This framework was designed for interdisciplinary collaboration in building and strengthening perioperative care programs.

The Society of Hospital Medicine recognized hospital medicine programs’ need for guidance in developing collaborative care in perioperative medicine and appointed the Perioperative Care Work Group in May 2015. Work group members are perioperative medicine experts from US medical centers. They have extensive knowledge of the literature as well as administrative and clinical experience in a variety of perioperative care models.

Topic Development. Initial work was focused on reviewing and discussing multiple models of perioperative care and exploring the roles that hospital medicine physicians have within these models. Useful information was summarized to guide hospitals and physicians in designing, implementing, and expanding patient-centric perioperative medicine services with a focus on preoperative and postoperative care. A final document was created; it outlines system-level issues in perioperative care, organized by perioperative phases.

Initial Framework. Group members submitted written descriptions of key issues in each of 4 phases: (1) preoperative, (2) day of surgery, (3) postoperative inpatient, and (4) postdischarge. These descriptions were merged and reviewed by the content experts. Editing and discussion from the entire group were incorporated into the final matrix, which highlighted (1) perioperative phase definitions, (2) requirements for patients to move to next phase, (3) elements of care coordination typically provided by surgery, anesthesiology, and medicine disciplines, (4) concerns and risks particular to each phase, (5) unique considerations for each phase, (6) suggested metrics of success, and (7) key questions for determining the effectiveness of perioperative care in an institution. All members provided final evaluation and editing.

Final Approval. The Perioperative Care Matrix for Inpatient Surgeries (PCMIS) was presented to the board of the Society of Hospital Medicine in fall 2015 and was approved for use in centering and directing discussions regarding perioperative care.

Models of Care. The Perioperative Care Work Group surveyed examples of hospitalist engagement in perioperative care and synthesized these into synopses of existing models of care for the preoperative, day-of-surgery, postoperative-inpatient, and postdischarge phases.

Defining Key Concepts and Issues

Hospitalists have participated in a variety of perioperative roles for more than a decade. Roles include performing in-depth preoperative assessments, providing oversight to presurgical advanced practice provider assessments, providing inpatient comanagement and consultation both before and after surgery, and providing postdischarge follow-up within the surgical period for medical comorbidities.

Although a comprehensive look at the entire perioperative period is important, 4 specific phases were defined to guide this work (Figure). The phases identified were based on time relative to surgery, with unique considerations as to the overall perioperative period. Concerns and potential risks specific to each phase were considered (Table 1).

Models of Care

Multiple examples of hospitalist involvement were collected to inform the program development guidelines. The specifics noted among the reviewed practice models are described here.

Preoperative. In some centers, all patients scheduled for surgery are required to undergo evaluation at the institution’s preoperative clinic. At most others, referral to the preoperative clinic is at the discretion of the surgical specialists, who have been informed of the clinic’s available resources. Factors determining whether a patient has an in-person clinic visit, undergoes a telephone-based medical evaluation, or has a referral deferred to the primary care physician (PCP) include patient complexity and surgery-specific risk. Patients who have major medical comorbidities (eg, chronic lung or heart disease) or are undergoing higher risk procedures (eg, those lasting >1 hour, laparotomy) most often undergo a formal clinic evaluation. Often, even for a patient whose preoperative evaluation is completed by a PCP, the preoperative nursing staff will call before surgery to provide instructions and to confirm that preoperative planning is complete. Confirmation includes ensuring that the surgery consent and preoperative history and physical examination documents are in the medical record, and that all recommended tests have been performed. If deficiencies are found, surgical and preoperative clinic staff are notified.

The United States is focused on sensible, high-value care. Perioperative care is burgeoning with opportunities for improvement, including reducing avoidable complications, developing systems for early recognition and treatment of complications, and streamlining processes to shorten length of stay and improve patient experience. The PCMIS provides the needed platform to catalyze detailed collaborative work between disciplines engaged in perioperative care.

As average age and level of medical comorbidity increase among surgical patients, hospitalists will increasingly be called on to assist in perioperative care. Hospitalists have long been involved in caring for medically complex surgical patients, through comanagement, consultation, and preoperative evaluations. As a provider group, hospitalists have comprehensive skills in quality and systems improvement, and in program development across hospital systems nationwide. Hospitalists have demonstrated their value by focusing on improving patient outcomes and enhancing patient engagement and experiences. Additionally, the perioperative period is fraught with multiple and complicated handoffs, a problem area for which hospital medicine has pioneered solutions and developed unique expertise. Hospital medicine is well prepared to provide skilled and proven leadership in the timely development, improvement, and expansion of perioperative care for this increasingly older and chronically ill population.

Hospitalists are established in multiple perioperative roles for high-risk surgical patients and have the opportunity to expand optimal patient-centric perioperative care systems working in close concert with surgeons and anesthesiologists. The basics of developing these systems include (1) assessing risk for medical complications, (2) planning for perioperative care, (3) developing programs aimed at risk reduction for preventable complications and early identification and intervention for unavoidable complications, and (4) guiding quality improvement efforts, including planning for frequent handoffs and transitions.

As a key partner in developing comprehensive programs in perioperative care, hospital medicine will continue to shape the future of hospital care for all patients. The PCMIS, as developed with support from the Society of Hospital Medicine, will aid efforts to achieve the best perioperative care models for our surgical patients.

Disclosures

Financial activities outside the submitted work: Drs. Pfeifer and Jaffer report payment for development of educational presentations; Dr. Grant reports payment for expert testimony pertaining to hospital medicine; Drs. Grant and Jaffer report royalties from publishing; Drs. Thompson, Pfiefer, Grant, Slawski, and Jaffer report travel expenses for speaking and serving on national committees; and Drs. Slawski and Jaffer serve on the board of the Society of Perioperative Assessment and Quality Improvement. The other authors have nothing to report.

Of the 36 million US hospitalizations each year, 22% are surgical.¹ Although less frequent than medical hospitalizations, surgical hospitalizations are more than twice as costly.² Additionally, surgical hospitalizations are on average longer than medical hospitalizations.² Given the increased scrutiny on cost and efficiency of care, attention has turned to optimizing perioperative care. Hospitalists are well positioned to provide specific expertise in the complex interdisciplinary medical management of surgical patients.

In recent decades, multiple models of hospitalist involvement in perioperative care have evolved across the United States.^3-19 To consolidate knowledge and experience and to develop a framework for providing the best care for surgical patients, the Society of Hospital Medicine organized the Perioperative Care Work Group in 2015. This framework was designed for interdisciplinary collaboration in building and strengthening perioperative care programs.

The Society of Hospital Medicine recognized hospital medicine programs’ need for guidance in developing collaborative care in perioperative medicine and appointed the Perioperative Care Work Group in May 2015. Work group members are perioperative medicine experts from US medical centers. They have extensive knowledge of the literature as well as administrative and clinical experience in a variety of perioperative care models.

Topic Development. Initial work was focused on reviewing and discussing multiple models of perioperative care and exploring the roles that hospital medicine physicians have within these models. Useful information was summarized to guide hospitals and physicians in designing, implementing, and expanding patient-centric perioperative medicine services with a focus on preoperative and postoperative care. A final document was created; it outlines system-level issues in perioperative care, organized by perioperative phases.

Initial Framework. Group members submitted written descriptions of key issues in each of 4 phases: (1) preoperative, (2) day of surgery, (3) postoperative inpatient, and (4) postdischarge. These descriptions were merged and reviewed by the content experts. Editing and discussion from the entire group were incorporated into the final matrix, which highlighted (1) perioperative phase definitions, (2) requirements for patients to move to next phase, (3) elements of care coordination typically provided by surgery, anesthesiology, and medicine disciplines, (4) concerns and risks particular to each phase, (5) unique considerations for each phase, (6) suggested metrics of success, and (7) key questions for determining the effectiveness of perioperative care in an institution. All members provided final evaluation and editing.

Final Approval. The Perioperative Care Matrix for Inpatient Surgeries (PCMIS) was presented to the board of the Society of Hospital Medicine in fall 2015 and was approved for use in centering and directing discussions regarding perioperative care.

Models of Care. The Perioperative Care Work Group surveyed examples of hospitalist engagement in perioperative care and synthesized these into synopses of existing models of care for the preoperative, day-of-surgery, postoperative-inpatient, and postdischarge phases.

Defining Key Concepts and Issues

Hospitalists have participated in a variety of perioperative roles for more than a decade. Roles include performing in-depth preoperative assessments, providing oversight to presurgical advanced practice provider assessments, providing inpatient comanagement and consultation both before and after surgery, and providing postdischarge follow-up within the surgical period for medical comorbidities.

Although a comprehensive look at the entire perioperative period is important, 4 specific phases were defined to guide this work (Figure). The phases identified were based on time relative to surgery, with unique considerations as to the overall perioperative period. Concerns and potential risks specific to each phase were considered (Table 1).

Models of Care

Multiple examples of hospitalist involvement were collected to inform the program development guidelines. The specifics noted among the reviewed practice models are described here.

Preoperative. In some centers, all patients scheduled for surgery are required to undergo evaluation at the institution’s preoperative clinic. At most others, referral to the preoperative clinic is at the discretion of the surgical specialists, who have been informed of the clinic’s available resources. Factors determining whether a patient has an in-person clinic visit, undergoes a telephone-based medical evaluation, or has a referral deferred to the primary care physician (PCP) include patient complexity and surgery-specific risk. Patients who have major medical comorbidities (eg, chronic lung or heart disease) or are undergoing higher risk procedures (eg, those lasting >1 hour, laparotomy) most often undergo a formal clinic evaluation. Often, even for a patient whose preoperative evaluation is completed by a PCP, the preoperative nursing staff will call before surgery to provide instructions and to confirm that preoperative planning is complete. Confirmation includes ensuring that the surgery consent and preoperative history and physical examination documents are in the medical record, and that all recommended tests have been performed. If deficiencies are found, surgical and preoperative clinic staff are notified.

The United States is focused on sensible, high-value care. Perioperative care is burgeoning with opportunities for improvement, including reducing avoidable complications, developing systems for early recognition and treatment of complications, and streamlining processes to shorten length of stay and improve patient experience. The PCMIS provides the needed platform to catalyze detailed collaborative work between disciplines engaged in perioperative care.

As average age and level of medical comorbidity increase among surgical patients, hospitalists will increasingly be called on to assist in perioperative care. Hospitalists have long been involved in caring for medically complex surgical patients, through comanagement, consultation, and preoperative evaluations. As a provider group, hospitalists have comprehensive skills in quality and systems improvement, and in program development across hospital systems nationwide. Hospitalists have demonstrated their value by focusing on improving patient outcomes and enhancing patient engagement and experiences. Additionally, the perioperative period is fraught with multiple and complicated handoffs, a problem area for which hospital medicine has pioneered solutions and developed unique expertise. Hospital medicine is well prepared to provide skilled and proven leadership in the timely development, improvement, and expansion of perioperative care for this increasingly older and chronically ill population.

Hospitalists are established in multiple perioperative roles for high-risk surgical patients and have the opportunity to expand optimal patient-centric perioperative care systems working in close concert with surgeons and anesthesiologists. The basics of developing these systems include (1) assessing risk for medical complications, (2) planning for perioperative care, (3) developing programs aimed at risk reduction for preventable complications and early identification and intervention for unavoidable complications, and (4) guiding quality improvement efforts, including planning for frequent handoffs and transitions.

As a key partner in developing comprehensive programs in perioperative care, hospital medicine will continue to shape the future of hospital care for all patients. The PCMIS, as developed with support from the Society of Hospital Medicine, will aid efforts to achieve the best perioperative care models for our surgical patients.

Disclosures

Financial activities outside the submitted work: Drs. Pfeifer and Jaffer report payment for development of educational presentations; Dr. Grant reports payment for expert testimony pertaining to hospital medicine; Drs. Grant and Jaffer report royalties from publishing; Drs. Thompson, Pfiefer, Grant, Slawski, and Jaffer report travel expenses for speaking and serving on national committees; and Drs. Slawski and Jaffer serve on the board of the Society of Perioperative Assessment and Quality Improvement. The other authors have nothing to report.

Of the 36 million US hospitalizations each year, 22% are surgical.¹ Although less frequent than medical hospitalizations, surgical hospitalizations are more than twice as costly.² Additionally, surgical hospitalizations are on average longer than medical hospitalizations.² Given the increased scrutiny on cost and efficiency of care, attention has turned to optimizing perioperative care. Hospitalists are well positioned to provide specific expertise in the complex interdisciplinary medical management of surgical patients.

In recent decades, multiple models of hospitalist involvement in perioperative care have evolved across the United States.^3-19 To consolidate knowledge and experience and to develop a framework for providing the best care for surgical patients, the Society of Hospital Medicine organized the Perioperative Care Work Group in 2015. This framework was designed for interdisciplinary collaboration in building and strengthening perioperative care programs.

The Society of Hospital Medicine recognized hospital medicine programs’ need for guidance in developing collaborative care in perioperative medicine and appointed the Perioperative Care Work Group in May 2015. Work group members are perioperative medicine experts from US medical centers. They have extensive knowledge of the literature as well as administrative and clinical experience in a variety of perioperative care models.

Topic Development. Initial work was focused on reviewing and discussing multiple models of perioperative care and exploring the roles that hospital medicine physicians have within these models. Useful information was summarized to guide hospitals and physicians in designing, implementing, and expanding patient-centric perioperative medicine services with a focus on preoperative and postoperative care. A final document was created; it outlines system-level issues in perioperative care, organized by perioperative phases.

Initial Framework. Group members submitted written descriptions of key issues in each of 4 phases: (1) preoperative, (2) day of surgery, (3) postoperative inpatient, and (4) postdischarge. These descriptions were merged and reviewed by the content experts. Editing and discussion from the entire group were incorporated into the final matrix, which highlighted (1) perioperative phase definitions, (2) requirements for patients to move to next phase, (3) elements of care coordination typically provided by surgery, anesthesiology, and medicine disciplines, (4) concerns and risks particular to each phase, (5) unique considerations for each phase, (6) suggested metrics of success, and (7) key questions for determining the effectiveness of perioperative care in an institution. All members provided final evaluation and editing.

Final Approval. The Perioperative Care Matrix for Inpatient Surgeries (PCMIS) was presented to the board of the Society of Hospital Medicine in fall 2015 and was approved for use in centering and directing discussions regarding perioperative care.

Models of Care. The Perioperative Care Work Group surveyed examples of hospitalist engagement in perioperative care and synthesized these into synopses of existing models of care for the preoperative, day-of-surgery, postoperative-inpatient, and postdischarge phases.

Defining Key Concepts and Issues

Hospitalists have participated in a variety of perioperative roles for more than a decade. Roles include performing in-depth preoperative assessments, providing oversight to presurgical advanced practice provider assessments, providing inpatient comanagement and consultation both before and after surgery, and providing postdischarge follow-up within the surgical period for medical comorbidities.

Although a comprehensive look at the entire perioperative period is important, 4 specific phases were defined to guide this work (Figure). The phases identified were based on time relative to surgery, with unique considerations as to the overall perioperative period. Concerns and potential risks specific to each phase were considered (Table 1).

Models of Care

Multiple examples of hospitalist involvement were collected to inform the program development guidelines. The specifics noted among the reviewed practice models are described here.

Preoperative. In some centers, all patients scheduled for surgery are required to undergo evaluation at the institution’s preoperative clinic. At most others, referral to the preoperative clinic is at the discretion of the surgical specialists, who have been informed of the clinic’s available resources. Factors determining whether a patient has an in-person clinic visit, undergoes a telephone-based medical evaluation, or has a referral deferred to the primary care physician (PCP) include patient complexity and surgery-specific risk. Patients who have major medical comorbidities (eg, chronic lung or heart disease) or are undergoing higher risk procedures (eg, those lasting >1 hour, laparotomy) most often undergo a formal clinic evaluation. Often, even for a patient whose preoperative evaluation is completed by a PCP, the preoperative nursing staff will call before surgery to provide instructions and to confirm that preoperative planning is complete. Confirmation includes ensuring that the surgery consent and preoperative history and physical examination documents are in the medical record, and that all recommended tests have been performed. If deficiencies are found, surgical and preoperative clinic staff are notified.

The United States is focused on sensible, high-value care. Perioperative care is burgeoning with opportunities for improvement, including reducing avoidable complications, developing systems for early recognition and treatment of complications, and streamlining processes to shorten length of stay and improve patient experience. The PCMIS provides the needed platform to catalyze detailed collaborative work between disciplines engaged in perioperative care.

As average age and level of medical comorbidity increase among surgical patients, hospitalists will increasingly be called on to assist in perioperative care. Hospitalists have long been involved in caring for medically complex surgical patients, through comanagement, consultation, and preoperative evaluations. As a provider group, hospitalists have comprehensive skills in quality and systems improvement, and in program development across hospital systems nationwide. Hospitalists have demonstrated their value by focusing on improving patient outcomes and enhancing patient engagement and experiences. Additionally, the perioperative period is fraught with multiple and complicated handoffs, a problem area for which hospital medicine has pioneered solutions and developed unique expertise. Hospital medicine is well prepared to provide skilled and proven leadership in the timely development, improvement, and expansion of perioperative care for this increasingly older and chronically ill population.

Hospitalists are established in multiple perioperative roles for high-risk surgical patients and have the opportunity to expand optimal patient-centric perioperative care systems working in close concert with surgeons and anesthesiologists. The basics of developing these systems include (1) assessing risk for medical complications, (2) planning for perioperative care, (3) developing programs aimed at risk reduction for preventable complications and early identification and intervention for unavoidable complications, and (4) guiding quality improvement efforts, including planning for frequent handoffs and transitions.

As a key partner in developing comprehensive programs in perioperative care, hospital medicine will continue to shape the future of hospital care for all patients. The PCMIS, as developed with support from the Society of Hospital Medicine, will aid efforts to achieve the best perioperative care models for our surgical patients.

Disclosures

Financial activities outside the submitted work: Drs. Pfeifer and Jaffer report payment for development of educational presentations; Dr. Grant reports payment for expert testimony pertaining to hospital medicine; Drs. Grant and Jaffer report royalties from publishing; Drs. Thompson, Pfiefer, Grant, Slawski, and Jaffer report travel expenses for speaking and serving on national committees; and Drs. Slawski and Jaffer serve on the board of the Society of Perioperative Assessment and Quality Improvement. The other authors have nothing to report.

A child who has a cold, flu, or other acute illness may be what parents often call “fussy”: irritable, teary, and clingy. Such changes in behavior and mood, or “sickness behavior (SB),” are usually thought to be linked to fever.  Actually, those symptoms are the immune system’s reactions to invasion by a pathogen, say French researchers—and they may be present whether the child has fever or not. The researchers’ say their multicenter study is the first to show dissociation between SB and the severity of the fever.

The researchers evaluated 6 parameters over the 2 hours preceding consultations with the parents of 200 children with and 200 without fever. Children with particularly painful illnesses and chronic diseases were excluded from the study. Parents used rating scales to report degrees of change in the time the child spent playing, the distance covered (ie. how far from the parent the child roamed), time the child spent seeking comfort, time spent whining or crying, time spent in a state of irritation or anger, most distorted facial expression (on a chart). The researchers also assessed time spent sleeping and appetite in the 24 hours before the consultation. Sickness behavior can’t be reduced to the observation of those 8 behavioral parameters, the researchers note, but they were easy for parents to use and assess.

The mean values of the 8 parameters differed significantly between the 2 groups but were independent of the height of fever in the febrile children. That independence suggests that SB and fever are expressions of 2 autonomous metabolic pathways that are activated simultaneously in febrile conditions, the researchers say, which is in accordance with current pathophysiologic knowledge.

Their findings are in harmony with current treatment recommendations, the researchers say. Because it’s hard to know when behavior changes are due to SB, pain, fatigue, or something else in a febrile child—especially one who is too young to talk about it—it’s more important to focus on relieving the discomfort than in reducing the fever.

Source:

Corrard F, Copin C, Wollner A, et al. PLoS One. 2017;12(3): e0171670.
doi: 10.1371/journal.pone.0171670.

A child who has a cold, flu, or other acute illness may be what parents often call “fussy”: irritable, teary, and clingy. Such changes in behavior and mood, or “sickness behavior (SB),” are usually thought to be linked to fever.  Actually, those symptoms are the immune system’s reactions to invasion by a pathogen, say French researchers—and they may be present whether the child has fever or not. The researchers’ say their multicenter study is the first to show dissociation between SB and the severity of the fever.

The researchers evaluated 6 parameters over the 2 hours preceding consultations with the parents of 200 children with and 200 without fever. Children with particularly painful illnesses and chronic diseases were excluded from the study. Parents used rating scales to report degrees of change in the time the child spent playing, the distance covered (ie. how far from the parent the child roamed), time the child spent seeking comfort, time spent whining or crying, time spent in a state of irritation or anger, most distorted facial expression (on a chart). The researchers also assessed time spent sleeping and appetite in the 24 hours before the consultation. Sickness behavior can’t be reduced to the observation of those 8 behavioral parameters, the researchers note, but they were easy for parents to use and assess.

The mean values of the 8 parameters differed significantly between the 2 groups but were independent of the height of fever in the febrile children. That independence suggests that SB and fever are expressions of 2 autonomous metabolic pathways that are activated simultaneously in febrile conditions, the researchers say, which is in accordance with current pathophysiologic knowledge.

Their findings are in harmony with current treatment recommendations, the researchers say. Because it’s hard to know when behavior changes are due to SB, pain, fatigue, or something else in a febrile child—especially one who is too young to talk about it—it’s more important to focus on relieving the discomfort than in reducing the fever.

Source:

Corrard F, Copin C, Wollner A, et al. PLoS One. 2017;12(3): e0171670.
doi: 10.1371/journal.pone.0171670.

A child who has a cold, flu, or other acute illness may be what parents often call “fussy”: irritable, teary, and clingy. Such changes in behavior and mood, or “sickness behavior (SB),” are usually thought to be linked to fever.  Actually, those symptoms are the immune system’s reactions to invasion by a pathogen, say French researchers—and they may be present whether the child has fever or not. The researchers’ say their multicenter study is the first to show dissociation between SB and the severity of the fever.

The researchers evaluated 6 parameters over the 2 hours preceding consultations with the parents of 200 children with and 200 without fever. Children with particularly painful illnesses and chronic diseases were excluded from the study. Parents used rating scales to report degrees of change in the time the child spent playing, the distance covered (ie. how far from the parent the child roamed), time the child spent seeking comfort, time spent whining or crying, time spent in a state of irritation or anger, most distorted facial expression (on a chart). The researchers also assessed time spent sleeping and appetite in the 24 hours before the consultation. Sickness behavior can’t be reduced to the observation of those 8 behavioral parameters, the researchers note, but they were easy for parents to use and assess.

The mean values of the 8 parameters differed significantly between the 2 groups but were independent of the height of fever in the febrile children. That independence suggests that SB and fever are expressions of 2 autonomous metabolic pathways that are activated simultaneously in febrile conditions, the researchers say, which is in accordance with current pathophysiologic knowledge.

Their findings are in harmony with current treatment recommendations, the researchers say. Because it’s hard to know when behavior changes are due to SB, pain, fatigue, or something else in a febrile child—especially one who is too young to talk about it—it’s more important to focus on relieving the discomfort than in reducing the fever.

Source:

Corrard F, Copin C, Wollner A, et al. PLoS One. 2017;12(3): e0171670.
doi: 10.1371/journal.pone.0171670.

A patient’s prognosis can strongly influence their medical care. Decisions about diagnostic modalities, treatment options, and the use of preventive therapies can all be affected by the likelihood of a patient’s death in the near future. For example, patients with severely limited survival might forego prophylactic therapy, avoid interventions for asymptomatic issues, and cease screening interventions. Knowing survival probability would also be very helpful as a controlling variable in research analyses whenever death risk might be a possible confounder.

Sixteen indices that aim to predict patient death risk have been described by Yourman et al.¹ They were all created from secondary analyses of clinical and administrative datasets, were applicable to patients in a variety of settings (including the community, nursing home, or hospital), and predicted survival probabilities in time horizons ranging from 6 months to 5 years. Prognostic factors that were most commonly included in these indices were comorbidity and functional status. In validation populations, the discrimination of these indices for 1-year survival in hospitalized patients was moderate (with C statistics that ranged from 0.64 to 0.79) with good calibration for broad prognostic ranges.

In 2014, we published the Hospitalized-patient One-year Mortality Risk (HOMR) score.² This study used health administrative data for all adult Ontarians admitted in 2011 to hospital under nonpsychiatric services (n = 640,022) to estimate the probability of dying within 1 year of admission to hospital (which happened in 11.7% of people). The HOMR score included 12 patient and hospitalization factors (Table 1). It was highly discriminative (C statistic, 0.923; [0.922-0.924]) and well calibrated (the mean relative difference between observed and expected death risk was 2.0% [range, 0.0% to 7.0%]). It was externally validated in more than 3 million adults from Ontario, Alberta, and Boston in whom the C statistic ranged from 0.89 to 0.92 and calibration was excellent.³ We concluded from these studies that the HOMR score is excellent for prognosticating a diverse group of patients using health administrative data.

Study Cohort

The study, which was approved by our local research ethics board, took place at the Ottawa Hospital, a 1000-bed teaching hospital that is the primary referral center in our region. We used the hospital admission registry to identify all people 18 years or older who were admitted to a nonpsychiatric service at our hospital between January 1, 2011 and December 31, 2011 (this time frame corresponds with the year used to derive the HOMR score). We excluded overnight patients in the same-day surgery or the bone-marrow transplant units (since they would not have been included in the original study) and those without a valid health card number (which was required to link to provincial data to identify outcomes). From this list, we randomly selected 5000 patients.

Primary Data Collection

For each patient, we retrieved all data required to calculate the HOMR score from the medical record (Table 1). Patient registration information in our electronic medical record was used to identify patient age, sex, admitting service, number of emergency department (ED) visits in the previous year, number of admissions in the previous year (the nursing triage note was reviewed for each admission to determine if it was by ambulance), and whether or not the patient had been discharged from hospital in the previous 30 days. The admitting service consult note was used to determine the admitting diagnosis and whether or not the patient was admitted directly to the intensive care unit. If they were present, the emergency nursing triage note, the ED record of treatment, the admission consult note, the pre-operative consult note, and consult notes were all used to determine the patient’s comorbidities, living status, and home oxygen status. Admission urgency was determined using information from the patient registration information and the ED nursing triage note. All data were abstracted from information that had been registered prior to when the patient was physically transferred to their hospital bed. This ensured that we used only data available at the start of the admission.

Patient functional status has been shown to be strongly associated with survival⁴ but HOMR only indirectly captures functional information (through the patient’s living status). We, therefore, collected more detailed functional information from the medical record by determining if the patient was dependent for any activities of daily living (ADL) from the emergency nursing triage note, the ED record of treatment, the admission consult note, and the pre-operative consultation. We also collected information that might indicate frailty, which we defined per Clegg et al.⁵ as “a state of increased vulnerability to poor resolution of homeostasis following a stress.” This information included: delirium or more than 1 fall recorded on the emergency nursing triage note, the ED record of treatment, or the admission consultation note; or whether a geriatric nursing specialist assessment occurred in the ED in the previous 6 months. Finally, we recorded possible indicators of limited social support (no fixed address [from patient registration and nursing triage note], primary contact is not a family member [from the emergency notes, consult, and patient registration], and no religion noted in system [from patient registration]). Patients for whom religion status was missing were classified as having “no religion.”

Analysis

These data were encrypted and linked anonymously to population-based databases to determine whether patients died within 1 year of admission to hospital. We calculated the chart-HOMR score using information from the chart review and determined its association with the outcome using bivariate logistic regression. We compared observed and expected risk of death within 1 year of admission to hospital for each chart-HOMR score value, with expected risks determined from the external validation study.³ We regressed observed death risks on expected death risks for chart-HOMR scores (clustered into 22 groups to ensure adequate numbers in each group); and we gauged overall deviations from expected risk and the relationship between the observed and expected death risk (based on the chart-HOMR score) using the line’s intercept and slope, respectively.⁶ Next, we replicated methods from our studies^2,3 to calculate the administrative-HOMR score in our study cohort using administrative databases. We compared these chart-HOMR and administrative-HOMR scores (and scores for each of its components). Finally, we determined which of the socio-functional factors were associated with 1-year death risk independent of the chart-HOMR score. We used the likelihood ratio test to determine whether these additional socio-functional factors significantly improved the model beyond the chart-HOMR score.⁷ This test subtracted the -2 logL value of the full model from that containing the chart-HOMR score alone, comparing its value to the χ² distribution (with degrees of freedom equivalent to the number of additional parameters in the nested model) to determine statistical significance. All analyses were completed using SAS v9.4 (SAS Institute Inc., Cary, North Carolina).

There were 43,883 overnight hospitalizations at our hospital in 2011, and 38,886 hospitalizations were excluded: 1883 hospitalizations were in the same-day surgery or the bone-marrow transplant unit; 2485 did not have a valid health card number; 34,515 were not randomly selected; the records of 3 randomly selected patients had been blocked by our hospital’s privacy department; and 1 patient could not be linked with the population-based administrative datasets.

The 4996 study patients were middle-aged and predominantly female (Table 2). The extensive majority of patients was admitted from the community, was independent for ADL, had a family member as the principal contact, and had no admissions by ambulance in the previous year. Most people had no significant comorbidities or ED visits in the year prior to their admission. The mean chart-HOMR score was 22 (standard deviation [SD], 12), which is associated with a 1.2% expected risk of death within 1 year of hospital admission (Appendix 1).³

In a large random sample of patients from our hospital, we found that the HOMR score using data abstracted from the medical record was significantly associated with 1-year death risk. The expected death risk based on the chart-HOMR score underestimated observed death risk but the relationship between the chart-HOMR score and death risk was similar to that in studies using administrative data. The HOMR score calculated using data from the chart was lower than that calculated using data from population-based administrative datasets; additional variables indicating patient frailty were significantly associated with 1-year death risk independent of the chart-HOMR score. Since the HOMR score was derived and initially validated using health administrative data, this study using data abstracted from the health record shows that the HOMR score has methodological generalizability.⁸

We think that our study has several notable findings. First, we found that data abstracted from the medical record can be used to calculate the HOMR score to accurately predict individual death risk. The chart-HOMR score discriminated very well between patients who did and did not die (C statistic, 0.88), which extensively exceeds the discrimination of published death risk indices (whose C statistics range between 0.69 and 0.82). It is also possible that chart abstraction for the HOMR score—without functional status—is simpler than other indices since its components are primarily very objective. (Other indices for hospital-based patients required factors that could be difficult to abstract reliably from the medical record including meeting more than 1 guideline for noncancer hospice care⁹; ambulation difficulties¹⁰; scales such as the Exton-Smith Scale or the Short Portable Mental Status Questionnaire¹¹; weight loss¹²; functional status⁴; and pressure sore risk.¹³) Although expected risks for the chart-HOMR consistently underestimated observed risks (Figure 1), the mean deviation was small (with an absolute difference of 3.5% that can be used as a correction factor when determining expected risks with HOMR scores calculated from chart review), but it was an association between the chart-HOMR score and death risk that remained consistent through the cohort. Second, we found a small but significant decrease in the chart-HOMR score vs. the administrative-HOMR score (Figure 2). Some of these underestimates such as those for the number of ED visits or admissions by ambulance were expected since population-based health administrative databases would best capture such data. However, we were surprised that the comorbidity score was less when calculated using chart vs. database data (Figure 2). This finding is distinct from studies finding that particular comorbidities are documented in the chart are sometimes not coded.^14,15 However, we identified comorbidities in the administrative databases using a 1-year ‘look-back’ period so that diagnostic codes from multiple hospitalizations (and from multiple hospitals) could be used to calculate the Charlson Comorbidity Index for a particular patient; this has been shown to increase the capture of comorbidities.¹⁶ Third, we found that variables from the chart review indicating frailty were predictive of 1-year death risk independent of the chart-HOMR score (Table 2). This illustrates that mortality risk prediction can be improved for particular patient groups by adding new covariates to the HOMR. Further work is required to determine how to incorporate these (and possibly other) covariates into the HOMR to create a unique chart-HOMR score. Finally, we found that a geriatrics assessment in the ED was associated with a significant (and notable) decrease in death risk. With these data, we are unable to indicate whether this association is causative. However, these findings indicate that the influence of emergency geriatric assessments on patient survival needs to be explored in more detail.

Several issues about our study should be considered when interpreting its results. First, this was a single-center study and the generalizability of our results to other centers is unknown. However, our study had the largest sample size of all primary data prognostic index validation studies¹ ensuring that our results are, at the very least, internally reliable. In addition, our simple random sample ensured that we studied a broad assortment of patients to be certain that our results are representative of our institution. Second, we used a single abstractor for the study, which could limit the generalizability of our results. However, almost all the data points that were abstracted for our study were very objective.

In summary, our study shows that the HOMR score can be used to accurately predict 1-year death risk using data abstracted from the patient record. These findings will aid in individual patient prognostication for clinicians and researchers.

Disclosure

The authors report no financial conflicts of interest.

A patient’s prognosis can strongly influence their medical care. Decisions about diagnostic modalities, treatment options, and the use of preventive therapies can all be affected by the likelihood of a patient’s death in the near future. For example, patients with severely limited survival might forego prophylactic therapy, avoid interventions for asymptomatic issues, and cease screening interventions. Knowing survival probability would also be very helpful as a controlling variable in research analyses whenever death risk might be a possible confounder.

Sixteen indices that aim to predict patient death risk have been described by Yourman et al.¹ They were all created from secondary analyses of clinical and administrative datasets, were applicable to patients in a variety of settings (including the community, nursing home, or hospital), and predicted survival probabilities in time horizons ranging from 6 months to 5 years. Prognostic factors that were most commonly included in these indices were comorbidity and functional status. In validation populations, the discrimination of these indices for 1-year survival in hospitalized patients was moderate (with C statistics that ranged from 0.64 to 0.79) with good calibration for broad prognostic ranges.

In 2014, we published the Hospitalized-patient One-year Mortality Risk (HOMR) score.² This study used health administrative data for all adult Ontarians admitted in 2011 to hospital under nonpsychiatric services (n = 640,022) to estimate the probability of dying within 1 year of admission to hospital (which happened in 11.7% of people). The HOMR score included 12 patient and hospitalization factors (Table 1). It was highly discriminative (C statistic, 0.923; [0.922-0.924]) and well calibrated (the mean relative difference between observed and expected death risk was 2.0% [range, 0.0% to 7.0%]). It was externally validated in more than 3 million adults from Ontario, Alberta, and Boston in whom the C statistic ranged from 0.89 to 0.92 and calibration was excellent.³ We concluded from these studies that the HOMR score is excellent for prognosticating a diverse group of patients using health administrative data.

Study Cohort

The study, which was approved by our local research ethics board, took place at the Ottawa Hospital, a 1000-bed teaching hospital that is the primary referral center in our region. We used the hospital admission registry to identify all people 18 years or older who were admitted to a nonpsychiatric service at our hospital between January 1, 2011 and December 31, 2011 (this time frame corresponds with the year used to derive the HOMR score). We excluded overnight patients in the same-day surgery or the bone-marrow transplant units (since they would not have been included in the original study) and those without a valid health card number (which was required to link to provincial data to identify outcomes). From this list, we randomly selected 5000 patients.

Primary Data Collection

For each patient, we retrieved all data required to calculate the HOMR score from the medical record (Table 1). Patient registration information in our electronic medical record was used to identify patient age, sex, admitting service, number of emergency department (ED) visits in the previous year, number of admissions in the previous year (the nursing triage note was reviewed for each admission to determine if it was by ambulance), and whether or not the patient had been discharged from hospital in the previous 30 days. The admitting service consult note was used to determine the admitting diagnosis and whether or not the patient was admitted directly to the intensive care unit. If they were present, the emergency nursing triage note, the ED record of treatment, the admission consult note, the pre-operative consult note, and consult notes were all used to determine the patient’s comorbidities, living status, and home oxygen status. Admission urgency was determined using information from the patient registration information and the ED nursing triage note. All data were abstracted from information that had been registered prior to when the patient was physically transferred to their hospital bed. This ensured that we used only data available at the start of the admission.

Patient functional status has been shown to be strongly associated with survival⁴ but HOMR only indirectly captures functional information (through the patient’s living status). We, therefore, collected more detailed functional information from the medical record by determining if the patient was dependent for any activities of daily living (ADL) from the emergency nursing triage note, the ED record of treatment, the admission consult note, and the pre-operative consultation. We also collected information that might indicate frailty, which we defined per Clegg et al.⁵ as “a state of increased vulnerability to poor resolution of homeostasis following a stress.” This information included: delirium or more than 1 fall recorded on the emergency nursing triage note, the ED record of treatment, or the admission consultation note; or whether a geriatric nursing specialist assessment occurred in the ED in the previous 6 months. Finally, we recorded possible indicators of limited social support (no fixed address [from patient registration and nursing triage note], primary contact is not a family member [from the emergency notes, consult, and patient registration], and no religion noted in system [from patient registration]). Patients for whom religion status was missing were classified as having “no religion.”

Analysis

These data were encrypted and linked anonymously to population-based databases to determine whether patients died within 1 year of admission to hospital. We calculated the chart-HOMR score using information from the chart review and determined its association with the outcome using bivariate logistic regression. We compared observed and expected risk of death within 1 year of admission to hospital for each chart-HOMR score value, with expected risks determined from the external validation study.³ We regressed observed death risks on expected death risks for chart-HOMR scores (clustered into 22 groups to ensure adequate numbers in each group); and we gauged overall deviations from expected risk and the relationship between the observed and expected death risk (based on the chart-HOMR score) using the line’s intercept and slope, respectively.⁶ Next, we replicated methods from our studies^2,3 to calculate the administrative-HOMR score in our study cohort using administrative databases. We compared these chart-HOMR and administrative-HOMR scores (and scores for each of its components). Finally, we determined which of the socio-functional factors were associated with 1-year death risk independent of the chart-HOMR score. We used the likelihood ratio test to determine whether these additional socio-functional factors significantly improved the model beyond the chart-HOMR score.⁷ This test subtracted the -2 logL value of the full model from that containing the chart-HOMR score alone, comparing its value to the χ² distribution (with degrees of freedom equivalent to the number of additional parameters in the nested model) to determine statistical significance. All analyses were completed using SAS v9.4 (SAS Institute Inc., Cary, North Carolina).

There were 43,883 overnight hospitalizations at our hospital in 2011, and 38,886 hospitalizations were excluded: 1883 hospitalizations were in the same-day surgery or the bone-marrow transplant unit; 2485 did not have a valid health card number; 34,515 were not randomly selected; the records of 3 randomly selected patients had been blocked by our hospital’s privacy department; and 1 patient could not be linked with the population-based administrative datasets.

The 4996 study patients were middle-aged and predominantly female (Table 2). The extensive majority of patients was admitted from the community, was independent for ADL, had a family member as the principal contact, and had no admissions by ambulance in the previous year. Most people had no significant comorbidities or ED visits in the year prior to their admission. The mean chart-HOMR score was 22 (standard deviation [SD], 12), which is associated with a 1.2% expected risk of death within 1 year of hospital admission (Appendix 1).³

In a large random sample of patients from our hospital, we found that the HOMR score using data abstracted from the medical record was significantly associated with 1-year death risk. The expected death risk based on the chart-HOMR score underestimated observed death risk but the relationship between the chart-HOMR score and death risk was similar to that in studies using administrative data. The HOMR score calculated using data from the chart was lower than that calculated using data from population-based administrative datasets; additional variables indicating patient frailty were significantly associated with 1-year death risk independent of the chart-HOMR score. Since the HOMR score was derived and initially validated using health administrative data, this study using data abstracted from the health record shows that the HOMR score has methodological generalizability.⁸

We think that our study has several notable findings. First, we found that data abstracted from the medical record can be used to calculate the HOMR score to accurately predict individual death risk. The chart-HOMR score discriminated very well between patients who did and did not die (C statistic, 0.88), which extensively exceeds the discrimination of published death risk indices (whose C statistics range between 0.69 and 0.82). It is also possible that chart abstraction for the HOMR score—without functional status—is simpler than other indices since its components are primarily very objective. (Other indices for hospital-based patients required factors that could be difficult to abstract reliably from the medical record including meeting more than 1 guideline for noncancer hospice care⁹; ambulation difficulties¹⁰; scales such as the Exton-Smith Scale or the Short Portable Mental Status Questionnaire¹¹; weight loss¹²; functional status⁴; and pressure sore risk.¹³) Although expected risks for the chart-HOMR consistently underestimated observed risks (Figure 1), the mean deviation was small (with an absolute difference of 3.5% that can be used as a correction factor when determining expected risks with HOMR scores calculated from chart review), but it was an association between the chart-HOMR score and death risk that remained consistent through the cohort. Second, we found a small but significant decrease in the chart-HOMR score vs. the administrative-HOMR score (Figure 2). Some of these underestimates such as those for the number of ED visits or admissions by ambulance were expected since population-based health administrative databases would best capture such data. However, we were surprised that the comorbidity score was less when calculated using chart vs. database data (Figure 2). This finding is distinct from studies finding that particular comorbidities are documented in the chart are sometimes not coded.^14,15 However, we identified comorbidities in the administrative databases using a 1-year ‘look-back’ period so that diagnostic codes from multiple hospitalizations (and from multiple hospitals) could be used to calculate the Charlson Comorbidity Index for a particular patient; this has been shown to increase the capture of comorbidities.¹⁶ Third, we found that variables from the chart review indicating frailty were predictive of 1-year death risk independent of the chart-HOMR score (Table 2). This illustrates that mortality risk prediction can be improved for particular patient groups by adding new covariates to the HOMR. Further work is required to determine how to incorporate these (and possibly other) covariates into the HOMR to create a unique chart-HOMR score. Finally, we found that a geriatrics assessment in the ED was associated with a significant (and notable) decrease in death risk. With these data, we are unable to indicate whether this association is causative. However, these findings indicate that the influence of emergency geriatric assessments on patient survival needs to be explored in more detail.

Several issues about our study should be considered when interpreting its results. First, this was a single-center study and the generalizability of our results to other centers is unknown. However, our study had the largest sample size of all primary data prognostic index validation studies¹ ensuring that our results are, at the very least, internally reliable. In addition, our simple random sample ensured that we studied a broad assortment of patients to be certain that our results are representative of our institution. Second, we used a single abstractor for the study, which could limit the generalizability of our results. However, almost all the data points that were abstracted for our study were very objective.

In summary, our study shows that the HOMR score can be used to accurately predict 1-year death risk using data abstracted from the patient record. These findings will aid in individual patient prognostication for clinicians and researchers.

Disclosure

The authors report no financial conflicts of interest.

A patient’s prognosis can strongly influence their medical care. Decisions about diagnostic modalities, treatment options, and the use of preventive therapies can all be affected by the likelihood of a patient’s death in the near future. For example, patients with severely limited survival might forego prophylactic therapy, avoid interventions for asymptomatic issues, and cease screening interventions. Knowing survival probability would also be very helpful as a controlling variable in research analyses whenever death risk might be a possible confounder.

Sixteen indices that aim to predict patient death risk have been described by Yourman et al.¹ They were all created from secondary analyses of clinical and administrative datasets, were applicable to patients in a variety of settings (including the community, nursing home, or hospital), and predicted survival probabilities in time horizons ranging from 6 months to 5 years. Prognostic factors that were most commonly included in these indices were comorbidity and functional status. In validation populations, the discrimination of these indices for 1-year survival in hospitalized patients was moderate (with C statistics that ranged from 0.64 to 0.79) with good calibration for broad prognostic ranges.

In 2014, we published the Hospitalized-patient One-year Mortality Risk (HOMR) score.² This study used health administrative data for all adult Ontarians admitted in 2011 to hospital under nonpsychiatric services (n = 640,022) to estimate the probability of dying within 1 year of admission to hospital (which happened in 11.7% of people). The HOMR score included 12 patient and hospitalization factors (Table 1). It was highly discriminative (C statistic, 0.923; [0.922-0.924]) and well calibrated (the mean relative difference between observed and expected death risk was 2.0% [range, 0.0% to 7.0%]). It was externally validated in more than 3 million adults from Ontario, Alberta, and Boston in whom the C statistic ranged from 0.89 to 0.92 and calibration was excellent.³ We concluded from these studies that the HOMR score is excellent for prognosticating a diverse group of patients using health administrative data.

Study Cohort

The study, which was approved by our local research ethics board, took place at the Ottawa Hospital, a 1000-bed teaching hospital that is the primary referral center in our region. We used the hospital admission registry to identify all people 18 years or older who were admitted to a nonpsychiatric service at our hospital between January 1, 2011 and December 31, 2011 (this time frame corresponds with the year used to derive the HOMR score). We excluded overnight patients in the same-day surgery or the bone-marrow transplant units (since they would not have been included in the original study) and those without a valid health card number (which was required to link to provincial data to identify outcomes). From this list, we randomly selected 5000 patients.

Primary Data Collection

For each patient, we retrieved all data required to calculate the HOMR score from the medical record (Table 1). Patient registration information in our electronic medical record was used to identify patient age, sex, admitting service, number of emergency department (ED) visits in the previous year, number of admissions in the previous year (the nursing triage note was reviewed for each admission to determine if it was by ambulance), and whether or not the patient had been discharged from hospital in the previous 30 days. The admitting service consult note was used to determine the admitting diagnosis and whether or not the patient was admitted directly to the intensive care unit. If they were present, the emergency nursing triage note, the ED record of treatment, the admission consult note, the pre-operative consult note, and consult notes were all used to determine the patient’s comorbidities, living status, and home oxygen status. Admission urgency was determined using information from the patient registration information and the ED nursing triage note. All data were abstracted from information that had been registered prior to when the patient was physically transferred to their hospital bed. This ensured that we used only data available at the start of the admission.

Patient functional status has been shown to be strongly associated with survival⁴ but HOMR only indirectly captures functional information (through the patient’s living status). We, therefore, collected more detailed functional information from the medical record by determining if the patient was dependent for any activities of daily living (ADL) from the emergency nursing triage note, the ED record of treatment, the admission consult note, and the pre-operative consultation. We also collected information that might indicate frailty, which we defined per Clegg et al.⁵ as “a state of increased vulnerability to poor resolution of homeostasis following a stress.” This information included: delirium or more than 1 fall recorded on the emergency nursing triage note, the ED record of treatment, or the admission consultation note; or whether a geriatric nursing specialist assessment occurred in the ED in the previous 6 months. Finally, we recorded possible indicators of limited social support (no fixed address [from patient registration and nursing triage note], primary contact is not a family member [from the emergency notes, consult, and patient registration], and no religion noted in system [from patient registration]). Patients for whom religion status was missing were classified as having “no religion.”

Analysis

These data were encrypted and linked anonymously to population-based databases to determine whether patients died within 1 year of admission to hospital. We calculated the chart-HOMR score using information from the chart review and determined its association with the outcome using bivariate logistic regression. We compared observed and expected risk of death within 1 year of admission to hospital for each chart-HOMR score value, with expected risks determined from the external validation study.³ We regressed observed death risks on expected death risks for chart-HOMR scores (clustered into 22 groups to ensure adequate numbers in each group); and we gauged overall deviations from expected risk and the relationship between the observed and expected death risk (based on the chart-HOMR score) using the line’s intercept and slope, respectively.⁶ Next, we replicated methods from our studies^2,3 to calculate the administrative-HOMR score in our study cohort using administrative databases. We compared these chart-HOMR and administrative-HOMR scores (and scores for each of its components). Finally, we determined which of the socio-functional factors were associated with 1-year death risk independent of the chart-HOMR score. We used the likelihood ratio test to determine whether these additional socio-functional factors significantly improved the model beyond the chart-HOMR score.⁷ This test subtracted the -2 logL value of the full model from that containing the chart-HOMR score alone, comparing its value to the χ² distribution (with degrees of freedom equivalent to the number of additional parameters in the nested model) to determine statistical significance. All analyses were completed using SAS v9.4 (SAS Institute Inc., Cary, North Carolina).

There were 43,883 overnight hospitalizations at our hospital in 2011, and 38,886 hospitalizations were excluded: 1883 hospitalizations were in the same-day surgery or the bone-marrow transplant unit; 2485 did not have a valid health card number; 34,515 were not randomly selected; the records of 3 randomly selected patients had been blocked by our hospital’s privacy department; and 1 patient could not be linked with the population-based administrative datasets.

The 4996 study patients were middle-aged and predominantly female (Table 2). The extensive majority of patients was admitted from the community, was independent for ADL, had a family member as the principal contact, and had no admissions by ambulance in the previous year. Most people had no significant comorbidities or ED visits in the year prior to their admission. The mean chart-HOMR score was 22 (standard deviation [SD], 12), which is associated with a 1.2% expected risk of death within 1 year of hospital admission (Appendix 1).³

In a large random sample of patients from our hospital, we found that the HOMR score using data abstracted from the medical record was significantly associated with 1-year death risk. The expected death risk based on the chart-HOMR score underestimated observed death risk but the relationship between the chart-HOMR score and death risk was similar to that in studies using administrative data. The HOMR score calculated using data from the chart was lower than that calculated using data from population-based administrative datasets; additional variables indicating patient frailty were significantly associated with 1-year death risk independent of the chart-HOMR score. Since the HOMR score was derived and initially validated using health administrative data, this study using data abstracted from the health record shows that the HOMR score has methodological generalizability.⁸

We think that our study has several notable findings. First, we found that data abstracted from the medical record can be used to calculate the HOMR score to accurately predict individual death risk. The chart-HOMR score discriminated very well between patients who did and did not die (C statistic, 0.88), which extensively exceeds the discrimination of published death risk indices (whose C statistics range between 0.69 and 0.82). It is also possible that chart abstraction for the HOMR score—without functional status—is simpler than other indices since its components are primarily very objective. (Other indices for hospital-based patients required factors that could be difficult to abstract reliably from the medical record including meeting more than 1 guideline for noncancer hospice care⁹; ambulation difficulties¹⁰; scales such as the Exton-Smith Scale or the Short Portable Mental Status Questionnaire¹¹; weight loss¹²; functional status⁴; and pressure sore risk.¹³) Although expected risks for the chart-HOMR consistently underestimated observed risks (Figure 1), the mean deviation was small (with an absolute difference of 3.5% that can be used as a correction factor when determining expected risks with HOMR scores calculated from chart review), but it was an association between the chart-HOMR score and death risk that remained consistent through the cohort. Second, we found a small but significant decrease in the chart-HOMR score vs. the administrative-HOMR score (Figure 2). Some of these underestimates such as those for the number of ED visits or admissions by ambulance were expected since population-based health administrative databases would best capture such data. However, we were surprised that the comorbidity score was less when calculated using chart vs. database data (Figure 2). This finding is distinct from studies finding that particular comorbidities are documented in the chart are sometimes not coded.^14,15 However, we identified comorbidities in the administrative databases using a 1-year ‘look-back’ period so that diagnostic codes from multiple hospitalizations (and from multiple hospitals) could be used to calculate the Charlson Comorbidity Index for a particular patient; this has been shown to increase the capture of comorbidities.¹⁶ Third, we found that variables from the chart review indicating frailty were predictive of 1-year death risk independent of the chart-HOMR score (Table 2). This illustrates that mortality risk prediction can be improved for particular patient groups by adding new covariates to the HOMR. Further work is required to determine how to incorporate these (and possibly other) covariates into the HOMR to create a unique chart-HOMR score. Finally, we found that a geriatrics assessment in the ED was associated with a significant (and notable) decrease in death risk. With these data, we are unable to indicate whether this association is causative. However, these findings indicate that the influence of emergency geriatric assessments on patient survival needs to be explored in more detail.

Several issues about our study should be considered when interpreting its results. First, this was a single-center study and the generalizability of our results to other centers is unknown. However, our study had the largest sample size of all primary data prognostic index validation studies¹ ensuring that our results are, at the very least, internally reliable. In addition, our simple random sample ensured that we studied a broad assortment of patients to be certain that our results are representative of our institution. Second, we used a single abstractor for the study, which could limit the generalizability of our results. However, almost all the data points that were abstracted for our study were very objective.

In summary, our study shows that the HOMR score can be used to accurately predict 1-year death risk using data abstracted from the patient record. These findings will aid in individual patient prognostication for clinicians and researchers.

Disclosure

The authors report no financial conflicts of interest.

Pneumonia is a leading cause of hospitalizations in the U.S., accounting for more than 1.1 million discharges annually.¹ Pneumonia is frequently complicated by hospital readmission, which is costly and potentially avoidable.^2,3 Due to financial penalties imposed on hospitals for higher than expected 30-day readmission rates, there is increasing attention to implementing interventions to reduce readmissions in this population.^4,5 However, because these programs are resource-intensive, interventions are thought to be most cost-effective if they are targeted to high-risk individuals who are most likely to benefit.^6-8

Current pneumonia-specific readmission risk-prediction models that could enable identification of high-risk patients suffer from poor predictive ability, greatly limiting their use, and most were validated among older adults or by using data from single academic medical centers, limiting their generalizability.^9-14 A potential reason for poor predictive accuracy is the omission of known robust clinical predictors of pneumonia-related outcomes, including pneumonia severity of illness and stability on discharge.^15-17 Approaches using electronic health record (EHR) data, which include this clinically granular data, could enable hospitals to more accurately and pragmatically identify high-risk patients during the index hospitalization and enable interventions to be initiated prior to discharge.

An alternative strategy to identifying high-risk patients for readmission is to use a multi-condition risk-prediction model. Developing and implementing models for every condition may be time-consuming and costly. We have derived and validated 2 multi-condition risk-prediction models using EHR data—1 using data from the first day of hospital admission (‘first-day’ model), and the second incorporating data from the entire hospitalization (‘full-stay’ model) to reflect in-hospital complications and clinical stability at discharge.^18,19 However, it is unknown if a multi-condition model for pneumonia would perform as well as a disease-specific model.

This study aimed to develop 2 EHR-based pneumonia-specific readmission risk-prediction models using data routinely collected in clinical practice—a ‘first-day’ and a ‘full-stay’ model—and compare the performance of each model to: 1) one another; 2) the corresponding multi-condition EHR model; and 3) to other potentially useful models in predicting pneumonia readmissions (the Centers for Medicare and Medicaid Services [CMS] pneumonia model, and 2 commonly used pneumonia severity of illness scores validated for predicting mortality). We hypothesized that the pneumonia-specific EHR models would outperform other models; and the full-stay pneumonia-specific model would outperform the first-day pneumonia-specific model.

Study Design, Population, and Data Sources

We conducted an observational study using EHR data collected from 6 hospitals (including safety net, community, teaching, and nonteaching hospitals) in north Texas between November 2009 and October 2010, All hospitals used the Epic EHR (Epic Systems Corporation, Verona, WI). Details of this cohort have been published.^18,19

We included consecutive hospitalizations among adults 18 years and older discharged from any medicine service with principal discharge diagnoses of pneumonia (ICD-9-CM codes 480-483, 485, 486-487), sepsis (ICD-9-CM codes 038, 995.91, 995.92, 785.52), or respiratory failure (ICD-9-CM codes 518.81, 518.82, 518.84, 799.1) when the latter 2 were also accompanied by a secondary diagnosis of pneumonia.²⁰ For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization or within 30 days of discharge, were transferred to another acute care facility, or left against medical advice.

Outcomes

The primary outcome was all-cause 30-day readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100-mile radius of Dallas, ascertained from an all-payer regional hospitalization database.

Predictor Variables for the Pneumonia-Specific Readmission Models

The selection of candidate predictors was informed by our validated multi-condition risk-prediction models using EHR data available within 24 hours of admission (‘first-day’ multi-condition EHR model) or during the entire hospitalization (‘full-stay’ multi-condition EHR model).^18,19 For the pneumonia-specific models, we included all variables in our published multi-condition models as candidate predictors, including sociodemographics, prior utilization, Charlson Comorbidity Index, select laboratory and vital sign abnormalities, length of stay, hospital complications (eg, venous thromboembolism), vital sign instabilities, and disposition status (see Supplemental Table 1 for complete list of variables). We also assessed additional variables specific to pneumonia for inclusion that were: (1) available in the EHR of all participating hospitals; (2) routinely collected or available at the time of admission or discharge; and (3) plausible predictors of adverse outcomes based on literature and clinical expertise. These included select comorbidities (eg, psychiatric conditions, chronic lung disease, history of pneumonia),^10,11,21,22 the pneumonia severity index (PSI),^16,23,24 intensive care unit stay, and receipt of invasive or noninvasive ventilation. We used a modified PSI score because certain data elements were missing. The modified PSI (henceforth referred to as PSI) did not include nursing home residence and included diagnostic codes as proxies for the presence of pleural effusion (ICD-9-CM codes 510, 511.1, and 511.9) and altered mental status (ICD-9-CM codes 780.0X, 780.97, 293.0, 293.1, and 348.3X).

Statistical Analysis

Model Derivation. Candidate predictor variables were classified as available in the EHR within 24 hours of admission and/or at the time of discharge. For example, socioeconomic factors could be ascertained within the first day of hospitalization, whereas length of stay would not be available until the day of discharge. Predictors with missing values were assumed to be normal (less than 1% missing for each variable). Univariate relationships between readmission and each candidate predictor were assessed in the overall cohort using a pre-specified significance threshold of P ≤ 0.10. Significant variables were entered in the respective first-day and full-stay pneumonia-specific multivariable logistic regression models using stepwise-backward selection with a pre-specified significance threshold of P ≤ 0.05. In sensitivity analyses, we alternately derived our models using stepwise-forward selection, as well as stepwise-backward selection minimizing the Bayesian information criterion and Akaike information criterion separately. These alternate modeling strategies yielded identical predictors to our final models.

Model Validation. Model validation was performed using 5-fold cross-validation, with the overall cohort randomly divided into 5 equal-size subsets.²⁵ For each cycle, 4 subsets were used for training to estimate model coefficients, and the fifth subset was used for validation. This cycle was repeated 5 times with each randomly-divided subset used once as the validation set. We repeated this entire process 50 times and averaged the C statistic estimates to derive an optimism-corrected C statistic. Model calibration was assessed qualitatively by comparing predicted to observed probabilities of readmission by quintiles of predicted risk, and with the Hosmer-Lemeshow goodness-of-fit test.

Comparison to Other Models. The main comparisons of the first-day and full-stay pneumonia-specific EHR model performance were to each other and the corresponding multi-condition EHR model.^18,19 The multi-condition EHR models were separately derived and validated within the larger parent cohort from which this study cohort was derived, and outperformed the CMS all-cause model, the HOSPITAL model, and the LACE index.¹⁹ To further triangulate our findings, given the lack of other rigorously validated pneumonia-specific risk-prediction models for readmission,¹⁴ we compared the pneumonia-specific EHR models to the CMS pneumonia model derived from administrative claims data,¹⁰ and 2 commonly used risk-prediction scores for short-term mortality among patients with community-acquired pneumonia, the PSI and CURB-65 scores.¹⁶ Although derived and validated using patient-level data, the CMS model was developed to benchmark hospitals according to hospital-level readmission rates.¹⁰ The CURB-65 score in this study was also modified to include the same altered mental status diagnostic codes according to the modified PSI as a proxy for “confusion.” Both the PSI and CURB-65 scores were calculated using the most abnormal values within the first 24 hours of admission. The ‘updated’ PSI and the ‘updated’ CURB-65 were calculated using the most abnormal values within 24 hours prior to discharge, or the last known observation prior to discharge if no results were recorded within this time period. A complete list of variables for each of the comparison models are shown in Supplemental Table 1.

We assessed model performance by calculating the C statistic, integrated discrimination index, and net reclassification index (NRI) compared to our pneumonia-specific models. The integrated discrimination index is the difference in the mean predicted probability of readmission between patients who were and were not actually readmitted between 2 models, where more positive values suggest improvement in model performance compared to a reference model.²⁶ The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.²⁷ Here, we calculated a category-based NRI to evaluate the performance of pneumonia-specific models in correctly classifying individuals with and without readmissions into the 2 highest readmission risk quintiles vs the lowest 3 risk quintiles compared to other models.²⁷ This pre-specified cutoff is relevant for hospitals interested in identifying the highest risk individuals for targeted intervention.⁷ Finally, we assessed calibration of comparator models in our cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, Texas). This study was approved by the University of Texas Southwestern Medical Center Institutional Review Board.

Of 1463 index hospitalizations (Supplemental Figure 1), the 30-day all-cause readmission rate was 13.6%. Individuals with a 30-day readmission had markedly different sociodemographic and clinical characteristics compared to those not readmitted (Table 1; see Supplemental Table 2 for additional clinical characteristics).

Derivation, Validation, and Performance of the Pneumonia-Specific Readmission Risk-Prediction Models

The final first-day pneumonia-specific EHR model included 7 variables, including sociodemographic characteristics; prior hospitalizations; thrombocytosis, and PSI (Table 2). The first-day pneumonia-specific model had adequate discrimination (C statistic, 0.695; optimism-corrected C statistic 0.675, 95% confidence interval [CI], 0.667-0.685; Table 3). It also effectively stratified individuals across a broad range of risk (average predicted decile of risk ranged from 4% to 33%; Table 3) and was well calibrated (Supplemental Table 3).

The final full-stay pneumonia-specific EHR readmission model included 8 predictors, including 3 variables from the first-day model (median income, thrombocytosis, and prior hospitalizations; Table 2). The full-stay pneumonia-specific EHR model also included vital sign instabilities on discharge, updated PSI, and disposition status (ie, being discharged with home health or to a post-acute care facility was associated with greater odds of readmission, and hospice with lower odds). The full-stay pneumonia-specific EHR model had good discrimination (C statistic, 0.731; optimism-corrected C statistic, 0.714; 95% CI, 0.706-0.720), and stratified individuals across a broad range of risk (average predicted decile of risk ranged from 3% to 37%; Table 3), and was also well calibrated (Supplemental Table 3).

First-Day Pneumonia-Specific EHR Model vs First-Day Multi-Condition EHR Model

The first-day pneumonia-specific EHR model outperformed the first-day multi-condition EHR model with better discrimination (P = 0.029) and more correctly classified individuals in the top 2 highest risk quintiles vs the bottom 3 risk quintiles (Table 3, Supplemental Table 4, and Supplemental Figure 2A). With respect to calibration, the first-day multi-condition EHR model overestimated risk among the highest quintile risk group compared to the first-day pneumonia-specific EHR model (Figure 1A, 1B).

Full-Stay Pneumonia-Specific EHR Model vs Other Models

The full-stay pneumonia-specific EHR model comparatively outperformed the corresponding full-stay multi-condition EHR model, as well as the first-day pneumonia-specific EHR model, the CMS pneumonia model, the updated PSI, and the updated CURB-65 (Table 3, Supplemental Table 5, Supplemental Table 6, and Supplemental Figures 2B and 2C). Compared to the full-stay multi-condition and first-day pneumonia-specific EHR models, the full-stay pneumonia-specific EHR model had better discrimination, better reclassification (NRI, 0.09 and 0.08, respectively), and was able to stratify individuals across a broader range of readmission risk (Table 3). It also had better calibration in the highest quintile risk group compared to the full-stay multi-condition EHR model (Figure 1C and 1D).

Updated vs First-Day Modified PSI and CURB-65 Scores

The updated PSI was more strongly predictive of readmission than the PSI calculated on the day of admission (Wald test, 9.83; P = 0.002). Each 10-point increase in the updated PSI was associated with a 22% increased odds of readmission vs an 11% increase for the PSI calculated upon admission (Table 2). The improved predictive ability of the updated PSI and CURB-65 scores was also reflected in the superior discrimination and calibration vs the respective first-day pneumonia severity of illness scores (Table 3).

Using routinely available EHR data from 6 diverse hospitals, we developed 2 pneumonia-specific readmission risk-prediction models that aimed to allow hospitals to identify patients hospitalized with pneumonia at high risk for readmission. Overall, we found that a pneumonia-specific model using EHR data from the entire hospitalization outperformed all other models—including the first-day pneumonia-specific model using data present only on admission, our own multi-condition EHR models, and the CMS pneumonia model based on administrative claims data—in all aspects of model performance (discrimination, calibration, and reclassification). We found that socioeconomic status, prior hospitalizations, thrombocytosis, and measures of clinical severity and stability were important predictors of 30-day all-cause readmissions among patients hospitalized with pneumonia. Additionally, an updated discharge PSI score was a stronger independent predictor of readmissions compared to the PSI score calculated upon admission; and inclusion of the updated PSI in our full-stay pneumonia model led to improved prediction of 30-day readmissions.

The marked improvement in performance of the full-stay pneumonia-specific EHR model compared to the first-day pneumonia-specific model suggests that clinical stability and trajectory during hospitalization (as modeled through disposition status, updated PSI, and vital sign instabilities at discharge) are important predictors of 30-day readmission among patients hospitalized for pneumonia, which was not the case for our EHR-based multi-condition models.¹⁹ With the inclusion of these measures, the full-stay pneumonia-specific model correctly reclassified an additional 8% of patients according to their true risk compared to the first-day pneumonia-specific model. One implication of these findings is that hospitals interested in targeting their highest risk individuals with pneumonia for transitional care interventions could do so using the first-day pneumonia-specific EHR model and could refine their targeted strategy at the time of discharge by using the full-stay pneumonia model. This staged risk-prediction strategy would enable hospitals to initiate transitional care interventions for high-risk individuals in the inpatient setting (ie, patient education).⁷ Then, hospitals could enroll both persistent and newly identified high-risk individuals for outpatient interventions (ie, follow-up telephone call) in the immediate post-discharge period, an interval characterized by heightened vulnerability for adverse events,²⁸ based on patients’ illness severity and stability at discharge. This approach can be implemented by hospitals by building these risk-prediction models directly into the EHR, or by extracting EHR data in near real time as our group has done successfully for heart failure.⁷

Another key implication of our study is that, for pneumonia, a disease-specific modeling approach has better predictive ability than using a multi-condition model. Compared to multi-condition models, the first-day and full-stay pneumonia-specific EHR models correctly reclassified an additional 6% and 9% of patients, respectively. Thus, hospitals interested in identifying the highest risk patients with pneumonia for targeted interventions should do so using the disease-specific models, if the costs and resources of doing so are within reach of the healthcare system.

An additional novel finding of our study is the added value of an updated PSI for predicting adverse events. Studies of pneumonia severity of illness scores have calculated the PSI and CURB-65 scores using data present only on admission.^16,24 While our study also confirms that the PSI calculated upon admission is a significant predictor of readmission,^23,29 this study extends this work by showing that an updated PSI score calculated at the time of discharge is an even stronger predictor for readmission, and its inclusion in the model significantly improves risk stratification and prognostication.

Our study was noteworthy for several strengths. First, we used data from a common EHR system, thus potentially allowing for the implementation of the pneumonia-specific models in real time across a number of hospitals. The use of routinely collected data for risk-prediction modeling makes this approach scalable and sustainable, because it obviates the need for burdensome data collection and entry. Second, to our knowledge, this is the first study to measure the additive influence of illness severity and stability at discharge on the readmission risk among patients hospitalized with pneumonia. Third, our study population was derived from 6 hospitals diverse in payer status, age, race/ethnicity, and socioeconomic status. Fourth, our models are less likely to be overfit to the idiosyncrasies of our data given that several predictors included in our final pneumonia-specific models have been associated with readmission in this population, including marital status,^13,30 income,^11,31 prior hospitalizations,^11,13 thrombocytosis,^32-34 and vital sign instabilities on discharge.¹⁷ Lastly, the discrimination of the CMS pneumonia model in our cohort (C statistic, 0.64) closely matched the discrimination observed in 4 independent cohorts (C statistic, 0.63), suggesting adequate generalizability of our study setting and population.^10,12

Our results should be interpreted in the context of several limitations. First, generalizability to other regions beyond north Texas is unknown. Second, although we included a diverse cohort of safety net, community, teaching, and nonteaching hospitals, the pneumonia-specific models were not externally validated in a separate cohort, which may lead to more optimistic estimates of model performance. Third, PSI and CURB-65 scores were modified to use diagnostic codes for altered mental status and pleural effusion, and omitted nursing home residence. Thus, the independent associations for the PSI and CURB-65 scores and their predictive ability are likely attenuated. Fourth, we were unable to include data on medications (antibiotics and steroid use) and outpatient visits, which may influence readmission risk.^2,9,13,35-40 Fifth, we included only the first pneumonia hospitalization per patient in this study. Had we included multiple hospitalizations per patient, we anticipate better model performance for the 2 pneumonia-specific EHR models since prior hospitalization was a robust predictor of readmission.

In conclusion, the full-stay pneumonia-specific EHR readmission risk-prediction model outperformed the first-day pneumonia-specific model, multi-condition EHR models, and the CMS pneumonia model. This suggests that: measures of clinical severity and stability at the time of discharge are important predictors for identifying patients at highest risk for readmission; and that EHR data routinely collected for clinical practice can be used to accurately predict risk of readmission among patients hospitalized for pneumonia.

Acknowledgments

The authors would like to acknowledge Ruben Amarasingham, MD, MBA, president and chief executive officer of Parkland Center for Clinical Innovation, and Ferdinand Velasco, MD, chief health information officer at Texas Health Resources for their assistance in assembling the 6-hospital cohort used in this study.

Disclosures

This work was supported by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24 HS022418-01); the Commonwealth Foundation (#20100323); the UT Southwestern KL2 Scholars Program supported by the National Institutes of Health (KL2 TR001103 to ANM and OKN); and the National Center for Advancing Translational Sciences at the National Institute of Health (U54 RFA-TR-12-006 to E.A.H.). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no financial conflicts of interest to disclose

Pneumonia is a leading cause of hospitalizations in the U.S., accounting for more than 1.1 million discharges annually.¹ Pneumonia is frequently complicated by hospital readmission, which is costly and potentially avoidable.^2,3 Due to financial penalties imposed on hospitals for higher than expected 30-day readmission rates, there is increasing attention to implementing interventions to reduce readmissions in this population.^4,5 However, because these programs are resource-intensive, interventions are thought to be most cost-effective if they are targeted to high-risk individuals who are most likely to benefit.^6-8

Current pneumonia-specific readmission risk-prediction models that could enable identification of high-risk patients suffer from poor predictive ability, greatly limiting their use, and most were validated among older adults or by using data from single academic medical centers, limiting their generalizability.^9-14 A potential reason for poor predictive accuracy is the omission of known robust clinical predictors of pneumonia-related outcomes, including pneumonia severity of illness and stability on discharge.^15-17 Approaches using electronic health record (EHR) data, which include this clinically granular data, could enable hospitals to more accurately and pragmatically identify high-risk patients during the index hospitalization and enable interventions to be initiated prior to discharge.

An alternative strategy to identifying high-risk patients for readmission is to use a multi-condition risk-prediction model. Developing and implementing models for every condition may be time-consuming and costly. We have derived and validated 2 multi-condition risk-prediction models using EHR data—1 using data from the first day of hospital admission (‘first-day’ model), and the second incorporating data from the entire hospitalization (‘full-stay’ model) to reflect in-hospital complications and clinical stability at discharge.^18,19 However, it is unknown if a multi-condition model for pneumonia would perform as well as a disease-specific model.

This study aimed to develop 2 EHR-based pneumonia-specific readmission risk-prediction models using data routinely collected in clinical practice—a ‘first-day’ and a ‘full-stay’ model—and compare the performance of each model to: 1) one another; 2) the corresponding multi-condition EHR model; and 3) to other potentially useful models in predicting pneumonia readmissions (the Centers for Medicare and Medicaid Services [CMS] pneumonia model, and 2 commonly used pneumonia severity of illness scores validated for predicting mortality). We hypothesized that the pneumonia-specific EHR models would outperform other models; and the full-stay pneumonia-specific model would outperform the first-day pneumonia-specific model.

Study Design, Population, and Data Sources

We conducted an observational study using EHR data collected from 6 hospitals (including safety net, community, teaching, and nonteaching hospitals) in north Texas between November 2009 and October 2010, All hospitals used the Epic EHR (Epic Systems Corporation, Verona, WI). Details of this cohort have been published.^18,19

We included consecutive hospitalizations among adults 18 years and older discharged from any medicine service with principal discharge diagnoses of pneumonia (ICD-9-CM codes 480-483, 485, 486-487), sepsis (ICD-9-CM codes 038, 995.91, 995.92, 785.52), or respiratory failure (ICD-9-CM codes 518.81, 518.82, 518.84, 799.1) when the latter 2 were also accompanied by a secondary diagnosis of pneumonia.²⁰ For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization or within 30 days of discharge, were transferred to another acute care facility, or left against medical advice.

Outcomes

The primary outcome was all-cause 30-day readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100-mile radius of Dallas, ascertained from an all-payer regional hospitalization database.

Predictor Variables for the Pneumonia-Specific Readmission Models

The selection of candidate predictors was informed by our validated multi-condition risk-prediction models using EHR data available within 24 hours of admission (‘first-day’ multi-condition EHR model) or during the entire hospitalization (‘full-stay’ multi-condition EHR model).^18,19 For the pneumonia-specific models, we included all variables in our published multi-condition models as candidate predictors, including sociodemographics, prior utilization, Charlson Comorbidity Index, select laboratory and vital sign abnormalities, length of stay, hospital complications (eg, venous thromboembolism), vital sign instabilities, and disposition status (see Supplemental Table 1 for complete list of variables). We also assessed additional variables specific to pneumonia for inclusion that were: (1) available in the EHR of all participating hospitals; (2) routinely collected or available at the time of admission or discharge; and (3) plausible predictors of adverse outcomes based on literature and clinical expertise. These included select comorbidities (eg, psychiatric conditions, chronic lung disease, history of pneumonia),^10,11,21,22 the pneumonia severity index (PSI),^16,23,24 intensive care unit stay, and receipt of invasive or noninvasive ventilation. We used a modified PSI score because certain data elements were missing. The modified PSI (henceforth referred to as PSI) did not include nursing home residence and included diagnostic codes as proxies for the presence of pleural effusion (ICD-9-CM codes 510, 511.1, and 511.9) and altered mental status (ICD-9-CM codes 780.0X, 780.97, 293.0, 293.1, and 348.3X).

Statistical Analysis

Model Derivation. Candidate predictor variables were classified as available in the EHR within 24 hours of admission and/or at the time of discharge. For example, socioeconomic factors could be ascertained within the first day of hospitalization, whereas length of stay would not be available until the day of discharge. Predictors with missing values were assumed to be normal (less than 1% missing for each variable). Univariate relationships between readmission and each candidate predictor were assessed in the overall cohort using a pre-specified significance threshold of P ≤ 0.10. Significant variables were entered in the respective first-day and full-stay pneumonia-specific multivariable logistic regression models using stepwise-backward selection with a pre-specified significance threshold of P ≤ 0.05. In sensitivity analyses, we alternately derived our models using stepwise-forward selection, as well as stepwise-backward selection minimizing the Bayesian information criterion and Akaike information criterion separately. These alternate modeling strategies yielded identical predictors to our final models.

Model Validation. Model validation was performed using 5-fold cross-validation, with the overall cohort randomly divided into 5 equal-size subsets.²⁵ For each cycle, 4 subsets were used for training to estimate model coefficients, and the fifth subset was used for validation. This cycle was repeated 5 times with each randomly-divided subset used once as the validation set. We repeated this entire process 50 times and averaged the C statistic estimates to derive an optimism-corrected C statistic. Model calibration was assessed qualitatively by comparing predicted to observed probabilities of readmission by quintiles of predicted risk, and with the Hosmer-Lemeshow goodness-of-fit test.

Comparison to Other Models. The main comparisons of the first-day and full-stay pneumonia-specific EHR model performance were to each other and the corresponding multi-condition EHR model.^18,19 The multi-condition EHR models were separately derived and validated within the larger parent cohort from which this study cohort was derived, and outperformed the CMS all-cause model, the HOSPITAL model, and the LACE index.¹⁹ To further triangulate our findings, given the lack of other rigorously validated pneumonia-specific risk-prediction models for readmission,¹⁴ we compared the pneumonia-specific EHR models to the CMS pneumonia model derived from administrative claims data,¹⁰ and 2 commonly used risk-prediction scores for short-term mortality among patients with community-acquired pneumonia, the PSI and CURB-65 scores.¹⁶ Although derived and validated using patient-level data, the CMS model was developed to benchmark hospitals according to hospital-level readmission rates.¹⁰ The CURB-65 score in this study was also modified to include the same altered mental status diagnostic codes according to the modified PSI as a proxy for “confusion.” Both the PSI and CURB-65 scores were calculated using the most abnormal values within the first 24 hours of admission. The ‘updated’ PSI and the ‘updated’ CURB-65 were calculated using the most abnormal values within 24 hours prior to discharge, or the last known observation prior to discharge if no results were recorded within this time period. A complete list of variables for each of the comparison models are shown in Supplemental Table 1.

We assessed model performance by calculating the C statistic, integrated discrimination index, and net reclassification index (NRI) compared to our pneumonia-specific models. The integrated discrimination index is the difference in the mean predicted probability of readmission between patients who were and were not actually readmitted between 2 models, where more positive values suggest improvement in model performance compared to a reference model.²⁶ The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.²⁷ Here, we calculated a category-based NRI to evaluate the performance of pneumonia-specific models in correctly classifying individuals with and without readmissions into the 2 highest readmission risk quintiles vs the lowest 3 risk quintiles compared to other models.²⁷ This pre-specified cutoff is relevant for hospitals interested in identifying the highest risk individuals for targeted intervention.⁷ Finally, we assessed calibration of comparator models in our cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, Texas). This study was approved by the University of Texas Southwestern Medical Center Institutional Review Board.

Of 1463 index hospitalizations (Supplemental Figure 1), the 30-day all-cause readmission rate was 13.6%. Individuals with a 30-day readmission had markedly different sociodemographic and clinical characteristics compared to those not readmitted (Table 1; see Supplemental Table 2 for additional clinical characteristics).

Derivation, Validation, and Performance of the Pneumonia-Specific Readmission Risk-Prediction Models

The final first-day pneumonia-specific EHR model included 7 variables, including sociodemographic characteristics; prior hospitalizations; thrombocytosis, and PSI (Table 2). The first-day pneumonia-specific model had adequate discrimination (C statistic, 0.695; optimism-corrected C statistic 0.675, 95% confidence interval [CI], 0.667-0.685; Table 3). It also effectively stratified individuals across a broad range of risk (average predicted decile of risk ranged from 4% to 33%; Table 3) and was well calibrated (Supplemental Table 3).

The final full-stay pneumonia-specific EHR readmission model included 8 predictors, including 3 variables from the first-day model (median income, thrombocytosis, and prior hospitalizations; Table 2). The full-stay pneumonia-specific EHR model also included vital sign instabilities on discharge, updated PSI, and disposition status (ie, being discharged with home health or to a post-acute care facility was associated with greater odds of readmission, and hospice with lower odds). The full-stay pneumonia-specific EHR model had good discrimination (C statistic, 0.731; optimism-corrected C statistic, 0.714; 95% CI, 0.706-0.720), and stratified individuals across a broad range of risk (average predicted decile of risk ranged from 3% to 37%; Table 3), and was also well calibrated (Supplemental Table 3).

First-Day Pneumonia-Specific EHR Model vs First-Day Multi-Condition EHR Model

The first-day pneumonia-specific EHR model outperformed the first-day multi-condition EHR model with better discrimination (P = 0.029) and more correctly classified individuals in the top 2 highest risk quintiles vs the bottom 3 risk quintiles (Table 3, Supplemental Table 4, and Supplemental Figure 2A). With respect to calibration, the first-day multi-condition EHR model overestimated risk among the highest quintile risk group compared to the first-day pneumonia-specific EHR model (Figure 1A, 1B).

Full-Stay Pneumonia-Specific EHR Model vs Other Models

The full-stay pneumonia-specific EHR model comparatively outperformed the corresponding full-stay multi-condition EHR model, as well as the first-day pneumonia-specific EHR model, the CMS pneumonia model, the updated PSI, and the updated CURB-65 (Table 3, Supplemental Table 5, Supplemental Table 6, and Supplemental Figures 2B and 2C). Compared to the full-stay multi-condition and first-day pneumonia-specific EHR models, the full-stay pneumonia-specific EHR model had better discrimination, better reclassification (NRI, 0.09 and 0.08, respectively), and was able to stratify individuals across a broader range of readmission risk (Table 3). It also had better calibration in the highest quintile risk group compared to the full-stay multi-condition EHR model (Figure 1C and 1D).

Updated vs First-Day Modified PSI and CURB-65 Scores

The updated PSI was more strongly predictive of readmission than the PSI calculated on the day of admission (Wald test, 9.83; P = 0.002). Each 10-point increase in the updated PSI was associated with a 22% increased odds of readmission vs an 11% increase for the PSI calculated upon admission (Table 2). The improved predictive ability of the updated PSI and CURB-65 scores was also reflected in the superior discrimination and calibration vs the respective first-day pneumonia severity of illness scores (Table 3).

Using routinely available EHR data from 6 diverse hospitals, we developed 2 pneumonia-specific readmission risk-prediction models that aimed to allow hospitals to identify patients hospitalized with pneumonia at high risk for readmission. Overall, we found that a pneumonia-specific model using EHR data from the entire hospitalization outperformed all other models—including the first-day pneumonia-specific model using data present only on admission, our own multi-condition EHR models, and the CMS pneumonia model based on administrative claims data—in all aspects of model performance (discrimination, calibration, and reclassification). We found that socioeconomic status, prior hospitalizations, thrombocytosis, and measures of clinical severity and stability were important predictors of 30-day all-cause readmissions among patients hospitalized with pneumonia. Additionally, an updated discharge PSI score was a stronger independent predictor of readmissions compared to the PSI score calculated upon admission; and inclusion of the updated PSI in our full-stay pneumonia model led to improved prediction of 30-day readmissions.

The marked improvement in performance of the full-stay pneumonia-specific EHR model compared to the first-day pneumonia-specific model suggests that clinical stability and trajectory during hospitalization (as modeled through disposition status, updated PSI, and vital sign instabilities at discharge) are important predictors of 30-day readmission among patients hospitalized for pneumonia, which was not the case for our EHR-based multi-condition models.¹⁹ With the inclusion of these measures, the full-stay pneumonia-specific model correctly reclassified an additional 8% of patients according to their true risk compared to the first-day pneumonia-specific model. One implication of these findings is that hospitals interested in targeting their highest risk individuals with pneumonia for transitional care interventions could do so using the first-day pneumonia-specific EHR model and could refine their targeted strategy at the time of discharge by using the full-stay pneumonia model. This staged risk-prediction strategy would enable hospitals to initiate transitional care interventions for high-risk individuals in the inpatient setting (ie, patient education).⁷ Then, hospitals could enroll both persistent and newly identified high-risk individuals for outpatient interventions (ie, follow-up telephone call) in the immediate post-discharge period, an interval characterized by heightened vulnerability for adverse events,²⁸ based on patients’ illness severity and stability at discharge. This approach can be implemented by hospitals by building these risk-prediction models directly into the EHR, or by extracting EHR data in near real time as our group has done successfully for heart failure.⁷

Another key implication of our study is that, for pneumonia, a disease-specific modeling approach has better predictive ability than using a multi-condition model. Compared to multi-condition models, the first-day and full-stay pneumonia-specific EHR models correctly reclassified an additional 6% and 9% of patients, respectively. Thus, hospitals interested in identifying the highest risk patients with pneumonia for targeted interventions should do so using the disease-specific models, if the costs and resources of doing so are within reach of the healthcare system.

An additional novel finding of our study is the added value of an updated PSI for predicting adverse events. Studies of pneumonia severity of illness scores have calculated the PSI and CURB-65 scores using data present only on admission.^16,24 While our study also confirms that the PSI calculated upon admission is a significant predictor of readmission,^23,29 this study extends this work by showing that an updated PSI score calculated at the time of discharge is an even stronger predictor for readmission, and its inclusion in the model significantly improves risk stratification and prognostication.

Our study was noteworthy for several strengths. First, we used data from a common EHR system, thus potentially allowing for the implementation of the pneumonia-specific models in real time across a number of hospitals. The use of routinely collected data for risk-prediction modeling makes this approach scalable and sustainable, because it obviates the need for burdensome data collection and entry. Second, to our knowledge, this is the first study to measure the additive influence of illness severity and stability at discharge on the readmission risk among patients hospitalized with pneumonia. Third, our study population was derived from 6 hospitals diverse in payer status, age, race/ethnicity, and socioeconomic status. Fourth, our models are less likely to be overfit to the idiosyncrasies of our data given that several predictors included in our final pneumonia-specific models have been associated with readmission in this population, including marital status,^13,30 income,^11,31 prior hospitalizations,^11,13 thrombocytosis,^32-34 and vital sign instabilities on discharge.¹⁷ Lastly, the discrimination of the CMS pneumonia model in our cohort (C statistic, 0.64) closely matched the discrimination observed in 4 independent cohorts (C statistic, 0.63), suggesting adequate generalizability of our study setting and population.^10,12

Our results should be interpreted in the context of several limitations. First, generalizability to other regions beyond north Texas is unknown. Second, although we included a diverse cohort of safety net, community, teaching, and nonteaching hospitals, the pneumonia-specific models were not externally validated in a separate cohort, which may lead to more optimistic estimates of model performance. Third, PSI and CURB-65 scores were modified to use diagnostic codes for altered mental status and pleural effusion, and omitted nursing home residence. Thus, the independent associations for the PSI and CURB-65 scores and their predictive ability are likely attenuated. Fourth, we were unable to include data on medications (antibiotics and steroid use) and outpatient visits, which may influence readmission risk.^2,9,13,35-40 Fifth, we included only the first pneumonia hospitalization per patient in this study. Had we included multiple hospitalizations per patient, we anticipate better model performance for the 2 pneumonia-specific EHR models since prior hospitalization was a robust predictor of readmission.

In conclusion, the full-stay pneumonia-specific EHR readmission risk-prediction model outperformed the first-day pneumonia-specific model, multi-condition EHR models, and the CMS pneumonia model. This suggests that: measures of clinical severity and stability at the time of discharge are important predictors for identifying patients at highest risk for readmission; and that EHR data routinely collected for clinical practice can be used to accurately predict risk of readmission among patients hospitalized for pneumonia.

Acknowledgments

The authors would like to acknowledge Ruben Amarasingham, MD, MBA, president and chief executive officer of Parkland Center for Clinical Innovation, and Ferdinand Velasco, MD, chief health information officer at Texas Health Resources for their assistance in assembling the 6-hospital cohort used in this study.

Disclosures

This work was supported by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24 HS022418-01); the Commonwealth Foundation (#20100323); the UT Southwestern KL2 Scholars Program supported by the National Institutes of Health (KL2 TR001103 to ANM and OKN); and the National Center for Advancing Translational Sciences at the National Institute of Health (U54 RFA-TR-12-006 to E.A.H.). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no financial conflicts of interest to disclose

Pneumonia is a leading cause of hospitalizations in the U.S., accounting for more than 1.1 million discharges annually.¹ Pneumonia is frequently complicated by hospital readmission, which is costly and potentially avoidable.^2,3 Due to financial penalties imposed on hospitals for higher than expected 30-day readmission rates, there is increasing attention to implementing interventions to reduce readmissions in this population.^4,5 However, because these programs are resource-intensive, interventions are thought to be most cost-effective if they are targeted to high-risk individuals who are most likely to benefit.^6-8

Current pneumonia-specific readmission risk-prediction models that could enable identification of high-risk patients suffer from poor predictive ability, greatly limiting their use, and most were validated among older adults or by using data from single academic medical centers, limiting their generalizability.^9-14 A potential reason for poor predictive accuracy is the omission of known robust clinical predictors of pneumonia-related outcomes, including pneumonia severity of illness and stability on discharge.^15-17 Approaches using electronic health record (EHR) data, which include this clinically granular data, could enable hospitals to more accurately and pragmatically identify high-risk patients during the index hospitalization and enable interventions to be initiated prior to discharge.

An alternative strategy to identifying high-risk patients for readmission is to use a multi-condition risk-prediction model. Developing and implementing models for every condition may be time-consuming and costly. We have derived and validated 2 multi-condition risk-prediction models using EHR data—1 using data from the first day of hospital admission (‘first-day’ model), and the second incorporating data from the entire hospitalization (‘full-stay’ model) to reflect in-hospital complications and clinical stability at discharge.^18,19 However, it is unknown if a multi-condition model for pneumonia would perform as well as a disease-specific model.

This study aimed to develop 2 EHR-based pneumonia-specific readmission risk-prediction models using data routinely collected in clinical practice—a ‘first-day’ and a ‘full-stay’ model—and compare the performance of each model to: 1) one another; 2) the corresponding multi-condition EHR model; and 3) to other potentially useful models in predicting pneumonia readmissions (the Centers for Medicare and Medicaid Services [CMS] pneumonia model, and 2 commonly used pneumonia severity of illness scores validated for predicting mortality). We hypothesized that the pneumonia-specific EHR models would outperform other models; and the full-stay pneumonia-specific model would outperform the first-day pneumonia-specific model.

Study Design, Population, and Data Sources

We conducted an observational study using EHR data collected from 6 hospitals (including safety net, community, teaching, and nonteaching hospitals) in north Texas between November 2009 and October 2010, All hospitals used the Epic EHR (Epic Systems Corporation, Verona, WI). Details of this cohort have been published.^18,19

We included consecutive hospitalizations among adults 18 years and older discharged from any medicine service with principal discharge diagnoses of pneumonia (ICD-9-CM codes 480-483, 485, 486-487), sepsis (ICD-9-CM codes 038, 995.91, 995.92, 785.52), or respiratory failure (ICD-9-CM codes 518.81, 518.82, 518.84, 799.1) when the latter 2 were also accompanied by a secondary diagnosis of pneumonia.²⁰ For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization or within 30 days of discharge, were transferred to another acute care facility, or left against medical advice.

Outcomes

The primary outcome was all-cause 30-day readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100-mile radius of Dallas, ascertained from an all-payer regional hospitalization database.

Predictor Variables for the Pneumonia-Specific Readmission Models

The selection of candidate predictors was informed by our validated multi-condition risk-prediction models using EHR data available within 24 hours of admission (‘first-day’ multi-condition EHR model) or during the entire hospitalization (‘full-stay’ multi-condition EHR model).^18,19 For the pneumonia-specific models, we included all variables in our published multi-condition models as candidate predictors, including sociodemographics, prior utilization, Charlson Comorbidity Index, select laboratory and vital sign abnormalities, length of stay, hospital complications (eg, venous thromboembolism), vital sign instabilities, and disposition status (see Supplemental Table 1 for complete list of variables). We also assessed additional variables specific to pneumonia for inclusion that were: (1) available in the EHR of all participating hospitals; (2) routinely collected or available at the time of admission or discharge; and (3) plausible predictors of adverse outcomes based on literature and clinical expertise. These included select comorbidities (eg, psychiatric conditions, chronic lung disease, history of pneumonia),^10,11,21,22 the pneumonia severity index (PSI),^16,23,24 intensive care unit stay, and receipt of invasive or noninvasive ventilation. We used a modified PSI score because certain data elements were missing. The modified PSI (henceforth referred to as PSI) did not include nursing home residence and included diagnostic codes as proxies for the presence of pleural effusion (ICD-9-CM codes 510, 511.1, and 511.9) and altered mental status (ICD-9-CM codes 780.0X, 780.97, 293.0, 293.1, and 348.3X).

Statistical Analysis

Model Derivation. Candidate predictor variables were classified as available in the EHR within 24 hours of admission and/or at the time of discharge. For example, socioeconomic factors could be ascertained within the first day of hospitalization, whereas length of stay would not be available until the day of discharge. Predictors with missing values were assumed to be normal (less than 1% missing for each variable). Univariate relationships between readmission and each candidate predictor were assessed in the overall cohort using a pre-specified significance threshold of P ≤ 0.10. Significant variables were entered in the respective first-day and full-stay pneumonia-specific multivariable logistic regression models using stepwise-backward selection with a pre-specified significance threshold of P ≤ 0.05. In sensitivity analyses, we alternately derived our models using stepwise-forward selection, as well as stepwise-backward selection minimizing the Bayesian information criterion and Akaike information criterion separately. These alternate modeling strategies yielded identical predictors to our final models.

Model Validation. Model validation was performed using 5-fold cross-validation, with the overall cohort randomly divided into 5 equal-size subsets.²⁵ For each cycle, 4 subsets were used for training to estimate model coefficients, and the fifth subset was used for validation. This cycle was repeated 5 times with each randomly-divided subset used once as the validation set. We repeated this entire process 50 times and averaged the C statistic estimates to derive an optimism-corrected C statistic. Model calibration was assessed qualitatively by comparing predicted to observed probabilities of readmission by quintiles of predicted risk, and with the Hosmer-Lemeshow goodness-of-fit test.

Comparison to Other Models. The main comparisons of the first-day and full-stay pneumonia-specific EHR model performance were to each other and the corresponding multi-condition EHR model.^18,19 The multi-condition EHR models were separately derived and validated within the larger parent cohort from which this study cohort was derived, and outperformed the CMS all-cause model, the HOSPITAL model, and the LACE index.¹⁹ To further triangulate our findings, given the lack of other rigorously validated pneumonia-specific risk-prediction models for readmission,¹⁴ we compared the pneumonia-specific EHR models to the CMS pneumonia model derived from administrative claims data,¹⁰ and 2 commonly used risk-prediction scores for short-term mortality among patients with community-acquired pneumonia, the PSI and CURB-65 scores.¹⁶ Although derived and validated using patient-level data, the CMS model was developed to benchmark hospitals according to hospital-level readmission rates.¹⁰ The CURB-65 score in this study was also modified to include the same altered mental status diagnostic codes according to the modified PSI as a proxy for “confusion.” Both the PSI and CURB-65 scores were calculated using the most abnormal values within the first 24 hours of admission. The ‘updated’ PSI and the ‘updated’ CURB-65 were calculated using the most abnormal values within 24 hours prior to discharge, or the last known observation prior to discharge if no results were recorded within this time period. A complete list of variables for each of the comparison models are shown in Supplemental Table 1.

We assessed model performance by calculating the C statistic, integrated discrimination index, and net reclassification index (NRI) compared to our pneumonia-specific models. The integrated discrimination index is the difference in the mean predicted probability of readmission between patients who were and were not actually readmitted between 2 models, where more positive values suggest improvement in model performance compared to a reference model.²⁶ The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.²⁷ Here, we calculated a category-based NRI to evaluate the performance of pneumonia-specific models in correctly classifying individuals with and without readmissions into the 2 highest readmission risk quintiles vs the lowest 3 risk quintiles compared to other models.²⁷ This pre-specified cutoff is relevant for hospitals interested in identifying the highest risk individuals for targeted intervention.⁷ Finally, we assessed calibration of comparator models in our cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, Texas). This study was approved by the University of Texas Southwestern Medical Center Institutional Review Board.

Of 1463 index hospitalizations (Supplemental Figure 1), the 30-day all-cause readmission rate was 13.6%. Individuals with a 30-day readmission had markedly different sociodemographic and clinical characteristics compared to those not readmitted (Table 1; see Supplemental Table 2 for additional clinical characteristics).

Derivation, Validation, and Performance of the Pneumonia-Specific Readmission Risk-Prediction Models

The final first-day pneumonia-specific EHR model included 7 variables, including sociodemographic characteristics; prior hospitalizations; thrombocytosis, and PSI (Table 2). The first-day pneumonia-specific model had adequate discrimination (C statistic, 0.695; optimism-corrected C statistic 0.675, 95% confidence interval [CI], 0.667-0.685; Table 3). It also effectively stratified individuals across a broad range of risk (average predicted decile of risk ranged from 4% to 33%; Table 3) and was well calibrated (Supplemental Table 3).

The final full-stay pneumonia-specific EHR readmission model included 8 predictors, including 3 variables from the first-day model (median income, thrombocytosis, and prior hospitalizations; Table 2). The full-stay pneumonia-specific EHR model also included vital sign instabilities on discharge, updated PSI, and disposition status (ie, being discharged with home health or to a post-acute care facility was associated with greater odds of readmission, and hospice with lower odds). The full-stay pneumonia-specific EHR model had good discrimination (C statistic, 0.731; optimism-corrected C statistic, 0.714; 95% CI, 0.706-0.720), and stratified individuals across a broad range of risk (average predicted decile of risk ranged from 3% to 37%; Table 3), and was also well calibrated (Supplemental Table 3).

First-Day Pneumonia-Specific EHR Model vs First-Day Multi-Condition EHR Model

The first-day pneumonia-specific EHR model outperformed the first-day multi-condition EHR model with better discrimination (P = 0.029) and more correctly classified individuals in the top 2 highest risk quintiles vs the bottom 3 risk quintiles (Table 3, Supplemental Table 4, and Supplemental Figure 2A). With respect to calibration, the first-day multi-condition EHR model overestimated risk among the highest quintile risk group compared to the first-day pneumonia-specific EHR model (Figure 1A, 1B).

Full-Stay Pneumonia-Specific EHR Model vs Other Models

The full-stay pneumonia-specific EHR model comparatively outperformed the corresponding full-stay multi-condition EHR model, as well as the first-day pneumonia-specific EHR model, the CMS pneumonia model, the updated PSI, and the updated CURB-65 (Table 3, Supplemental Table 5, Supplemental Table 6, and Supplemental Figures 2B and 2C). Compared to the full-stay multi-condition and first-day pneumonia-specific EHR models, the full-stay pneumonia-specific EHR model had better discrimination, better reclassification (NRI, 0.09 and 0.08, respectively), and was able to stratify individuals across a broader range of readmission risk (Table 3). It also had better calibration in the highest quintile risk group compared to the full-stay multi-condition EHR model (Figure 1C and 1D).

Updated vs First-Day Modified PSI and CURB-65 Scores

The updated PSI was more strongly predictive of readmission than the PSI calculated on the day of admission (Wald test, 9.83; P = 0.002). Each 10-point increase in the updated PSI was associated with a 22% increased odds of readmission vs an 11% increase for the PSI calculated upon admission (Table 2). The improved predictive ability of the updated PSI and CURB-65 scores was also reflected in the superior discrimination and calibration vs the respective first-day pneumonia severity of illness scores (Table 3).

Using routinely available EHR data from 6 diverse hospitals, we developed 2 pneumonia-specific readmission risk-prediction models that aimed to allow hospitals to identify patients hospitalized with pneumonia at high risk for readmission. Overall, we found that a pneumonia-specific model using EHR data from the entire hospitalization outperformed all other models—including the first-day pneumonia-specific model using data present only on admission, our own multi-condition EHR models, and the CMS pneumonia model based on administrative claims data—in all aspects of model performance (discrimination, calibration, and reclassification). We found that socioeconomic status, prior hospitalizations, thrombocytosis, and measures of clinical severity and stability were important predictors of 30-day all-cause readmissions among patients hospitalized with pneumonia. Additionally, an updated discharge PSI score was a stronger independent predictor of readmissions compared to the PSI score calculated upon admission; and inclusion of the updated PSI in our full-stay pneumonia model led to improved prediction of 30-day readmissions.

The marked improvement in performance of the full-stay pneumonia-specific EHR model compared to the first-day pneumonia-specific model suggests that clinical stability and trajectory during hospitalization (as modeled through disposition status, updated PSI, and vital sign instabilities at discharge) are important predictors of 30-day readmission among patients hospitalized for pneumonia, which was not the case for our EHR-based multi-condition models.¹⁹ With the inclusion of these measures, the full-stay pneumonia-specific model correctly reclassified an additional 8% of patients according to their true risk compared to the first-day pneumonia-specific model. One implication of these findings is that hospitals interested in targeting their highest risk individuals with pneumonia for transitional care interventions could do so using the first-day pneumonia-specific EHR model and could refine their targeted strategy at the time of discharge by using the full-stay pneumonia model. This staged risk-prediction strategy would enable hospitals to initiate transitional care interventions for high-risk individuals in the inpatient setting (ie, patient education).⁷ Then, hospitals could enroll both persistent and newly identified high-risk individuals for outpatient interventions (ie, follow-up telephone call) in the immediate post-discharge period, an interval characterized by heightened vulnerability for adverse events,²⁸ based on patients’ illness severity and stability at discharge. This approach can be implemented by hospitals by building these risk-prediction models directly into the EHR, or by extracting EHR data in near real time as our group has done successfully for heart failure.⁷

Another key implication of our study is that, for pneumonia, a disease-specific modeling approach has better predictive ability than using a multi-condition model. Compared to multi-condition models, the first-day and full-stay pneumonia-specific EHR models correctly reclassified an additional 6% and 9% of patients, respectively. Thus, hospitals interested in identifying the highest risk patients with pneumonia for targeted interventions should do so using the disease-specific models, if the costs and resources of doing so are within reach of the healthcare system.

An additional novel finding of our study is the added value of an updated PSI for predicting adverse events. Studies of pneumonia severity of illness scores have calculated the PSI and CURB-65 scores using data present only on admission.^16,24 While our study also confirms that the PSI calculated upon admission is a significant predictor of readmission,^23,29 this study extends this work by showing that an updated PSI score calculated at the time of discharge is an even stronger predictor for readmission, and its inclusion in the model significantly improves risk stratification and prognostication.

Our study was noteworthy for several strengths. First, we used data from a common EHR system, thus potentially allowing for the implementation of the pneumonia-specific models in real time across a number of hospitals. The use of routinely collected data for risk-prediction modeling makes this approach scalable and sustainable, because it obviates the need for burdensome data collection and entry. Second, to our knowledge, this is the first study to measure the additive influence of illness severity and stability at discharge on the readmission risk among patients hospitalized with pneumonia. Third, our study population was derived from 6 hospitals diverse in payer status, age, race/ethnicity, and socioeconomic status. Fourth, our models are less likely to be overfit to the idiosyncrasies of our data given that several predictors included in our final pneumonia-specific models have been associated with readmission in this population, including marital status,^13,30 income,^11,31 prior hospitalizations,^11,13 thrombocytosis,^32-34 and vital sign instabilities on discharge.¹⁷ Lastly, the discrimination of the CMS pneumonia model in our cohort (C statistic, 0.64) closely matched the discrimination observed in 4 independent cohorts (C statistic, 0.63), suggesting adequate generalizability of our study setting and population.^10,12

Our results should be interpreted in the context of several limitations. First, generalizability to other regions beyond north Texas is unknown. Second, although we included a diverse cohort of safety net, community, teaching, and nonteaching hospitals, the pneumonia-specific models were not externally validated in a separate cohort, which may lead to more optimistic estimates of model performance. Third, PSI and CURB-65 scores were modified to use diagnostic codes for altered mental status and pleural effusion, and omitted nursing home residence. Thus, the independent associations for the PSI and CURB-65 scores and their predictive ability are likely attenuated. Fourth, we were unable to include data on medications (antibiotics and steroid use) and outpatient visits, which may influence readmission risk.^2,9,13,35-40 Fifth, we included only the first pneumonia hospitalization per patient in this study. Had we included multiple hospitalizations per patient, we anticipate better model performance for the 2 pneumonia-specific EHR models since prior hospitalization was a robust predictor of readmission.

In conclusion, the full-stay pneumonia-specific EHR readmission risk-prediction model outperformed the first-day pneumonia-specific model, multi-condition EHR models, and the CMS pneumonia model. This suggests that: measures of clinical severity and stability at the time of discharge are important predictors for identifying patients at highest risk for readmission; and that EHR data routinely collected for clinical practice can be used to accurately predict risk of readmission among patients hospitalized for pneumonia.

Acknowledgments

The authors would like to acknowledge Ruben Amarasingham, MD, MBA, president and chief executive officer of Parkland Center for Clinical Innovation, and Ferdinand Velasco, MD, chief health information officer at Texas Health Resources for their assistance in assembling the 6-hospital cohort used in this study.

Disclosures

This work was supported by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24 HS022418-01); the Commonwealth Foundation (#20100323); the UT Southwestern KL2 Scholars Program supported by the National Institutes of Health (KL2 TR001103 to ANM and OKN); and the National Center for Advancing Translational Sciences at the National Institute of Health (U54 RFA-TR-12-006 to E.A.H.). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no financial conflicts of interest to disclose

Patients typically show signs and symptoms of deterioration hours to days prior to cardiorespiratory arrest.^1,2 The rate of inhospital cardiorespiratory arrest (CRA) requiring cardiopulmonary resuscitation is estimated to be 0.174 per bed per year in the United States.³ After CRA, survival to discharge is estimated to be as low as 18%.^3,4 Efforts to predict and prevent arrest could prove beneficial.^1,2

Rapid response systems (RRS) have been proposed as a means of identifying clinical deterioration and facilitating a timely response. These systems were designed to bring clinicians with critical care expertise to the bedside to prevent unnecessary deaths. They typically include an afferent limb (detects deteriorating patients), an efferent limb (responds to calls and acts to avoid further deterioration), and administrative and data analysis limbs.^5,6 Automatic provision of recommendations and computer-based systems are desirable components of the afferent limb of the detection system.⁶ Both are independent predictors of improved clinical practices for clinical decision support systems.⁷ However, the existing early warning scores (EWS) may not be ready for automation due to low positive predictive values (PPV) and sensitivities.⁸

It is possible that the low discriminatory accuracy of the published EWS may be secondary to the use of aggregate patient populations for derivation of scores. We hypothesized that these EWS perform differently in medical and in surgical subpopulations. Also, the EWS need to be tested in a time-dependent manner to serve as a realistic clinical support tool for hospitalized patients.

STUDY AIM

The aim of this study was to evaluate the differential performance of widely used EWS in medical vs surgical patients.

Site

The study was conducted in an academic center with 2 hospitals in Southeastern Minnesota totaling approximately 1500 general care nonintensive care unit (ICU) beds. The Mayo Clinic Institutional Review Board approved the research proposal.

Subjects

Our retrospective cohort was comprised of all adult inpatients discharged from 2 academic hospitals between January 1, 2011 and December 31, 2011 who spent any time in a general care (non-ICU) unit. We excluded patients younger than 18 years, psychiatric or rehabilitation inpatients, those without research authorization, and patients admitted for research purposes.

Study patients were divided into medical and surgical cohorts. Hospitalizations were considered surgical if patients had surgery at any time during their hospital stay according to billing data. A trigger was an instance in which a patient met the conditions of a specific rule (score/vital sign exceeded the published/defined threshold).

A resuscitation call was defined as a call for cardiopulmonary resuscitation when a patient has a CRA.

An event was an occurrence of 1 of the following in a general care setting: unplanned transfer to the ICU, resuscitation call, or RRS activation.

The RRS activation criteria consisted of an “acute and persistent change” in any 1 or more of the following: oxygen saturations less than 90%, heart rate less than 40 or greater than 130 beats/minute, systolic blood pressure less than 90 mm Hg, or respiratory rate less than 10 or greater than 28 breaths/minute. The RRS activation requires health provider action; they are not electronically generated. Nurses and physicians may also activate the RRS if they are concerned about a patient, even if calling criteria are not met. This is in contrast to the EWS analyzed, which are aggregate composites of multiple parameters. However, whether or not a derangement in vital signs is considered an “acute and persistent change” still involves clinical judgment. Any movement from a general care bed to an ICU bed, or from a general care bed to a procedure area, and from there to an ICU, was considered unplanned. Transfers to the ICU directly from the emergency department or operating room (OR) were not considered as an unplanned transfer and were not included in the analyses.

Coverage time was the period observed for events after a rule was triggered. In this analysis, a coverage time of 24 hours was considered, with a 1-hour look-back. A trigger was counted as a true positive if an event occurred during the following 24 hours. The 1-hour look-back was included to take into account the nursing clinical process of prioritizing a call to the RRS followed by documentation of the altered vital signs that prompted the call.

An episode was the continuous time on the general care floor within a hospitalization, excluding times when a patient was in the OR or ICU. For example, if a patient was admitted to a general bed on a surgery floor, subsequently went to the OR, and then returned to the surgery floor, the 2 episodes were considered separate: the time on the floor before surgery, and the time on the floor after surgery.

Assessment of implementation of RRS in our hospitals showed a significant drop in the failure-to-rescue rate (issues considered related to delay or failure to identify or intervene appropriately when a patient was deteriorating, as identified through mortality review) and a decrease in non-ICU mortality.^9,10 This suggests that our current process captures many of the relevant episodes of acute deterioration when a rapid response team is needed and supports using RRS activation as outcomes.

Data Sources

We developed a time-stamped longitudinal database of patient data from the electronic health record, including vital signs, laboratory test results, demographics (age, sex), administrative data (including length of stay), comorbidities, resuscitation code status, location in hospital, and at the minute level throughout each patient’s hospital stay. Physiologically impossible values (eg, blood pressures of 1200 mm Hg) were considered entered in error and eliminated from the database. Time spent in the OR or ICU was excluded because RRS activation would not be applied in these already highly monitored areas. SAS Statistical software (SAS Institute Inc. Cary, North Carolina) was used for database creation.

We applied the current RRS calling criteria in our institution and calculated the Kirkland score,¹¹ along with some of the most widely used early warning scores:¹² Modified Early Warning System (MEWS),¹³ Standardized Early Warning Scoring System (SEWS),¹⁴ Global Modified Early Warning Score (GMEWS),¹⁵ Worthing physiologic scoring system,¹⁶ National Early Warning Score (NEWS),¹⁷ and VitaPAC Early Warning Score (ViEWS).¹⁸ Published thresholds for these scores were used to create rule triggers in the data. Once a trigger was created to calculate the number of false positives and true positives, all subsequent triggers were ignored until the end of the episode or until 24 hours elapsed. We calculated triggers in a rolling fashion throughout the episodes of care. The EWS score was updated every time a new parameter was entered into the analytical electronic health record, and the most recent value for each was used to calculate the score. SAS statistical software was used for calculation of scores and identification of outcomes.

For our analysis, events were treated as dependent variables, and triggers were independent variables. We calculated the score for each EWS to the minute level throughout our retrospective database. If the score for a specific EWS was higher than the published/recommended threshold for that EWS, an alert was considered to have been issued, and the patient was followed for 24 hours. If the patient had an event in the subsequent 24 hours, or 1 hour before (1-hour look-back), the alert was considered a true positive; if not, a false positive. Events that were not preceded by an alert were false negatives, and 24-hour intervals without either an alert or an event were considered true negatives. This simulation exercise was performed for each EWS in both subcohorts (medical and surgical). Clusters of RRS calls followed by transfers to the ICU within 3 hours were considered as a single adverse event (RRS calls, as it was the first event to occur) to avoid double counting. We have described how well this simulation methodology,⁸ correlates with results from prospective studies.¹⁹

Statistical Analysis

To calculate whether results were statistically significant for subgroups, a jackknife method of calculating variance²⁰ was used. The jackknife method calculates variance by repeating the calculations of the statistic leaving out 1 sample at a time. In our case, we repeated the calculation of sensitivity and PPV leaving out 1 patient at a time. Once the simulation method had been run and the false/true positives/negatives had been assigned, calculation of each metric (PPV and sensitivity) was repeated for n subsamples, each leaving out 1 patient. The variance was calculated and 2 Student t tests were performed for each EWS: 1 for PPV and another for sensitivity. SAS statistical software v 9.3 was used for the simulation analysis; R statistical software v 3.0.2 (The R Foundation, Vienna, Austria) was used for the calculation of the statistical significance of results. A univariable analysis was also performed to assess the sensitivity and PPVs for the published thresholds of the most common variables in each EWS: respiratory rate, systolic blood pressure, heart rate, temperature, and mental status as measured by the modified Richmond Agitation Sedation Score.²¹

The initial cohort included 60,020 hospitalizations, of which the following were excluded: 2751 because of a lack of appropriate research authorization; 6433 because the patients were younger than 18 years; 2129 as psychiatric admissions; 284 as rehabilitation admissions; 872 as research purposes-only admissions; and 1185 because the patient was never in a general care bed (eg, they were either admitted directly to the ICU, or they were admitted for an outpatient surgical procedure and spent time in the postanesthesia care unit).

Table 1 summarizes patient and trigger characteristics, overall and by subgroup. The final cohort included 75,240 total episodes in 46,366 hospitalizations, from 34,898 unique patients, of which 48.7% were male. There were 23,831 medical and 22,535 surgical hospitalizations. Median length of episode was 2 days both for medical and surgical patients. Median length of stay was 3 days, both for medical and for surgical patients.

We hypothesized that EWS may perform differently when applied to medical rather than surgical patients. Studies had not analyzed this in a time-dependent manner,^24-26 which limited the applicability of the results.⁸

All analyzed scores performed better in medical patients than in surgical patients (Figure). This could reflect a behavioral difference by the teams on surgical and medical floors in the decision to activate the RRS, or a bias of the clinicians who designed the scores (mostly nonsurgeons). The difference could also mean that physiological deteriorations are intrinsically different in patients who have undergone anesthesia and surgery. For example, in surgical patients, a bleeding episode is more likely to be the cause of their physiological deterioration, or the lingering effects of anesthesia could mask underlying deterioration. Such patients would benefit from scores where variables such as heart rate, blood pressure, or hemoglobin had more influence.

When comparing the different scores, it was much easier for a patient to meet the alerting score with the Worthing score than with GMEWS. In the Worthing score, a respiratory rate greater than 22 breaths per minute, or a systolic blood pressure less than 100 mm Hg, already meet alerting criteria. Similar vital signs result in 0 and 1 points (respectively) in GMEWS, far from its alerting score of 5. This reflects the intrinsic tradeoff of EWS: as the threshold for considering a patient “at risk” drops, not only does the number of true positives (and the sensitivity) increase, but also the number of false positives, thus lowering the PPV.

However, none of the scores analyzed were considered to perform well based on their PPV and sensitivity, particularly in the surgical subpopulation. Focusing on another metric, the area under the receiver operator curve can give misleadingly optimistic results.^24,27 However, the extremely low prevalence of acute physiological deterioration can produce low PPVs even when specificity seems acceptable, which is why it is important to evaluate PPV directly.²⁸

To use EWS effectively to activate RRS, they need to be combined with clinical judgment to avoid high levels of false alerts, particularly in surgical patients. It has been reported that RRS is activated only 30% of the time a patient meets RRS calling criteria.²⁹ While there may be cultural characteristics inhibiting the decision to call,³⁰ our study hints at another explanation: if RRS was activated every time a patient met calling criteria based on the scores analyzed, the number of RRS calls would be very high and difficult to manage. So health providers may be doing the right thing when “filtering” RRS calls and not applying the criteria strictly, but in conjunction with clinical judgment.

A limitation of any study like this is how to define “acute physiological deterioration.” We defined an event as recognized episodes of acute physiological deterioration that are signaled by escalations of care (eg, RRS, resuscitation calls, or transfers to an ICU) or unexpected death. By definition, our calculated PPV is affected by clinicians’ recognition of clinical deteriorations. This definition, common in the literature, has the limitation of potentially underestimating EWS’ performance by missing some events that are resolved by the primary care team without an escalation of care. However, we believe our interpretation is not unreasonable since the purpose of EWS is to trigger escalations of care in a timely fashion. Prospective studies could define an event in a way that is less affected by the clinicians’ judgment.

Regarding patient demographics, age was similar between the 2 groups (average, 58.2 years for medical vs 58.9 years for surgical), and there was only a small difference in gender ratios (45.1% male in the medical vs 51.4% in the surgical group). These differences are unlikely to have affected the results significantly, but unknown differences in demographics or other patient characteristics between groups may account for differences in score performance between surgical and medical patients.

Several of the EWS analyzed had overlapping trigger criteria with our own RRS activation criteria (although as single-parameter triggers and not as aggregate). To test how these potential biases could affect our results, we performed a post hoc sensitivity analysis eliminating calls to the RRS as an outcome (so using the alternative outcome of unexpected transfers to the ICU and resuscitation calls). The results are similar to those of our main analysis, with all analyzed scores having lower sensitivity and PPV in surgical hospitalizations when compared to medical hospitalizations.

Our study suggests that, to optimize detection of physiological deterioration events, EWS should try to take into account different patient types, with the most basic distinction being surgical vs medical. This tailoring will make EWS more complex, and less suited for paper-based calculation, but new electronic health records are increasingly able to incorporate decision support, and some EWS have been developed for electronic calculation only. Of particular interest in this regard is the score developed by Escobar et al,³¹ which groups patients into categories according to the reason for admission, and calculates a different subscore based on that category. While the score by Escobar et al. does not split patients based on medical or surgical status, a more general interpretation of our results suggests that a score may be more accurate if it classifies patients into subgroups with different subscores. This seems to be confirmed by the fact that the score by Escobar et al performs better than MEWS.²⁸ Unfortunately, the paper describing it does not provide enough detail to use it in our database.

A recent systematic review showed increasing evidence that RRS may be effective in reducing CRAs occurring in a non-ICU setting and, more important, overall inhospital mortality.³² While differing implementation strategies (eg, different length of the educational effort, changes in the frequency of vital signs monitoring) can impact the success of such an initiative, it has been speculated that the afferent limb (which often includes an EWS) might be the most critical part of the system.³³ Our results show that the most widely used EWS perform significantly worse on surgical patients, and suggest that a way to improve the accuracy of EWS would be to tailor the risk calculation to different patient subgroups (eg, medical and surgical patients). Plausible next steps would be to demonstrate that tailoring risk calculation to medical and surgical patients separately can improve risk predictions and accuracy of EWS.

Disclosure

The authors report no financial conflicts of interest.

Patients typically show signs and symptoms of deterioration hours to days prior to cardiorespiratory arrest.^1,2 The rate of inhospital cardiorespiratory arrest (CRA) requiring cardiopulmonary resuscitation is estimated to be 0.174 per bed per year in the United States.³ After CRA, survival to discharge is estimated to be as low as 18%.^3,4 Efforts to predict and prevent arrest could prove beneficial.^1,2

Rapid response systems (RRS) have been proposed as a means of identifying clinical deterioration and facilitating a timely response. These systems were designed to bring clinicians with critical care expertise to the bedside to prevent unnecessary deaths. They typically include an afferent limb (detects deteriorating patients), an efferent limb (responds to calls and acts to avoid further deterioration), and administrative and data analysis limbs.^5,6 Automatic provision of recommendations and computer-based systems are desirable components of the afferent limb of the detection system.⁶ Both are independent predictors of improved clinical practices for clinical decision support systems.⁷ However, the existing early warning scores (EWS) may not be ready for automation due to low positive predictive values (PPV) and sensitivities.⁸

It is possible that the low discriminatory accuracy of the published EWS may be secondary to the use of aggregate patient populations for derivation of scores. We hypothesized that these EWS perform differently in medical and in surgical subpopulations. Also, the EWS need to be tested in a time-dependent manner to serve as a realistic clinical support tool for hospitalized patients.

STUDY AIM

The aim of this study was to evaluate the differential performance of widely used EWS in medical vs surgical patients.

Site

The study was conducted in an academic center with 2 hospitals in Southeastern Minnesota totaling approximately 1500 general care nonintensive care unit (ICU) beds. The Mayo Clinic Institutional Review Board approved the research proposal.

Subjects

Our retrospective cohort was comprised of all adult inpatients discharged from 2 academic hospitals between January 1, 2011 and December 31, 2011 who spent any time in a general care (non-ICU) unit. We excluded patients younger than 18 years, psychiatric or rehabilitation inpatients, those without research authorization, and patients admitted for research purposes.

Study patients were divided into medical and surgical cohorts. Hospitalizations were considered surgical if patients had surgery at any time during their hospital stay according to billing data. A trigger was an instance in which a patient met the conditions of a specific rule (score/vital sign exceeded the published/defined threshold).

A resuscitation call was defined as a call for cardiopulmonary resuscitation when a patient has a CRA.

An event was an occurrence of 1 of the following in a general care setting: unplanned transfer to the ICU, resuscitation call, or RRS activation.

The RRS activation criteria consisted of an “acute and persistent change” in any 1 or more of the following: oxygen saturations less than 90%, heart rate less than 40 or greater than 130 beats/minute, systolic blood pressure less than 90 mm Hg, or respiratory rate less than 10 or greater than 28 breaths/minute. The RRS activation requires health provider action; they are not electronically generated. Nurses and physicians may also activate the RRS if they are concerned about a patient, even if calling criteria are not met. This is in contrast to the EWS analyzed, which are aggregate composites of multiple parameters. However, whether or not a derangement in vital signs is considered an “acute and persistent change” still involves clinical judgment. Any movement from a general care bed to an ICU bed, or from a general care bed to a procedure area, and from there to an ICU, was considered unplanned. Transfers to the ICU directly from the emergency department or operating room (OR) were not considered as an unplanned transfer and were not included in the analyses.

Coverage time was the period observed for events after a rule was triggered. In this analysis, a coverage time of 24 hours was considered, with a 1-hour look-back. A trigger was counted as a true positive if an event occurred during the following 24 hours. The 1-hour look-back was included to take into account the nursing clinical process of prioritizing a call to the RRS followed by documentation of the altered vital signs that prompted the call.

An episode was the continuous time on the general care floor within a hospitalization, excluding times when a patient was in the OR or ICU. For example, if a patient was admitted to a general bed on a surgery floor, subsequently went to the OR, and then returned to the surgery floor, the 2 episodes were considered separate: the time on the floor before surgery, and the time on the floor after surgery.

Assessment of implementation of RRS in our hospitals showed a significant drop in the failure-to-rescue rate (issues considered related to delay or failure to identify or intervene appropriately when a patient was deteriorating, as identified through mortality review) and a decrease in non-ICU mortality.^9,10 This suggests that our current process captures many of the relevant episodes of acute deterioration when a rapid response team is needed and supports using RRS activation as outcomes.

Data Sources

We developed a time-stamped longitudinal database of patient data from the electronic health record, including vital signs, laboratory test results, demographics (age, sex), administrative data (including length of stay), comorbidities, resuscitation code status, location in hospital, and at the minute level throughout each patient’s hospital stay. Physiologically impossible values (eg, blood pressures of 1200 mm Hg) were considered entered in error and eliminated from the database. Time spent in the OR or ICU was excluded because RRS activation would not be applied in these already highly monitored areas. SAS Statistical software (SAS Institute Inc. Cary, North Carolina) was used for database creation.

We applied the current RRS calling criteria in our institution and calculated the Kirkland score,¹¹ along with some of the most widely used early warning scores:¹² Modified Early Warning System (MEWS),¹³ Standardized Early Warning Scoring System (SEWS),¹⁴ Global Modified Early Warning Score (GMEWS),¹⁵ Worthing physiologic scoring system,¹⁶ National Early Warning Score (NEWS),¹⁷ and VitaPAC Early Warning Score (ViEWS).¹⁸ Published thresholds for these scores were used to create rule triggers in the data. Once a trigger was created to calculate the number of false positives and true positives, all subsequent triggers were ignored until the end of the episode or until 24 hours elapsed. We calculated triggers in a rolling fashion throughout the episodes of care. The EWS score was updated every time a new parameter was entered into the analytical electronic health record, and the most recent value for each was used to calculate the score. SAS statistical software was used for calculation of scores and identification of outcomes.

For our analysis, events were treated as dependent variables, and triggers were independent variables. We calculated the score for each EWS to the minute level throughout our retrospective database. If the score for a specific EWS was higher than the published/recommended threshold for that EWS, an alert was considered to have been issued, and the patient was followed for 24 hours. If the patient had an event in the subsequent 24 hours, or 1 hour before (1-hour look-back), the alert was considered a true positive; if not, a false positive. Events that were not preceded by an alert were false negatives, and 24-hour intervals without either an alert or an event were considered true negatives. This simulation exercise was performed for each EWS in both subcohorts (medical and surgical). Clusters of RRS calls followed by transfers to the ICU within 3 hours were considered as a single adverse event (RRS calls, as it was the first event to occur) to avoid double counting. We have described how well this simulation methodology,⁸ correlates with results from prospective studies.¹⁹

Statistical Analysis

To calculate whether results were statistically significant for subgroups, a jackknife method of calculating variance²⁰ was used. The jackknife method calculates variance by repeating the calculations of the statistic leaving out 1 sample at a time. In our case, we repeated the calculation of sensitivity and PPV leaving out 1 patient at a time. Once the simulation method had been run and the false/true positives/negatives had been assigned, calculation of each metric (PPV and sensitivity) was repeated for n subsamples, each leaving out 1 patient. The variance was calculated and 2 Student t tests were performed for each EWS: 1 for PPV and another for sensitivity. SAS statistical software v 9.3 was used for the simulation analysis; R statistical software v 3.0.2 (The R Foundation, Vienna, Austria) was used for the calculation of the statistical significance of results. A univariable analysis was also performed to assess the sensitivity and PPVs for the published thresholds of the most common variables in each EWS: respiratory rate, systolic blood pressure, heart rate, temperature, and mental status as measured by the modified Richmond Agitation Sedation Score.²¹

The initial cohort included 60,020 hospitalizations, of which the following were excluded: 2751 because of a lack of appropriate research authorization; 6433 because the patients were younger than 18 years; 2129 as psychiatric admissions; 284 as rehabilitation admissions; 872 as research purposes-only admissions; and 1185 because the patient was never in a general care bed (eg, they were either admitted directly to the ICU, or they were admitted for an outpatient surgical procedure and spent time in the postanesthesia care unit).

Table 1 summarizes patient and trigger characteristics, overall and by subgroup. The final cohort included 75,240 total episodes in 46,366 hospitalizations, from 34,898 unique patients, of which 48.7% were male. There were 23,831 medical and 22,535 surgical hospitalizations. Median length of episode was 2 days both for medical and surgical patients. Median length of stay was 3 days, both for medical and for surgical patients.

We hypothesized that EWS may perform differently when applied to medical rather than surgical patients. Studies had not analyzed this in a time-dependent manner,^24-26 which limited the applicability of the results.⁸

All analyzed scores performed better in medical patients than in surgical patients (Figure). This could reflect a behavioral difference by the teams on surgical and medical floors in the decision to activate the RRS, or a bias of the clinicians who designed the scores (mostly nonsurgeons). The difference could also mean that physiological deteriorations are intrinsically different in patients who have undergone anesthesia and surgery. For example, in surgical patients, a bleeding episode is more likely to be the cause of their physiological deterioration, or the lingering effects of anesthesia could mask underlying deterioration. Such patients would benefit from scores where variables such as heart rate, blood pressure, or hemoglobin had more influence.

When comparing the different scores, it was much easier for a patient to meet the alerting score with the Worthing score than with GMEWS. In the Worthing score, a respiratory rate greater than 22 breaths per minute, or a systolic blood pressure less than 100 mm Hg, already meet alerting criteria. Similar vital signs result in 0 and 1 points (respectively) in GMEWS, far from its alerting score of 5. This reflects the intrinsic tradeoff of EWS: as the threshold for considering a patient “at risk” drops, not only does the number of true positives (and the sensitivity) increase, but also the number of false positives, thus lowering the PPV.

However, none of the scores analyzed were considered to perform well based on their PPV and sensitivity, particularly in the surgical subpopulation. Focusing on another metric, the area under the receiver operator curve can give misleadingly optimistic results.^24,27 However, the extremely low prevalence of acute physiological deterioration can produce low PPVs even when specificity seems acceptable, which is why it is important to evaluate PPV directly.²⁸

To use EWS effectively to activate RRS, they need to be combined with clinical judgment to avoid high levels of false alerts, particularly in surgical patients. It has been reported that RRS is activated only 30% of the time a patient meets RRS calling criteria.²⁹ While there may be cultural characteristics inhibiting the decision to call,³⁰ our study hints at another explanation: if RRS was activated every time a patient met calling criteria based on the scores analyzed, the number of RRS calls would be very high and difficult to manage. So health providers may be doing the right thing when “filtering” RRS calls and not applying the criteria strictly, but in conjunction with clinical judgment.

A limitation of any study like this is how to define “acute physiological deterioration.” We defined an event as recognized episodes of acute physiological deterioration that are signaled by escalations of care (eg, RRS, resuscitation calls, or transfers to an ICU) or unexpected death. By definition, our calculated PPV is affected by clinicians’ recognition of clinical deteriorations. This definition, common in the literature, has the limitation of potentially underestimating EWS’ performance by missing some events that are resolved by the primary care team without an escalation of care. However, we believe our interpretation is not unreasonable since the purpose of EWS is to trigger escalations of care in a timely fashion. Prospective studies could define an event in a way that is less affected by the clinicians’ judgment.

Regarding patient demographics, age was similar between the 2 groups (average, 58.2 years for medical vs 58.9 years for surgical), and there was only a small difference in gender ratios (45.1% male in the medical vs 51.4% in the surgical group). These differences are unlikely to have affected the results significantly, but unknown differences in demographics or other patient characteristics between groups may account for differences in score performance between surgical and medical patients.

Several of the EWS analyzed had overlapping trigger criteria with our own RRS activation criteria (although as single-parameter triggers and not as aggregate). To test how these potential biases could affect our results, we performed a post hoc sensitivity analysis eliminating calls to the RRS as an outcome (so using the alternative outcome of unexpected transfers to the ICU and resuscitation calls). The results are similar to those of our main analysis, with all analyzed scores having lower sensitivity and PPV in surgical hospitalizations when compared to medical hospitalizations.

Our study suggests that, to optimize detection of physiological deterioration events, EWS should try to take into account different patient types, with the most basic distinction being surgical vs medical. This tailoring will make EWS more complex, and less suited for paper-based calculation, but new electronic health records are increasingly able to incorporate decision support, and some EWS have been developed for electronic calculation only. Of particular interest in this regard is the score developed by Escobar et al,³¹ which groups patients into categories according to the reason for admission, and calculates a different subscore based on that category. While the score by Escobar et al. does not split patients based on medical or surgical status, a more general interpretation of our results suggests that a score may be more accurate if it classifies patients into subgroups with different subscores. This seems to be confirmed by the fact that the score by Escobar et al performs better than MEWS.²⁸ Unfortunately, the paper describing it does not provide enough detail to use it in our database.

A recent systematic review showed increasing evidence that RRS may be effective in reducing CRAs occurring in a non-ICU setting and, more important, overall inhospital mortality.³² While differing implementation strategies (eg, different length of the educational effort, changes in the frequency of vital signs monitoring) can impact the success of such an initiative, it has been speculated that the afferent limb (which often includes an EWS) might be the most critical part of the system.³³ Our results show that the most widely used EWS perform significantly worse on surgical patients, and suggest that a way to improve the accuracy of EWS would be to tailor the risk calculation to different patient subgroups (eg, medical and surgical patients). Plausible next steps would be to demonstrate that tailoring risk calculation to medical and surgical patients separately can improve risk predictions and accuracy of EWS.

Disclosure

The authors report no financial conflicts of interest.

Patients typically show signs and symptoms of deterioration hours to days prior to cardiorespiratory arrest.^1,2 The rate of inhospital cardiorespiratory arrest (CRA) requiring cardiopulmonary resuscitation is estimated to be 0.174 per bed per year in the United States.³ After CRA, survival to discharge is estimated to be as low as 18%.^3,4 Efforts to predict and prevent arrest could prove beneficial.^1,2

Rapid response systems (RRS) have been proposed as a means of identifying clinical deterioration and facilitating a timely response. These systems were designed to bring clinicians with critical care expertise to the bedside to prevent unnecessary deaths. They typically include an afferent limb (detects deteriorating patients), an efferent limb (responds to calls and acts to avoid further deterioration), and administrative and data analysis limbs.^5,6 Automatic provision of recommendations and computer-based systems are desirable components of the afferent limb of the detection system.⁶ Both are independent predictors of improved clinical practices for clinical decision support systems.⁷ However, the existing early warning scores (EWS) may not be ready for automation due to low positive predictive values (PPV) and sensitivities.⁸

It is possible that the low discriminatory accuracy of the published EWS may be secondary to the use of aggregate patient populations for derivation of scores. We hypothesized that these EWS perform differently in medical and in surgical subpopulations. Also, the EWS need to be tested in a time-dependent manner to serve as a realistic clinical support tool for hospitalized patients.

STUDY AIM

The aim of this study was to evaluate the differential performance of widely used EWS in medical vs surgical patients.

Site

The study was conducted in an academic center with 2 hospitals in Southeastern Minnesota totaling approximately 1500 general care nonintensive care unit (ICU) beds. The Mayo Clinic Institutional Review Board approved the research proposal.

Subjects

Our retrospective cohort was comprised of all adult inpatients discharged from 2 academic hospitals between January 1, 2011 and December 31, 2011 who spent any time in a general care (non-ICU) unit. We excluded patients younger than 18 years, psychiatric or rehabilitation inpatients, those without research authorization, and patients admitted for research purposes.

Study patients were divided into medical and surgical cohorts. Hospitalizations were considered surgical if patients had surgery at any time during their hospital stay according to billing data. A trigger was an instance in which a patient met the conditions of a specific rule (score/vital sign exceeded the published/defined threshold).

A resuscitation call was defined as a call for cardiopulmonary resuscitation when a patient has a CRA.

An event was an occurrence of 1 of the following in a general care setting: unplanned transfer to the ICU, resuscitation call, or RRS activation.

The RRS activation criteria consisted of an “acute and persistent change” in any 1 or more of the following: oxygen saturations less than 90%, heart rate less than 40 or greater than 130 beats/minute, systolic blood pressure less than 90 mm Hg, or respiratory rate less than 10 or greater than 28 breaths/minute. The RRS activation requires health provider action; they are not electronically generated. Nurses and physicians may also activate the RRS if they are concerned about a patient, even if calling criteria are not met. This is in contrast to the EWS analyzed, which are aggregate composites of multiple parameters. However, whether or not a derangement in vital signs is considered an “acute and persistent change” still involves clinical judgment. Any movement from a general care bed to an ICU bed, or from a general care bed to a procedure area, and from there to an ICU, was considered unplanned. Transfers to the ICU directly from the emergency department or operating room (OR) were not considered as an unplanned transfer and were not included in the analyses.

Coverage time was the period observed for events after a rule was triggered. In this analysis, a coverage time of 24 hours was considered, with a 1-hour look-back. A trigger was counted as a true positive if an event occurred during the following 24 hours. The 1-hour look-back was included to take into account the nursing clinical process of prioritizing a call to the RRS followed by documentation of the altered vital signs that prompted the call.

An episode was the continuous time on the general care floor within a hospitalization, excluding times when a patient was in the OR or ICU. For example, if a patient was admitted to a general bed on a surgery floor, subsequently went to the OR, and then returned to the surgery floor, the 2 episodes were considered separate: the time on the floor before surgery, and the time on the floor after surgery.

Assessment of implementation of RRS in our hospitals showed a significant drop in the failure-to-rescue rate (issues considered related to delay or failure to identify or intervene appropriately when a patient was deteriorating, as identified through mortality review) and a decrease in non-ICU mortality.^9,10 This suggests that our current process captures many of the relevant episodes of acute deterioration when a rapid response team is needed and supports using RRS activation as outcomes.

Data Sources

We developed a time-stamped longitudinal database of patient data from the electronic health record, including vital signs, laboratory test results, demographics (age, sex), administrative data (including length of stay), comorbidities, resuscitation code status, location in hospital, and at the minute level throughout each patient’s hospital stay. Physiologically impossible values (eg, blood pressures of 1200 mm Hg) were considered entered in error and eliminated from the database. Time spent in the OR or ICU was excluded because RRS activation would not be applied in these already highly monitored areas. SAS Statistical software (SAS Institute Inc. Cary, North Carolina) was used for database creation.

We applied the current RRS calling criteria in our institution and calculated the Kirkland score,¹¹ along with some of the most widely used early warning scores:¹² Modified Early Warning System (MEWS),¹³ Standardized Early Warning Scoring System (SEWS),¹⁴ Global Modified Early Warning Score (GMEWS),¹⁵ Worthing physiologic scoring system,¹⁶ National Early Warning Score (NEWS),¹⁷ and VitaPAC Early Warning Score (ViEWS).¹⁸ Published thresholds for these scores were used to create rule triggers in the data. Once a trigger was created to calculate the number of false positives and true positives, all subsequent triggers were ignored until the end of the episode or until 24 hours elapsed. We calculated triggers in a rolling fashion throughout the episodes of care. The EWS score was updated every time a new parameter was entered into the analytical electronic health record, and the most recent value for each was used to calculate the score. SAS statistical software was used for calculation of scores and identification of outcomes.

For our analysis, events were treated as dependent variables, and triggers were independent variables. We calculated the score for each EWS to the minute level throughout our retrospective database. If the score for a specific EWS was higher than the published/recommended threshold for that EWS, an alert was considered to have been issued, and the patient was followed for 24 hours. If the patient had an event in the subsequent 24 hours, or 1 hour before (1-hour look-back), the alert was considered a true positive; if not, a false positive. Events that were not preceded by an alert were false negatives, and 24-hour intervals without either an alert or an event were considered true negatives. This simulation exercise was performed for each EWS in both subcohorts (medical and surgical). Clusters of RRS calls followed by transfers to the ICU within 3 hours were considered as a single adverse event (RRS calls, as it was the first event to occur) to avoid double counting. We have described how well this simulation methodology,⁸ correlates with results from prospective studies.¹⁹

Statistical Analysis

To calculate whether results were statistically significant for subgroups, a jackknife method of calculating variance²⁰ was used. The jackknife method calculates variance by repeating the calculations of the statistic leaving out 1 sample at a time. In our case, we repeated the calculation of sensitivity and PPV leaving out 1 patient at a time. Once the simulation method had been run and the false/true positives/negatives had been assigned, calculation of each metric (PPV and sensitivity) was repeated for n subsamples, each leaving out 1 patient. The variance was calculated and 2 Student t tests were performed for each EWS: 1 for PPV and another for sensitivity. SAS statistical software v 9.3 was used for the simulation analysis; R statistical software v 3.0.2 (The R Foundation, Vienna, Austria) was used for the calculation of the statistical significance of results. A univariable analysis was also performed to assess the sensitivity and PPVs for the published thresholds of the most common variables in each EWS: respiratory rate, systolic blood pressure, heart rate, temperature, and mental status as measured by the modified Richmond Agitation Sedation Score.²¹

The initial cohort included 60,020 hospitalizations, of which the following were excluded: 2751 because of a lack of appropriate research authorization; 6433 because the patients were younger than 18 years; 2129 as psychiatric admissions; 284 as rehabilitation admissions; 872 as research purposes-only admissions; and 1185 because the patient was never in a general care bed (eg, they were either admitted directly to the ICU, or they were admitted for an outpatient surgical procedure and spent time in the postanesthesia care unit).

Table 1 summarizes patient and trigger characteristics, overall and by subgroup. The final cohort included 75,240 total episodes in 46,366 hospitalizations, from 34,898 unique patients, of which 48.7% were male. There were 23,831 medical and 22,535 surgical hospitalizations. Median length of episode was 2 days both for medical and surgical patients. Median length of stay was 3 days, both for medical and for surgical patients.

We hypothesized that EWS may perform differently when applied to medical rather than surgical patients. Studies had not analyzed this in a time-dependent manner,^24-26 which limited the applicability of the results.⁸

All analyzed scores performed better in medical patients than in surgical patients (Figure). This could reflect a behavioral difference by the teams on surgical and medical floors in the decision to activate the RRS, or a bias of the clinicians who designed the scores (mostly nonsurgeons). The difference could also mean that physiological deteriorations are intrinsically different in patients who have undergone anesthesia and surgery. For example, in surgical patients, a bleeding episode is more likely to be the cause of their physiological deterioration, or the lingering effects of anesthesia could mask underlying deterioration. Such patients would benefit from scores where variables such as heart rate, blood pressure, or hemoglobin had more influence.

When comparing the different scores, it was much easier for a patient to meet the alerting score with the Worthing score than with GMEWS. In the Worthing score, a respiratory rate greater than 22 breaths per minute, or a systolic blood pressure less than 100 mm Hg, already meet alerting criteria. Similar vital signs result in 0 and 1 points (respectively) in GMEWS, far from its alerting score of 5. This reflects the intrinsic tradeoff of EWS: as the threshold for considering a patient “at risk” drops, not only does the number of true positives (and the sensitivity) increase, but also the number of false positives, thus lowering the PPV.

However, none of the scores analyzed were considered to perform well based on their PPV and sensitivity, particularly in the surgical subpopulation. Focusing on another metric, the area under the receiver operator curve can give misleadingly optimistic results.^24,27 However, the extremely low prevalence of acute physiological deterioration can produce low PPVs even when specificity seems acceptable, which is why it is important to evaluate PPV directly.²⁸

To use EWS effectively to activate RRS, they need to be combined with clinical judgment to avoid high levels of false alerts, particularly in surgical patients. It has been reported that RRS is activated only 30% of the time a patient meets RRS calling criteria.²⁹ While there may be cultural characteristics inhibiting the decision to call,³⁰ our study hints at another explanation: if RRS was activated every time a patient met calling criteria based on the scores analyzed, the number of RRS calls would be very high and difficult to manage. So health providers may be doing the right thing when “filtering” RRS calls and not applying the criteria strictly, but in conjunction with clinical judgment.

A limitation of any study like this is how to define “acute physiological deterioration.” We defined an event as recognized episodes of acute physiological deterioration that are signaled by escalations of care (eg, RRS, resuscitation calls, or transfers to an ICU) or unexpected death. By definition, our calculated PPV is affected by clinicians’ recognition of clinical deteriorations. This definition, common in the literature, has the limitation of potentially underestimating EWS’ performance by missing some events that are resolved by the primary care team without an escalation of care. However, we believe our interpretation is not unreasonable since the purpose of EWS is to trigger escalations of care in a timely fashion. Prospective studies could define an event in a way that is less affected by the clinicians’ judgment.

Regarding patient demographics, age was similar between the 2 groups (average, 58.2 years for medical vs 58.9 years for surgical), and there was only a small difference in gender ratios (45.1% male in the medical vs 51.4% in the surgical group). These differences are unlikely to have affected the results significantly, but unknown differences in demographics or other patient characteristics between groups may account for differences in score performance between surgical and medical patients.

Several of the EWS analyzed had overlapping trigger criteria with our own RRS activation criteria (although as single-parameter triggers and not as aggregate). To test how these potential biases could affect our results, we performed a post hoc sensitivity analysis eliminating calls to the RRS as an outcome (so using the alternative outcome of unexpected transfers to the ICU and resuscitation calls). The results are similar to those of our main analysis, with all analyzed scores having lower sensitivity and PPV in surgical hospitalizations when compared to medical hospitalizations.

Our study suggests that, to optimize detection of physiological deterioration events, EWS should try to take into account different patient types, with the most basic distinction being surgical vs medical. This tailoring will make EWS more complex, and less suited for paper-based calculation, but new electronic health records are increasingly able to incorporate decision support, and some EWS have been developed for electronic calculation only. Of particular interest in this regard is the score developed by Escobar et al,³¹ which groups patients into categories according to the reason for admission, and calculates a different subscore based on that category. While the score by Escobar et al. does not split patients based on medical or surgical status, a more general interpretation of our results suggests that a score may be more accurate if it classifies patients into subgroups with different subscores. This seems to be confirmed by the fact that the score by Escobar et al performs better than MEWS.²⁸ Unfortunately, the paper describing it does not provide enough detail to use it in our database.

A recent systematic review showed increasing evidence that RRS may be effective in reducing CRAs occurring in a non-ICU setting and, more important, overall inhospital mortality.³² While differing implementation strategies (eg, different length of the educational effort, changes in the frequency of vital signs monitoring) can impact the success of such an initiative, it has been speculated that the afferent limb (which often includes an EWS) might be the most critical part of the system.³³ Our results show that the most widely used EWS perform significantly worse on surgical patients, and suggest that a way to improve the accuracy of EWS would be to tailor the risk calculation to different patient subgroups (eg, medical and surgical patients). Plausible next steps would be to demonstrate that tailoring risk calculation to medical and surgical patients separately can improve risk predictions and accuracy of EWS.

Disclosure

The authors report no financial conflicts of interest.

High-dose immunosuppressive therapy and autologous hematopoietic cell transplant (HDIT/HCT) has had “highly promising” results for patients with relapsing-remitting multiple sclerosis (MS), according to researchers from the HALT-MS trial. In the 5-year study, 69% of 24 participants survived without progression of disability, relapse, or new brain lesions, despite not taking MS medications.

Findings published at the 3-year mark were encouraging. The event-free survival rate was 78%. The extended findings suggest that “onetime treatment with HDIT/HCT may be substantially more effective than long-term treatment with the best available medications” for these patients, said NIAID Director Anthony Fauci, MD.

The treatment “resets” the immune system, the researchers say. First, doctors collect the patient’s blood-forming stem cells, then give the patient chemotherapy to deplete the immune system. Finally, the doctors return the patient’s stem cells to rebuild the immune system.

Adverse events were consistent with those routinely observed after HDIT/HCT. Adverse effects recorded at 4 and 5 years were not related to the transplant and were not considered severe. Three patients died, but their deaths were not related to the study treatments.

Five years later, most trial participants remained in remission and stabilized, and some showed improvements, such as recovering mobility.

High-dose immunosuppressive therapy and autologous hematopoietic cell transplant (HDIT/HCT) has had “highly promising” results for patients with relapsing-remitting multiple sclerosis (MS), according to researchers from the HALT-MS trial. In the 5-year study, 69% of 24 participants survived without progression of disability, relapse, or new brain lesions, despite not taking MS medications.

Findings published at the 3-year mark were encouraging. The event-free survival rate was 78%. The extended findings suggest that “onetime treatment with HDIT/HCT may be substantially more effective than long-term treatment with the best available medications” for these patients, said NIAID Director Anthony Fauci, MD.

The treatment “resets” the immune system, the researchers say. First, doctors collect the patient’s blood-forming stem cells, then give the patient chemotherapy to deplete the immune system. Finally, the doctors return the patient’s stem cells to rebuild the immune system.

Adverse events were consistent with those routinely observed after HDIT/HCT. Adverse effects recorded at 4 and 5 years were not related to the transplant and were not considered severe. Three patients died, but their deaths were not related to the study treatments.

Five years later, most trial participants remained in remission and stabilized, and some showed improvements, such as recovering mobility.

High-dose immunosuppressive therapy and autologous hematopoietic cell transplant (HDIT/HCT) has had “highly promising” results for patients with relapsing-remitting multiple sclerosis (MS), according to researchers from the HALT-MS trial. In the 5-year study, 69% of 24 participants survived without progression of disability, relapse, or new brain lesions, despite not taking MS medications.

Findings published at the 3-year mark were encouraging. The event-free survival rate was 78%. The extended findings suggest that “onetime treatment with HDIT/HCT may be substantially more effective than long-term treatment with the best available medications” for these patients, said NIAID Director Anthony Fauci, MD.

The treatment “resets” the immune system, the researchers say. First, doctors collect the patient’s blood-forming stem cells, then give the patient chemotherapy to deplete the immune system. Finally, the doctors return the patient’s stem cells to rebuild the immune system.

Adverse events were consistent with those routinely observed after HDIT/HCT. Adverse effects recorded at 4 and 5 years were not related to the transplant and were not considered severe. Three patients died, but their deaths were not related to the study treatments.

Five years later, most trial participants remained in remission and stabilized, and some showed improvements, such as recovering mobility.